Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Core Data Migration #176

Draft
wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

Conversation

NachoEmbrace
Copy link
Contributor

  • Removes GRDB dependency and replaces it with CoreData.
  • Added protocols in EmbraceCommonInternal to define our data model (ie EmbraceSession, EmbraceSpan, etc).
    • This was necessary to stop manipulating the SessionRecord, SpanRecord, etc objects directly
    • These objects are now NSManagedObjects and can't be initialized without a CoreData stack
    • Using these protocols instead of the objects allows for easier testing.

Note: Still missing some tests and A LOT of manual testing.

Copy link

github-actions bot commented Feb 6, 2025

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

OpenSSF Scorecard

PackageVersionScoreDetails

Scanned Files

  • Package.resolved

Copy link

github-actions bot commented Feb 6, 2025

Warnings
⚠️ No CHANGELOG entry added.

Generated by 🚫 Danger Swift against 40ff859

Copy link
Contributor

@sergiothinks2 sergiothinks2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NachoEmbrace great work here. A lotttt of work. One thought, regarding thread safety, given CoreData requires careful handling of contexts. Do we have something that does explicit handling of background contexts vs main context?

Thinking something like this
`public class CoreDataContextManager {
let mainContext: NSManagedObjectContext
let backgroundContext: NSManagedObjectContext
private let logger: InternalLogger

init(persistentContainer: NSPersistentContainer, logger: InternalLogger) {
    self.logger = logger
    
    // Main context for UI operations
    mainContext = persistentContainer.viewContext
    mainContext.automaticallyMergesChangesFromParent = true
    
    // Background context for async operations
    backgroundContext = NSManagedObjectContext(concurrencyType: .privateQueueConcurrencyType)
    backgroundContext.parent = mainContext
}

func performBackgroundTask(_ block: @escaping (NSManagedObjectContext) -> Void) {
    backgroundContext.perform { [weak self] in
        guard let self = self else { return }
        block(self.backgroundContext)
        
        do {
            try self.backgroundContext.save()
            self.mainContext.performAndWait {
                try? self.mainContext.save()
            }
        } catch {
            self.logger.error("Failed to save context: \(error)")
        }
    }
}

}`

which I know would be a good amount of added work but it could be used in some way like

`public func cleanUpSpans(date: Date? = nil) {
contextManager.performBackgroundTask { context in
let request = SpanRecord.createFetchRequest()

    if let date = date {
        request.predicate = NSPredicate(format: "endTime != nil AND endTime < %@", date as NSDate)
    } else {
        request.predicate = NSPredicate(format: "endTime != nil")
    }
    
    let spans = try? context.fetch(request)
    spans?.forEach { context.delete($0) }
}

}`

@NachoEmbrace
Copy link
Contributor Author

@NachoEmbrace great work here. A lotttt of work. One thought, regarding thread safety, given CoreData requires careful handling of contexts. Do we have something that does explicit handling of background contexts vs main context?

I haven't found the need to use the mainContext at all. All operations are done in a single background context. This seems to work just fine.
I've only run into issues with threads / race conditions when running unit tests (since many of them are run in parallel).

I managed to make it stable by making all core data operations synchronous (which is not the best in terms of speed, to be fair); and also using a NSLock inside the CoreDataWrapper whenever the context is used.

NachoEmbrace and others added 8 commits February 13, 2025 10:49
# Conflicts:
#	Sources/EmbraceCore/Internal/Tracing/StorageSpanExporter.swift
#	Sources/EmbraceStorageInternal/Migration/Migrations/Migrations+Current.swift
#	Sources/EmbraceStorageInternal/Queries/SpanRecord+SessionQuery.swift
#	Sources/EmbraceStorageInternal/Records/SpanRecord.swift
#	Tests/EmbraceCoreTests/IntegrationTests/EmbraceOTelStorageIntegration/SpanStorageIntegrationTests.swift
#	Tests/EmbraceCoreTests/Internal/EmbraceSpanProcessor+StorageTests.swift
#	Tests/EmbraceStorageInternalTests/FetchMethods/EmbraceStorage+SpanForSessionRecordTests.swift
#	Tests/EmbraceStorageInternalTests/Migration/Migrations/Migrations+CurrentTests.swift
#	Tests/EmbraceStorageInternalTests/SpanRecordTests.swift
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants