Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shift from s3fs to boto3 for consistency #34

Closed
metazool opened this issue Sep 12, 2024 · 1 comment
Closed

Shift from s3fs to boto3 for consistency #34

metazool opened this issue Sep 12, 2024 · 1 comment
Assignees
Labels
improvement a.k.a. "tech debt" - packaging or refactoring work for ease of future maintenance

Comments

@metazool
Copy link
Collaborator

s3fs was picked arbitrarily, the object_store_api project uses boto3 and I agree with @albags that it's better to have consistency across projects (both for interfaces and for environment configuration)

Hopefully a small set of changes that are helpful to make while working through use of DVC for setting up a simple pipeline for extracting embeddings from images held in s3 (for building an initial classifier) #32

@metazool metazool added the improvement a.k.a. "tech debt" - packaging or refactoring work for ease of future maintenance label Sep 12, 2024
@metazool metazool self-assigned this Sep 12, 2024
@metazool
Copy link
Collaborator Author

Closed in #35 (there's a way further to go with DVC, but this covered the object store python interface parts)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement a.k.a. "tech debt" - packaging or refactoring work for ease of future maintenance
Projects
None yet
Development

No branches or pull requests

1 participant