Skip to content

Commit

Permalink
docs: removed legace s3 sync
Browse files Browse the repository at this point in the history
  • Loading branch information
MBenediktF committed Sep 2, 2024
1 parent 8a159a6 commit 85f9f44
Show file tree
Hide file tree
Showing 2 changed files with 0 additions and 17 deletions.
9 changes: 0 additions & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -18,15 +18,6 @@ remove_mlflow_ui:
@docker rm mlflow_ui || true
@docker rmi mlflow_ui_s3 || true

download_datasets:
@aws s3 sync s3://${BUCKET_NAME}/datasets datasets --exact-timestamps

download_dataset:
@aws s3 sync s3://${BUCKET_NAME}/datasets/$(NAME) datasets/$(NAME) --exact-timestamps

upload_datasets:
@aws s3 sync datasets s3://${BUCKET_NAME}/datasets --exact-timestamps

start_prod_deployment_workflow:
curl -X POST \
-H "Accept: application/vnd.github+json" \
Expand Down
8 changes: 0 additions & 8 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,3 @@
- Deploy to target system
- Export the model
- Run tests with the compiles model and the implementation code

## How to manage raw datasets with aws cli (legacy)

- Datasets are stored in the folder `datasets`, which is not synced with github, but stored in s3 instead
- To download all existing datasets, use `make download_datasets`
- To download a specific dataset use `make download_dataset NAME=<dataset_name>`
- To upload a new dataset to s3, add it to the `datasets`folder and use `make upload_datasets`
- This feature is only for storing raw data. Procecced datasets are stored as artifacts and can be accessed using the mlflow ui

0 comments on commit 85f9f44

Please sign in to comment.