Skip to content
upload-and-deploy

[ETL-616] Implement Great Expectations to run on parquet data (#139) #778

Sign in to view logs

[ETL-616] Implement Great Expectations to run on parquet data (#139)

[ETL-616] Implement Great Expectations to run on parquet data (#139) #778

Triggered via push September 13, 2024 08:02
@rxu17rxu17
pushed 30d1873
main
Status Success
Total duration 22m 42s
Artifacts –

upload-and-deploy.yaml

on: push
Run pre-commit hooks against all files
16s
Run pre-commit hooks against all files
Matrix: Build and push testing docker images to the pytest ECR repository.
Upload files to S3 bucket in development
1m 18s
Upload files to S3 bucket in development
Runs unit tests that are not dependent on aws-glue package resources
1m 17s
Runs unit tests that are not dependent on aws-glue package resources
Deploy Snowflake resources
3m 36s
Deploy Snowflake resources
Matrix: Run Pytest unit tests for AWS glue
Deploys branch using sceptre
2m 4s
Deploys branch using sceptre
Cleanup non-main branch data before integration tests
52s
Cleanup non-main branch data before integration tests
Triggers ETL workflow with S3 test files
50s
Triggers ETL workflow with S3 test files
Deploys to staging of prod using sceptre
3m 5s
Deploys to staging of prod using sceptre
Cleanup main branch staging data before integration tests
2m 34s
Cleanup main branch staging data before integration tests
Triggers staging workflow with production data
2m 48s
Triggers staging workflow with production data
Fit to window
Zoom out
Zoom in

Annotations

20 warnings
Run pre-commit hooks against all files
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, actions/cache@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run pre-commit hooks against all files
The `python-version` input is not set. The version of Python currently in `PATH` will be used.
Runs unit tests that are not dependent on aws-glue package resources
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Upload files to S3 bucket in development
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Build and push testing docker images to the pytest ECR repository. (aws_glue_3, tests/Dockerfile....
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: aws-actions/configure-aws-credentials@v2, aws-actions/amazon-ecr-login@v1, actions/checkout@v3, docker/setup-buildx-action@v2, docker/build-push-action@v4. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Build and push testing docker images to the pytest ECR repository. (aws_glue_3, tests/Dockerfile....
Your docker password is not masked. See https://github.com/aws-actions/amazon-ecr-login#docker-credentials for more information.
Build and push testing docker images to the pytest ECR repository. (aws_glue_4, tests/Dockerfile....
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: aws-actions/configure-aws-credentials@v2, aws-actions/amazon-ecr-login@v1, actions/checkout@v3, docker/setup-buildx-action@v2, docker/build-push-action@v4. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Build and push testing docker images to the pytest ECR repository. (aws_glue_4, tests/Dockerfile....
Your docker password is not masked. See https://github.com/aws-actions/amazon-ecr-login#docker-credentials for more information.
Deploy Snowflake resources
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Pytest unit tests for AWS glue (aws_glue_3)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: aws-actions/configure-aws-credentials@v2, actions/checkout@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Run Pytest unit tests for AWS glue (aws_glue_4)
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: aws-actions/configure-aws-credentials@v2, actions/checkout@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Deploys branch using sceptre
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2, gagoar/invoke-aws-lambda@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Deploys branch using sceptre
The following actions uses node12 which is deprecated and will be forced to run on node16: gagoar/invoke-aws-lambda@v3. For more info: https://github.blog/changelog/2023-06-13-github-actions-all-actions-will-run-on-node16-instead-of-node12-by-default/
Deploys branch using sceptre
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Deploys branch using sceptre
The `set-output` command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/
Cleanup non-main branch data before integration tests
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Triggers ETL workflow with S3 test files
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Deploys to staging of prod using sceptre
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Cleanup main branch staging data before integration tests
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.
Triggers staging workflow with production data
Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3, actions/setup-python@v4, aws-actions/configure-aws-credentials@v2. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/.