yield compressed data up to part threshold #745

Job	Run time
Run pre-commit hooks against all files	14s
Upload files to S3 bucket in development	1m 21s
Build and push testing docker images to the pytest ECR repository. (aws_glue_3, tests/Dockerfile....	2m 20s
Runs unit tests that are not dependent on aws-glue package resources	1m 12s
Deploy Snowflake resources	3m 36s
Build and push testing docker images to the pytest ECR repository. (aws_glue_4, tests/Dockerfile....	2m 33s
Run Pytest unit tests for AWS glue (aws_glue_3)	2m 7s
Run Pytest unit tests for AWS glue (aws_glue_4)	6m 15s
Deploys branch using sceptre	2m 5s
Cleanup non-main branch data before integration tests	43s
Triggers ETL workflow with S3 test files	56s
Deploys to staging of prod using sceptre	0s
Cleanup main branch staging data before integration tests	0s
Triggers staging workflow with production data	0s
	23m 22s

Provide feedback