Serverless Data Lake Framework 2.3.0
This is a new release for SDLF 2.x. See the release notes for SDLF 2.0.0 to learn about all the changes in this major version.
For users of SDLF 1.x, version 1 is still available on the master branch. Development of newer versions of SDLF (2.x) happens on branch main. The workshop still contains sections for version 1 as well.
What's Changed
- parameterize team main repository prefix by @cnfait in #290
- add ssm parameter for octagon-DataSchemas table by @cnfait in #291
- get dataschemas table name from ssm in replicate lambda by @cnfait in #292
- restrict sdlf-catalog lambda to its dedicated sqs queues by @cnfait in #293
- use RetainExceptOnCreate deletion policy for KMS keys and aliases by @cnfait in #294
- stageA/stageB: replace hardcoded suffix -a/-b with stage name by @cnfait in #295
- Glue security config SSM by @cnfait in #296
- add ssm parameter for Glue Crawler name by @cnfait in #297
- single role (per stage) for all metadata steps by @cnfait in #299
- legislators deploy.sh linting by @cnfait in #301
- add flags for optional features to deploy.sh multi-account deployment by @kangsoon-dev in #289
- add support for nested stacks in datasets template by @cnfait in #302
- glue job deployer: job/connection Update permissions by @cnfait in #304
- S3 Storage Lens permissions by @cnfait in #305
- update ruff linter settings by @gwenika in #306
- remove bogus SQS DeleteMessageBatch/SendMessageBatch permissions by @cnfait in #307
- Enforce TLS on devops S3 access logs bucket by @cnfait in #313
- Rework rMain CodePipeline by @cnfait in #314
- Glue Job Deployer: use /service-role/ path for role by @cnfait in #315
- Glue Job Deployer rework by @cnfait in #316
- do not set MaxItems when paginating by @cnfait in #318
- Glue Job Deployer: handle multiple Glue Job Connections, and their AZ by @cnfait in #319
- Glue Job Deployer CodeBuild: use crossaccount role by @cnfait in #320
- use proper DynamoDB JSON structure for items in sdlf-foundations replicate lambda by @cnfait in #321
- use proper DynamoDB JSON structure for items in sdlf-foundations catalog lambda by @cnfait in #322
Thanks
We thank all the contributors/users for their work on this release, in particular @gwenika and @kangsoon-dev!
Full Changelog: 2.2.0...2.3.0