Skip to content

Serverless Data Lake Framework 2.3.0

Compare
Choose a tag to compare
@cnfait cnfait released this 21 Apr 16:31
· 267 commits to main since this release

This is a new release for SDLF 2.x. See the release notes for SDLF 2.0.0 to learn about all the changes in this major version.

For users of SDLF 1.x, version 1 is still available on the master branch. Development of newer versions of SDLF (2.x) happens on branch main. The workshop still contains sections for version 1 as well.

What's Changed

  • parameterize team main repository prefix by @cnfait in #290
  • add ssm parameter for octagon-DataSchemas table by @cnfait in #291
  • get dataschemas table name from ssm in replicate lambda by @cnfait in #292
  • restrict sdlf-catalog lambda to its dedicated sqs queues by @cnfait in #293
  • use RetainExceptOnCreate deletion policy for KMS keys and aliases by @cnfait in #294
  • stageA/stageB: replace hardcoded suffix -a/-b with stage name by @cnfait in #295
  • Glue security config SSM by @cnfait in #296
  • add ssm parameter for Glue Crawler name by @cnfait in #297
  • single role (per stage) for all metadata steps by @cnfait in #299
  • legislators deploy.sh linting by @cnfait in #301
  • add flags for optional features to deploy.sh multi-account deployment by @kangsoon-dev in #289
  • add support for nested stacks in datasets template by @cnfait in #302
  • glue job deployer: job/connection Update permissions by @cnfait in #304
  • S3 Storage Lens permissions by @cnfait in #305
  • update ruff linter settings by @gwenika in #306
  • remove bogus SQS DeleteMessageBatch/SendMessageBatch permissions by @cnfait in #307
  • Enforce TLS on devops S3 access logs bucket by @cnfait in #313
  • Rework rMain CodePipeline by @cnfait in #314
  • Glue Job Deployer: use /service-role/ path for role by @cnfait in #315
  • Glue Job Deployer rework by @cnfait in #316
  • do not set MaxItems when paginating by @cnfait in #318
  • Glue Job Deployer: handle multiple Glue Job Connections, and their AZ by @cnfait in #319
  • Glue Job Deployer CodeBuild: use crossaccount role by @cnfait in #320
  • use proper DynamoDB JSON structure for items in sdlf-foundations replicate lambda by @cnfait in #321
  • use proper DynamoDB JSON structure for items in sdlf-foundations catalog lambda by @cnfait in #322

Thanks

We thank all the contributors/users for their work on this release, in particular @gwenika and @kangsoon-dev!

Full Changelog: 2.2.0...2.3.0