Serverless Data Lake Framework 2.1.0

This is a new release for SDLF 2.x. See the release notes for SDLF 2.0.0 to learn about all the changes in this major version.

For users of SDLF 1.x, version 1 is still available on the master branch. Development of newer versions of SDLF (2.x) happens on branch main. The workshop still contains sections for version 1 as well.

SDLF 2.1 brings an example stage for data quality with Glue (thank you @mureddy19!), improvements to monitoring with S3 Storage Lens, Lake Formation tag support for teams, plus a number of smaller improvements and bug fixes.

What's Changed

• Fixed incorrectly named variable in the for loop which deletes legacy… by @waltmayfield in #243
• Transform data quality steps into a proper sdlf stage (sdlf-stage-dataquality) by @cnfait in #244
• Lake Formation tag (LF-Tag) support for sdlf pipelines by @cnfait in #247
• add DescribeStackEvents to CodePipeline role permissions (#245) by @cnfait in #248
• dedicated event bus per team in child accounts (#165) by @cnfait in #249
• rename pOrganizationName parameter to pOrg to be consistent across all SDLF by @cnfait in #250
• add kms encryption for codecommit repositories by @gwenika in #253
• Update GitHub actions to the latest actions version by @fleventy-5 in #254
• add on-demand github workflow testing sdlf deployment by @gwenika in #255
• make sure lf-tag is created before using it in permissions by @cnfait in #257
• create team repository in a separate stack from team codepipeline by @cnfait in #258
• add policies to retain KMS key aliases on update/delete by @gwenika in #259
• sdlf-monitoring rewrite, add storage lens by @cnfait in #260
• event rule to stop codepipeline first run (rMain only for now) by @cnfait in #261
• create sdlf repositories in a separate stack from sdlf-main codepipeline by @cnfait in #262
• create sdlf-main-{domain}-{team} repository with default content and branches by @cnfait in #263
• update user agent by @cnfait in #264
• update event rule to stop rDomain CodePipeline first run by @cnfait in #265

New Contributors

• @waltmayfield made their first contribution in #243
• @fleventy-5 made their first contribution in #254

Thanks

We thank all the contributors/users for their work on this release, in particular @fleventy-5, @gwenika, @mureddy19 and @waltmayfield!

Full Changelog: 2.0.1...2.1.0