v1.8.3

What's Changed

• Fix #712: missing catalog in one metadata gathering query by @benc-db in #714

Full Changelog: v1.8.2...v1.8.3

v1.8.2

What's Changed

• Reverting the decision to remove 'spark.sql.sources.partitionOverwriteMode = DYNAMIC' for insert_overwrite by @benc-db in #688
• Liquid clustering is now responsive to config changes with incremental models by @benc-db in #686
• Cancel running python jobs on Ctrl-C by @benc-db (thanks @gaoshihang for kicking this off!) in #693
• Change over to system catalog for metadata gathering to fix bug with renamed catalogs by @benc-db in #692
• Fix insert_overwrite replacement when changing from a different partition strategy by @benc-db in #697
• Feature/issue640 refresh spa token no longer writes error log by (thanks @thijs-nijhuis!) in #700
• Migrate remaining unit tests to pytest by @benc-db in #701
• Change default redirect_url and scopes of client dbt-databricks by @jackyhu-db in #704
• Fix serialization issue detected by dbt Labs by @benc-db in #708

New Contributors

• @jackyhu-db made their first contribution in #704

Full Changelog: v1.8.1...v1.8.2

v1.8.1

What's New?

Features

• Support Liquid Clustering for python models (663)

Fixes

• Rerunning seed with external location + persist_doc now more resilient (662)
• Fix issue with running while a refresh is in progress with MV/ST (674)
• Fix issue with running a refresh with MV/ST that need names to be escaped (674)

Under the Hood

• Delay loading of agate library to improve startup (thanks @dwreeves for getting this started!) (661)
• Updating to dbt-adapters~=1.2.0 (683)

Full Changelog: v1.8.0...v1.8.1

v1.8.0

This release carries a substantial structural change as it is the first release after moving to the new 'decoupled' dbt architecture; while today we retain a dependence on dbt-core so that users do not need to install/specify versions for both libraries, we have moved to depending on a shared abstraction layer between the adapter and dbt-core. As a result, we no longer need to match our feature version to that of dbt-core, and are free to adopt semantic versioning. No more releasing significant features like 'compute-per-model` as a patch version!

This release also brings improvements to the declaration and operation of Materialized Views and Streaming Tables, including the ability to schedule automatic refreshes.

A new feature introduced in this release is support for tags. To distinguish from dbt tags, which are metadata that is often used for selecting models in a dbt operation, these tags are named as databricks_tags in the model configuration.

Big thanks to dbt Labs for significant help during the development and testing of this release.

What's Changed

Features

• Support on_config_change for materialized views, expand the supported config options (536)
• Support on_config_change for streaming tables, expand the supported config options (569)
• Support insert overwrite on SQL Warehouses (623)
• Support Databricks tags at model level for tables/views/incrementals (631)

Under the Hood

• Upgrade databricks-sql-connector to 3.1.0 (593)
• Migrate to decoupled architecture (596)
• Finish migrating integration tests (623)
• Streamline the process of determining materialization types (655)
• Improve catalog performance by getting column description from project for UC (658)

v1.7.16

It was brought to my attention that 1.7.15 didn't fix what it was intended to; this version should actually fix the issue where instead of getting a meaningful exception message when a connection could be created, you would get the message 'conn is referenced before being initialized'.

What's Changed

• Fix issue with conn not initialized by @benc-db in #671

Full Changelog: v1.7.15...v1.7.16

v1.7.15

What's Changed

• Backport fix to log events by @benc-db in #666

Full Changelog: v1.7.14...v1.7.15

v1.8.0rc1

In concert with dbt Labs, we've decided the next release will be 1.8.0 after all. This release pulls everything that was in the 2.0.latest branch into 1.8.latest.

What's Changed

• Support for tags by @benc-db in #627
• Metadata gathering improvements by @benc-db in #655 and in #658

Everything from the 1.7.latest branch that had not yet made it to 1.8.latest

Full Changelog: v1.8.0b2...v1.8.0rc1

v1.7.14

Note: user configurable OAuth scopes currently only work for AWS, as the Databricks SDK does not let us override them for Azure. I'm working with the owning team to get that addressed.

What's Changed

• FIX #582: User Configurable OAuth Scopes in profiles.yml by @stevenayers in #641
• Fixing idle issue and improving debug logging by @benc-db in #648

New Contributors

• @stevenayers made their first contribution in #641

Full Changelog: v1.7.13...v1.7.14

v1.7.13

What's Changed

• Up default socket timeout to 600 by @benc-db in #625
• Expose oauth redirect url for user config by @benc-db (thanks to the users that pushed me to get this done) in #635
• Support tblproperties (weakly) in python models by @benc-db in #633

Notes

• v1.7.12 had to be yanked after I found another piece of oauth code that needed to be updated
• oauth_redirect_url in your profile can now be used to override the redirect url (to solve the issue where we assume port 8020 vs 8050)
• tblproperties can now be applied to python models, but they are applied as an ALTER. This means they can be used to annotate your table, but are unlikely to work if you're trying to enable some sophisticated Databricks use case with them. I tried a number of ways to get them applied at table creation, none of which worked; hopefully at some point in the future we can get this fully working.

Full Changelog: v1.7.11...v1.7.13

v1.7.11

What's Changed

• When making catalog, don't populate column comments from Databricks (too slow) by @benc-db in #618

Full Changelog: v1.7.10...v1.7.11