Releases

v0.0.10

Enhancements

  • Added apply_changes_from_snapshot support in silver layer PR
  • Added UI using databricks lakehouse app for onboarding/deploy commands PR
  • Added support for non-Delta as sinks(delta, kafka) PR
  • Added quarantine support in silver layer for data quality rules PR
  • Added support for table comments, column comments, and cluster_by PR
  • Added catalog support for sourceDetails and targetDetails PR
  • Added DBDemos for dlt-meta PR
  • Added YAML support for onboarding PR
  • Fixed issue cluster by not working with bronze append only table PR
  • Fixed issue view name containing period when using DPM PR
  • Fixed issue CLI onboarding overwrite option always set to True PR
  • Fixed issue Silver Lakeflow Declarative Pipeline not creating based on passed database PR
  • Fixed issue PyPI download stats display PR
  • Fixed issue Silver Data Quality not working PR
  • Fixed issue Removed DPM flag check inside dataflowpipeline PR
  • Fixed issue Updated dlt-meta demos into Delta Live Tables Notebook github PR

v0.0.9

Enhancements

  • Added apply_changes_from_snapshot api support in bronze layer: PR
  • Added dlt append_flow api support for silver layer: PR
  • Added support for file metadata columns for autoloader: PR
  • Added support for Bring your own custom transformation: Issue
  • Added support to Unify PyPI releases with GitHub OIDC: PR
  • Added demo for append_flow and file_metadata options: PR
  • Added Demo for silver fanout architecture: PR
  • Added hugo-theme-relearn themee: PR
  • Added unit tests to showcase silver layer fanout examples: PR
  • Added liquid cluster support: PR
  • Added support for UC Volume + Serverless support for CLI, Integration tests and Demos: PR
  • Added Chaining bronze/silver pipelines into single DLT: PR

Updates

  • Fixed issue for No such file or directory: ‘/demo’ :PR
  • Fixed issue DLT-META CLI onboard command issue for Azure: databricks.sdk.errors.platform.ResourceAlreadyExists :PR
  • Fixed issue Changed dbfs.create to mkdirs for CLI: PR
  • Fixed issue DLT-META CLI should use pypi lib instead of whl : PR
  • Fixed issue Onboarding with multiple partition columns errors out: PR

v0.0.8

Enhancements

  • Added dlt append_flow api support: PR
  • Added dlt append_flow api support for silver layer: PR
  • Added support for file metadata columns for autoloader: PR
  • Added support for Bring your own custom transformation: Issue
  • Added support to Unify PyPI releases with GitHub OIDC: PR
  • Added demo for append_flow and file_metadata options: PR
  • Added Demo for silver fanout architecture: PR
  • Added documentation in docs site for new features: PR
  • Added unit tests to showcase silver layer fanout examples: PR

Updates

  • Fixed issue for No such file or directory: ‘/demo’ :PR
  • Fixed issue DLT-META CLI onboard command issue for Azure: databricks.sdk.errors.platform.ResourceAlreadyExists :PR
  • Fixed issue Changed dbfs.create to mkdirs for CLI: PR
  • Fixed issue DLT-META CLI should use pypi lib instead of whl : PR

v0.0.7

Enhancements

1. Mismatched Keys: Update read_dlt_delta() with key “source_database” instead of “database” #33

2. Create dlt-meta cli documentation #45

  • Readme and docs to include above features

v0.0.6

Enhancements

1. Migrate to create_streaming_table api from create_streaming_live_table #37

Updates

  • Readme and docs to include above features
  • Added Data Quality support for silver layer
  • Updated existing demos to incorporate unity catalog and integration test framework
  • integration tests framework which can be used to launch demos

v0.0.5

New Features

1. Unity Catalog Support (#28)

2. Databricks Labs CLI Support (#28)

  • Added two commands for DLT-META
  • onboard: Captures all onboarding details from command line and launch onboarding job to your databricks workspace
  • deploy: Captures all Lakeflow Declarative Pipeline details from command line and launch Lakeflow Declarative Pipeline to your databricks workspace

Updates

  • Readme and docs to include above features
  • Updated existing demos to incorporate unity catalog and integration test framework
  • integration tests framework which can be used to launch demos

v0.0.4

Bug Fixes

v0.0.3

Bug Fixes

v0.0.2

New Features

  • Table properties support for bronze, quarantine and silver tables using create_streaming_live_table api call
  • Support for track history column using apply_changes api
  • Support for delta as source
  • Validation for bronze/silver onboarding

Bug Fixes

  • Input schema parsing issue in onboarding

Updates

  • Readme and docs to include above features

v.0.0.1

Release v.0.0.1