DataHub v0.8.22

Disclaimers!

• Ingesting Chart Inputs was broken in a PR that got into this release. This will be fixed in v0.8.23. If you plan to ingest Charts / Dashboards, we recommend skipping this version and upgrading to v0.8.23 directly.

Release Highlights:

• Support for mapping DBT meta properties of a dataset to metadata operations, such as add_owner, add_term, add_tag etc.
• Java REST emitter library to programmatically generate metadata events from Java-based clients such as from Spark jobs.
• Data freshness indication via Last Updated Timestamp.
• Improvements to data profiling performance and lineage extraction

What's Changed

• feat(snowflake-usage): Generate email address if not exists by @treff7es in #3791
• feat(java datahub-client): add Java REST emitter by @MugdhaHardikar-GSLab in #3781
• fix(docker): Fix path to elastic definition in dev docker compose by @MikeSchlosser16 in #3808
• feat(nocode): Add get entities v2 endpoint that can get without snapshot by @dexter-mh-lee in #3738
• docs(modeling): Add a link to MXE page inside the Metadata Modeling page by @pramodbiligiri in #3765
• docs(fix): fix broken reference by @RyanHolstien in #3814
• feat(java-emitter): improvements to builder API-s, moving spark-linea… by @swaroopjagadish in #3819
• fix(ingestion): Make url an optional field of the DefaultConfig for business glossary by @rslanka in #3817
• fix(ingest): Handle string redshift type by @treff7es in #3811
• feat(gms): add schema registry support for tls in gms by @MikeSchlosser16 in #3804
• Add table, changed formatting and wording by @dannylee8 in #3802
• feat(mae/mcl): Make ingestAspect produce both MCLs and MAEs by @dexter-mh-lee in #3737
• docs(confluent): Add new topic names by @anshbansal in #3825
• (feat)(glossary): Increase number of autocomplete results shown to 25 by @aditya-radhakrishnan in #3821
• feat(sql-parser): Replacing sqlmetadata sql parser lib with sqlineage parser lib by @treff7es in #3806
• feat(profiler): using approximate queries for profiling by @treff7es in #3752
• docs: improve docs for kafka configuration by @abiwill in #3828
• test(fixEbeanEntityServiceTest): fix bug on verification for EbeanEntityService by @RyanHolstien in #3829
• fix(ingest): ignore custom connectors for Glue ingestion by @kevinhu in #3805
• fix(java-emitter): check for null callback by @swaroopjagadish in #3830
• feat(dbt-meta): add support for dbt meta mapping by @swaroopjagadish in #3832
• fix(ingestion): Fix the datetime parsing issue in the metabase source. by @rslanka in #3831
• feat(removeGMA): remove all dependencies on gma libraries by @RyanHolstien in #3835
• perf(ingest): changes to improve ingest performance a bit by @anshbansal in #3837
• fix(azure AD): fix problem with missing key causing failures in ingestion by @anshbansal in #3824
• docs: fix typo by @anshbansal in #3848
• docs(cli): fix wrong heading, add link to release notes by @anshbansal in #3700
• feat(ci): split metadata-ingestion ci to streamline build by @swaroopjagadish in #3854
• fix(dbt): fix warning due to struct type not being mapped by @anshbansal in #3846
• fix(ingest): bigquery-usage - fix remove_extras to remove all partitions by @gfalcone in #3842
• fix(ingestion): handle database=None for dbt ingestion by @iasoon in #3851
• feat(ingest): last updated - show last updated for sql usage sources by @aditya-radhakrishnan in #3845
• feat(lineage): allow for expanding of lineage node titles in the lineage explorer by @gabe-lyons in #3856

New Contributors

• @MikeSchlosser16 made their first contribution in #3808
• @pramodbiligiri made their first contribution in #3765
• @aditya-radhakrishnan made their first contribution in #3821
• @abiwill made their first contribution in #3828
• @gfalcone made their first contribution in #3842
• @iasoon made their first contribution in #3851

Full Changelog: v0.8.21...v0.8.22