Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​ServiceNow collector:​ Introduced a new collector that harvests metadata for scoped ​applications, tables, fields, views, and data fabric tables​ from a ​ServiceNow​​ instance.
• ​​Dremio collector:​ The API server (​--api-server​​) option is no longer required. When it is not provided, the collector does not harvest the API-based lineage and dataset information.
• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors:​​ Improved memory usage during the harvesting of column statistics, enhancing stability for large datasets.
• ​​Tableau collector: ​​
  • Added hostname mapping configuration support, enabling more flexible environment and network setups.
  • Added a new option, ​--tableau-exclude-unpublished-views​​, to exclude unpublished views (sheets and dashboards) from harvesting. ​

Bug fixes

• ​Snowflake collector: ​​Updated SQL parsing behavior to ensure parsing terminates correctly when a timeout occurs, and made the timeout value configurable to better handle complex or long-running queries.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• Salesforce collector: Added support for grouping and aggregation columns, expanding metadata coverage for analytical queries.
• Databricks collector: Added support for harvesting jobs from external workspaces, enabling visibility into cross-environment job metadata.
• Azure Data Factory collector: Now harvests SQL queries used in activities, providing richer lineage and operational context.
• Dremio collector:
  • Migrated to the Arrow Flight JDBC driver in place of the legacy Dremio driver, improving performance and compatibility.
  • Updated the --graphApiServer parameter to --api-server for clarity and consistency, and added a new --use-tls parameter to support secure connections.

Bug fixes

• Microsoft Fabric collector: Fixed an issue where some files were not being harvested, ensuring more complete metadata collection.
• QlikSense collector: Updated logic to exclude writing the API key in the TTL file.
• Fabric collector: Fixed a modeling issue where Lakehouse tables were incorrectly displayed as folders, ensuring proper representation of table structures.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​Microsoft SQL Server collector: The collector now harvests column comments from the MS_Description extended property, improving metadata detail and context.
• SQL Server Integration Services (SSIS) collector: Added support for lineage when Oracle is used as a source or target, expanding cross-platform lineage coverage.

Bug fixes

• ​​​Azure Data factory collector: Fixed an issue where lineage relationships were missing when a Snowflake credential had an assigned default role, ensuring complete lineage capture.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​OpenAPI collector: Added a new configuration option --environment-qualifier​​. When provided, this option ensures that ​harvested resources are unique​​ and distinct for that environment qualifier, improving multi-environment cataloging.
• ​​Power BI collector:​ The collector now harvests ​calculation groups​ and ​calculation items​​, expanding metadata coverage for Power BI semantic models.

Bug fixes

• ​​​dbt core and dbt cloud collectors:​​ Fixed an issue where tags were represented as strings instead of relationships. Tags are now properly modeled as relationships in the catalog graph.
• ​​Power BI collector:​​ Fixed an issue where descriptions in parameter definitions were incorrectly appended to parameter values.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• Power BI collector: Added support for Power BI–BigQuery column lineage.
• Databricks collector: Added support for harvesting metrics views, expanding metadata coverage within Databricks. A new optional parameter, --enable-metric-views, is now available to enable this feature.

Bug fixes

• ​​Informatica CDI collector: Fixed an issue that caused incomplete harvesting in large CDI instances by ensuring the collector continues harvesting beyond the Informatica CDI API’s default pagination limit.
• dbt core and dbt cloud collectors: Corrected tag representation so that tags are now modeled as resources rather than strings, improving metadata consistency and usability.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• OpenAPI collector: Added handling for schema references and array item references, improving completeness of OpenAPI metadata harvesting.
• Power BI collector: Now harvests endorsement details for reports, datasets, and dataflows, providing better visibility into trusted and certified content.
• Databricks collector: Added support for harvesting materialized views, expanding metadata coverage within Databricks.
• SQL Server collector: Now harvests database synonyms, improving understanding of object relationships and dependencies.
• Tableau collector: Updated logic to associate published data sources with the project that contains them, improving metadata organization and context.
• SSRS collector: Now harvests the external URL for linked reports, providing direct traceability to linked content.
• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Updated lineage logic to represent INSERT stored procedures as the triggering agent in lineage relationships, improving lineage accuracy.

Bug fixes

• ​​Informatica collector: Fixed an issue that limited harvesting to a maximum of 4,200 mapping tasks. The collector now processes all available mappings.
• Databricks collector: Fixed handling of non-alphanumeric characters in identifiers. Resolved issues with the creation of invalid identifiers.
• dbt core and dbt cloud collectors: Fixed a case mismatch issue when coining database IRIs for Snowflake accounts, ensuring consistent IRI generation.
• SSIS collector: Corrected package lookup logic to prioritize the Package GUID, using the Version GUID only when multiple versions share the same Package GUID.
• Talend collector: Fixed a classification issue where Talend Jobs were incorrectly identified as activities instead of agents.
• Snowflake collector: Implemented a workaround for a Snowflake JDBC driver defect that was triggered by unexpected datatypes, preventing runtime errors.
• SQL Server collector: The collector now harvests table size for tables in non-dbo schemas, ensuring complete table metadata coverage.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• Qlik Sense collector: Added lineage between Qlik Sense apps and datasets, providing greater visibility into data flow and dependencies.
• Marquez collector: Added column-level lineage, offering more granular insight into data transformations and relationships.

Bug fixes

• ​​Databricks collector: Fixed the creation of invalid agent IRIs, ensuring accurate and consistent identifier generation.
• Qlik Sense collector: Resolved a memory leak related to closing WebSocket connections, improving stability and performance.
• AWS Glue collector: Added handling for large object size properties, preventing errors during metadata harvesting for large datasets.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​​​Marquez collector:​ The collector now harvests job run history. A new optional parameter, ​--job-run-count​​, is now available to enable this feature.

Bug fixes

• ​​Tableau collector:​​ Fixed an issue where relationships between sheets and fields were not being harvested under certain conditions, ensuring more complete lineage capture.

​ ​ ​​ ​ ​ ​ ​​

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​​PostgresSQL collector:​ Now harvests ​all view definitions​​, regardless of user select privileges, ensuring more complete metadata capture.
• ​​OpenAPI collector:​
  • Changed ​hasParameter​ to be a ​containment property​​, improving accuracy of relationships.
  • Updated titles for ​OpenAPI resources​​ to provide clearer labeling in the catalog.
• Databricks collector: ​
  • Added support for ​Volume resources​ and harvesting lineage across​ tables, jobs, and notebooks.​​
  • Improved performance when harvesting lineage, reducing runtime for large environments.
• ​​dbt Core and dbt Cloud collectors​: Now extract the ​host​ from the ​database-server​​ option when it is provided as a URL, ensuring correct connection details.
• Tableau collector:​​
  • Added support for ​JWT tokens​​ for authentication, expanding login options.
  • Added support for ​Databricks database servers​​ as a data source.
  • Now harvests ​combinedFields​​, improving completeness of metadata.
• ​​Marquez collector:​ Added support for ​AWS S3​​, enabling metadata harvesting for S3 datasets through Marquez.
• ​​Oracle collector:​ Now harvests ​Oracle roles​​, extending coverage of user and access metadata.

​ ​ ​​ ​ ​ ​ ​​

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​Databricks collector:​ Now harvests ​models​ registered in ​Unity Catalog​​, expanding coverage of Databricks assets.
• ​​Confluent Cloud and Confluent Platform collectors:​​ Added support for TLS connections to Confluent Kafka, improving security and compatibility.

​Bug fixes

• ​​​​Microsoft Fabric collector:​ Fixed issues with missing ​DirectLake column-level lineage​​ and corrected some improperly formed source IRIs for DirectQuery lineage.
• ​​Monte Carlo collector:​ Updated ​enum​ types from ​Monte Carlo GraphQL definitions​​ to fix an exception caused by unrecognized types.

​ ​ ​​ ​ ​ ​ ​​