Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​ServiceNow collector:​​
  • Added option (​​--skip-glide​​) to skip Glide table harvesting for more targeted metadata collection. ​
  • ​The collector now supports Platform Analytics, with extended capabilities for comprehensive analytics metadata harvesting.
  • ​Enhanced field metadata now includes column size and nullable properties, providing more detailed data profiling.
• ​Databricks collector:​ Added option (​--governance-metadata-collection) to harvest ​ Unity Catalog governance​​ metadata, enabling enhanced data governance visibility for Databricks environments.​
• ​​Microsoft Fabric collector:​​ Added support for Oracle table datasets, expanding metadata coverage for Fabric workspaces.​​

Bug fixes

• ​ ​​Power BI Service and Microsoft Fabric collectors:​​ Fixed an issue where large formulas in measures and calculated columns were not being cataloged, ensuring complete metadata capture for complex calculations. ​ ​ ​ ​​
• Azure Data Factory collector:​​ Fixed an issue where certain pipeline variable structures caused collection failures, improving collector stability.​Azure Data Factory collector:​​ Fixed an issue where certain pipeline variable structures caused collection failures, improving collector stability.​
  

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​AWS Glue collector​​: Support added for harvesting metadata from multiple AWS regions in a single run, improving efficiency for multi-region environments. The collector now also harvests table parameters (advanced properties), expanding metadata coverage.​​
• ​ServiceNow collector:​​ The collector now harvests Vault classification metadata, providing enhanced data governance visibility.

Bug fixes

• ​BigQuery collector: ​
  • Fixed acceptance of invalid credential formats, ensuring proper authentication validation.
  • Resolved an issue that caused crashes when the datasets parameter was not specified.
  • Fixed collection failures due to dependency conflicts.​
• ​​Databricks collector:​​ Fixed Service Principal authentication failures, ensuring reliable authentication.​​
• ​Dremio collector:​​ Fixed an issue where out-of-scope schemas were being collected during API calls, improving collection accuracy.​
• ​​Tableau collector​​: ​
  • Improved error messaging for API base URL configuration issues, making troubleshooting easier.
  • Fixed collection failures for tenants with deeply nested composite fields.​​
• ​Oracle collector: ​​ ​ ​ ​
  • Fixed missing default values when using DBA access. ​ ​ ​
  • Resolved collection failures on wrapped (encrypted) package bodies, ensuring complete metadata capture.​
• ​Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors:​​ Fixed errors when encountering Infinity float values in numeric columns, preventing data type errors.​

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

Bug fixes

• ​Microsoft Fabric collector:​​ Fixed an issue where Direct Lake tables displayed null schema names when the source schema was not explicitly defined, improving metadata accuracy.​Microsoft Fabric collector:​​ Fixed an issue where Direct Lake tables displayed null schema names when the source schema was not explicitly defined, improving metadata accuracy.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• ​Hive Metastore collector​​: New collector added for Hive Metastore, enabling direct metadata harvesting from the Hive Metastore database (PostgreSQL, Oracle, MySQL/MariaDB, or SQL Server) without requiring HiveServer2. The collector harvests databases, schemas, tables, columns, Hive-specific table properties, and basic view lineage. This provides a lightweight alternative for environments where HiveServer2 is unavailable or when only structural metadata is needed.

  This collector replaces the existing Hive and Hive Metastore collectors.

• ​​Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, Microsoft SQL Server, and SAP HANA collectors:​​ Added sample data preview capability, enabling the catalog to display sample of table data. When Sensitive Data Classification is enabled, sensitive columns are automatically masked in the preview, ensuring secure data visibility.​​

Bug fixes

• Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Reduced unnecessary warning messages during SQL lineage processing, improving log clarity.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• BigQuery collector:​​ The collector now harvests primary and foreign keys, expanding metadata coverage for BigQuery table relationships.

Bug fixes

• All collectors​​: Fixed an issue where boolean false values in YAML configuration files were not properly recognized.
• Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Resolved memory issues when processing columns with very large string values, improving collector stability.
• Tableau collector: ​​ Improved handling of recursive field queries, enhancing collection reliability for complex Tableau workbooks.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• Qlik Replicate collector: New collector added for Qlik Replicate, enabling metadata harvesting from Qlik's data replication platform.

Bug fixes

• Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Fixed column statistics metrics to ensure consistent data type handling across database sources.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• ​​Microsoft SQL Server collector:​​ The collector now collects usage metrics, providing insights into table and query access patterns.​
• BigQuery collector:​​ Support added for harvesting multiple BigQuery projects in a single run, improving efficiency for multi-project environments.

​

Bug fixes

• ​Qlik Talend Data Integration collector:​​ Resolved an issue that caused collector failures in certain configurations.​
• ServiceNow collector:​​ Improved metadata structure for tables and fields, ensuring better catalog integration.erviceNow collector:​​ Improved metadata structure for tables and fields, ensuring better catalog integration.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• ​​​Airflow collector: Support added for for Airflow 3.x, enabling metadata collection from the latest Airflow version.
• ServiceNow collector: The collector now harvests data interfaces, expanding metadata coverage.
• Snowflake collector: Added support for semantic views, providing metadata capture for Snowflake's business-friendly data representations.
• SQL Server Integration Services (SSIS) collector: Added lineage support for tables and SQL statements with variable references, improving lineage accuracy for dynamic SSIS workflows.

​

Bug fixes

• ​​Databricks collector: Fixed regex used for parsing metric views, resolving parsing errors for certain metric view configurations.
• Microsoft Fabric collector: Fixed an issue that occurred when property values were not present, improving collector stability.
• Power BI Service and Microsoft Fabric collectors: Performance improvement while harvesting calculation groups, reducing collection time for Power BI models with calculation groups.
• Microsoft SQL Server collector: Addressed issues with sample data size, ensuring consistent sample data collection across different table sizes.
• ServiceNow collector: Fixed CatalogingEventReporter when off the main thread, resolving threading issues during collection.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• ​Databricks collector:​​
  • Harvest information on Databricks Lakeflow, enabling complete metadata capture for Lakeflow workflows and pipelines.
  • Harvest Unity Catalog metastore metadata, expanding coverage of Databricks governance and catalog structures.
• Snowflake collector:​​ Added Tasks harvesting capability, providing complete visibility into Snowflake automated processes and dependencies.
• ​​Microsoft Fabric and Power BI Service collectors:​​ Support for harvesting Paginated Power BI Reports, including preview images, extending metadata coverage for Power BI assets.
• Microsoft Fabric collector:​​ Added page size configuration and improved logging for better performance monitoring and troubleshooting.

​

Bug fixes

• ​​SQL Server Integration Services (SSIS) collector:​​
  • Removed stored procedure dependency cataloging to improve accuracy and reduce false positive dependencies.
    • Fixed missing lineage for SQL statements, ensuring complete data flow visibility across SSIS packages.
• ​​Tableau collector:​​ ​​Fixed null pointer exception (NPE) in field harvesting.

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes​

• ​Tableau collector:​​ Added an option to exclude hidden fields within published data sources.
• Microsoft Fabric collector:​​ Added an option to exclude internal Delta table files from Lakehouse file cataloging, reducing noise in harvested metadata.
• ​​Snowflake collector:​​ Now logs warnings for unsupported stage references in SQL, improving transparency during lineage harvesting.
• ​​Athena collector:​​ Added include/exclude options for databases, giving users more control over collection scope.
• ​Snowflake, Redshift, Databricks, Denodo, Dremio, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors:​​ Reorganized database index harvesting for improved structure and consistency.

​

Bug fixes

• ​Power BI and Microsoft Fabric collectors:​ Fixed the relationship between semantic models/dataflows and data sources to properly represent ​uses​​ relationships, improving lineage accuracy.
• ​​Tableau collector:​​ ​​Corrected ​embedsView​​ relationships for unpublished views in published dashboards, ensuring complete dashboard-to-view associations.