Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​Marquez collector: ​ Added support for harvesting Marquez datasets associated with Databricks, extending lineage and metadata coverage.
• ​​SQL Server collector​: Added support for SQL Server replications, improving visibility into replicated database environments.
• ​​QlikSense collector Introduced validation for user-configured site identifiers and added warnings when identifiers are invalid or inaccessible.
• Microsoft Fabric collector:
  • Added support for variables and parameters in data pipeline activities even when those activities have not had a recent run.
  • Added support for warehouse sources that use SQL queries in Copy Activities, broadening coverage of pipeline sources.
• Sigma collector: Now supports lineage from datasets to source tables or other datasets, improving traceability of dataset dependencies.
• Databricks collector: Enhanced lineage harvesting to support SQL statements containing the struct function.
• dbt core and dbt clould collectors: Added support for dbt projects targeting SQL Server databases using encryption, improving compatibility in secure environments.
• Power BI collector: Added support for harvesting report images embedded in a zip file, ensuring complete metadata capture from reports.

​Bug fixes

• ​​Databricks collector:
  • Fixed an issue where column properties were not cataloging the correct values due to an API bug.
  • Fixed redundant collection of workspace resources and jobs.
  • Updated the gitProvider property to support both uppercase and camelCase values returned by the Databricks Jobs API.
• ​​Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors: Resolved parsing errors that occurred when harvesting view lineage from views whose SQL contained column comments with parentheses.
• SQL Server Reporting Services (SSRS) collector: Corrected incorrect detection of when to use SOAP vs REST API, ensuring proper connectivity for older and newer SSRS versions.
• OpenAPI collector: Fixed an issue with duplicate identification of API resources, ensuring unique resource cataloging.

​ ​ ​​ ​ ​ ​ ​​

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to the RSS feed from your favorite RSS reader.

Details about the release

New features and changes

• ​​Tableau collector: ​​Harvests the ​last published date​​ for workbooks and data sources, providing greater visibility into update history.
• ​​Alteryx collector​: Catalogs additional metadata, including the ​caption tag​ in ​ToolContainer​​, the ​query​ in ​LockInInput​​, and the ​SQL​ in ​DbFileInput​​.
• ​​QlikSense collector Added a configuration option to ​include​ or ​exclude applications​​, giving users more control over the scope of harvested metadata.

​Bug fixes

• ​​Athena collector : Fixed an issue where the collector stopped after 100 tables. It now correctly ​harvests more than 100 tables​​ in a database.
• ​​Sigma collector​​: Enhanced the workbook filter to avoid missing workbook exceptions, improving reliability during harvesting.
• ​​MySQL collector:​ Fixed an error in fetching statistics for columns whose names are ​reserved SQL keywords​​.
• ​​Microsoft Fabric collector: Resolved issues in stored procedure harvesting by properly resolving names when using ​pipeline variables and parameters​​, and updated relationship types to r​epresent dependencies​​ more accurately.

​
​ ​​ ​ ​ ​ ​​

Bug fixes and enhancements for metadata collectors are released regularly. Go here to to view the past release notes.

📘

Stay updated on collector releases!

To keep up with the latest updates and enhancements to data.world collectors, subscribe to our RSS feed.

Details about the release

​Bug fixes

• ​Power BI collector:​ Fixed an issue to ensure that when a ​database name​ is provided in the ​datasources.yaml​​ file, it is always used and not overridden by values retrieved from a database query.
• ​Tableau collector:​ Fixed an issue so that ​Published Datasources​ are only cataloged when they are being used in a ​Project that is in scope​​, preventing unnecessary or irrelevant catalog entries.
• ​​Informatica CDI collector:​​ Resolved an unexpected exception that was causing the collector to fail, improving stability and reliability.

​
​ ​​ ​ ​ ​ ​​

Details about the release​

​New features and changes​

• SQL Server collector: Added support for harvesting ​agent jobs​​.
• Sigma collector: Added configuration options to include or exclude workspaces, providing greater control over which resources are harvested.
• Redshift collector: Now supports harvesting external tables defined via AWS Glue.
• Microsoft Fabric collector:
  • Dataflow Gen2 is now treated as a separate resource type from Dataflows.
  • Added support for cataloging destinations and table-level lineage for sources and destinations in Dataflow Gen2 CI/CD types.
• Microsoft Fabric and Power BI collectors: Now catalog refresh schedules for resources where refresh configuration is available, helping track automated data updates.
• AWS Glue collector: Now identifies partitioned columns separately from other columns.

​​Bug Fixes​​
​

• Marquez collector: Fixed a null pointer exception that could occur when a job lacked a latest run.
  ​

​
​ ​​ ​ ​ ​ ​​

Details about the release​

​New features and changes​

• ​​Microsoft Fabric collector:​:​
  • Added support for harvesting ​Apps​ and ​Org Apps​​ in Microsoft Fabric.
  • Added support for harvesting GraphQL instances.

​​Bug Fixes​​
​

• Marquez collector: Now skips unsupported dataset types, preventing errors during harvesting.
  ​

​
​ ​​ ​ ​ ​ ​​

Details about the release​

​New features and changes​

• A ​new collector​​, the ​​OpenAPI collector, is now available in ​public preview​​. It supports harvesting metadata from APIs described using OpenAPI v3.0, enabling documentation and cataloging of API assets.
• ​​Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server, SAP HANA collectors:​
  • Added a ​Sensitive Data Classification​ option to allow classification using a ​hosted private-ai instance.​​
  • Now include ​column statistics support​ for ​Date, Timestamp, and Boolean data types, enhancing profiling depth​​ across supported databases.
• ​​Postgres collector:​ Supports ​AWS IAM authentication​ via ​secret and access key parameters​​, offering more secure and flexible credential management.

​​Bug Fixes​​
​

• Oracle collector: Fixed an issue in the ​table index feature​ that previously caused permission errors or max open cursor issues by updating the query logic to use ​DBA_​​ views when available.
• SSIS Collector: Now harvests ​deeply nested control flow executables​​, ensuring complete control flow visibility.
• Snowflake, Redshift, Databricks, Denodo, Oracle, PostgreSQL, Teradata, MySQL, Db2, Netezza, SQL Server SAP HANA collectors: Improved sampling behavior for environments where ​TABLESAMPLE​ is unsupported by falling back to ​LIMIT​ or ​TOP​​ clauses to compute statistics more reliably.
  ​

​
​ ​​ ​ ​ ​ ​​