'Azure Purview Data Lineage with Databricks
I am using Azure Purview for Data Governance, and Data Lineage. We use Databricks in our Data Architecture, but there isn't any native support for capturing Data Lineage with Databricks.
I found the following links that will allow you to create custom processes in Azure Purview.
Databricks notebooks lineage in Azure Purview
Can someone let me know if there is any recent methods of achieving Data Lineage in Azure Purview with Databricks?
Solution 1:[1]
Data integration and ETL tools can push lineage into Microsoft Purview at execution time. Tools such as Data Factory, Data Share, Synapse, Azure Databricks, and so on, belong to this category of data processing systems. The data processing systems reference datasets as source from different databases and storage solutions to create target datasets. The list of data processing systems currently integrated with Microsoft Purview for lineage are listed in below table.
Refer - https://docs.microsoft.com/en-us/azure/purview/catalog-lineage-user-guide#data-processing-systems
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | AbhishekKhandave-MT |