Category "azure-data-factory-pipeline"

Azure Copy Files from AZDatalake to AZBlob sorage Dynamic

I want to copy files from azure data lake storage to azure blob storage. when i use static file name it's copying successfully but it is not getting success whi

How to use an OData access token with Azure Data Factory Web activity to query Dynamics 365 web API?

Purpose: To consume D365 web API services from Azure Data Factory. What I have done so far: Using Postman I was able to successfully generate a web request to c

Is there any guideline how track changes to an ADF pipeline source code in a Git repository?

I am developing have a few relatively complex ADF pipelines and I like to track my changes. Traditionally with programming languages I keep my code in a git rep

Complex Derived Column Pattern with Cache Sink Lookup

I'm not sure if this is a bug or a limitation but I have a some what complex Derived Column transform using a Column Pattern and Cache Sink to map/set the value

creating metadata of table in Sink/Target in Data Factory automatically

Is it possible on Azure Data Factory to create the table definition in the target/sink database. SSIS used to create the metadata automatically based on the Sou

loop through next url into api url in Azure data factory

I have a an api that contains some data and another api url named "nextapi" I want to loop through each api under api and store the data of each api page to azu

ADF: manage token expiration in copy data activity

I am trying to copy data from a REST API into an Azure SQL Database table. The copy data activity uses pagination and can run longer than 10 minutes after which