AZURE DATA FACTORY TRAINING WITH REAL TIME SCENARIOS

Copy all files from the landing container to the sink container in a data lake storage gen two using Azure Data Factory pipelines, leveraging get metadata to enumerate files.

Use a get metadata and filter activity to select mpq1.csv and mpq2.csv from the lending container, then loop with a for each and copy them to the sink container.

Copy all Azure SQL tables to Azure Data Lake as csv using a lookup and for each loop, with dynamic file naming for daily backups.

Demystify Azure Data Factory real time scenarios by loading q1 and q2 to one sink directory and q3, q4, and q5 to another using get metadata, filter, and copy.

Load data from Azure blob storage into Snowflake using Azure Data Factory, with SAS token link services and sequential for each to populate multiple CSV files into Snowflake tables.

Migrate on-premise SQL Server data to Azure Data Lake Storage via Azure Data Factory, using a self-hosted integration runtime and lookup for-each to copy tables daily as CSV.

Migrate data from on premise sql server to Azure SQL using Azure Data Factory, employing self-hosted integration runtime, lookup and for each loop, with parameterized datasets.

demonstrates azure data factory migration from azure sql to adls, selecting only required columns via a config table, lookup, and copy activity with per-table queries.

Migrate data from on premise sql server to azure sql by selecting only required columns, using a self-hosted integration runtime and lookup activity with a configuration details table.

Explore real-time data transfers with Azure Data Factory, copying CSV and JSON files from Blob Storage to ADLS Gen2 using linked services, datasets, and pipelines.

Use a wildcard pattern to copy only amazon-prefixed csv files from blob storage to data lake storage gen2, excluding excel files, via a dataset-driven copy pipeline.

Use azure data factory copy data activity with recursive access on adls gen2 to merge multiple CSV files into a single output file using the merge files behavior, preserving hierarchy.

Log errors during Azure SQL to Azure SQL data transfer by enabling login in the copy activity, capturing nulls, truncation, and primary key violations to a storage path for analysis.

Demonstrate a merge (upsert) in Azure Data Factory by syncing a source emp table with a destination emp1 table, updating matching employee ids and inserting new rows as needed.

Load a nested JSON into a CSV file using Azure Data Factory by mapping employee arrays to CSV columns, configuring source and sink in ADLS Gen2, and validating the pipeline.

Explore precedence constraints in Azure Data Factory, including on success, on fail, on completion, and on skip, demonstrated with parallel copy activities and parameterized datasets.

Showcases parameterization in Azure Data Factory by creating parameterized datasets and a parameterized Azure SQL link service, plus a pipeline with database name and table inputs.

Implement slowly changing dimension type 1 with Azure Data Factory data flows to compare source and destination via a four-column hash, inserting new records and updating changed ones.

Explore an Azure Data Factory data flow implementing slowly changing dimension type 2, using surrogate keys, start dates, end dates, and is active flags to preserve historical employee data.

Implement slowly changing dimension type three in Azure Data Factory by processing delta data, updating the previous name column and storing the latest name, while upserting new records.

Learn to handle schema drift and inferred drifted data types in Azure Data Factory by validating schemas, importing projections, and mapping changes from CSV to Azure SQL.