Introduction To Azure Data Factory

Perform an end-to-end demo of azure data factory to copy data from azure sql db to azure storage account using linked services, datasets, pipelines, and triggers for automation.

Create an Azure Data Factory instance by choosing a resource group, unique name, region, and v2; then deploy with a public endpoint.

Create an Azure SQL database as the source system for Azure Data Factory, enable public access, load a sample database, and prepare to copy data to a storage account.

Create a target by provisioning an Azure storage account under infrastructure, with a unique name (ADF demo plus a random number), selecting Azure blob storage and locally redundant for testing.

Access an Azure SQL DB from Azure Data Factory and SSMS by configuring firewall rules and enabling your IP, then copy the customer table data (847 records).

Navigate Azure Data Battery’s interface to create pipelines, data sets, data flows, and triggers, manage linked services and integration runtime, and monitor executions across home, other, and manage tabs.

Create linked services in Azure Data Factory to connect to Azure SQL database as source and Azure Blob storage as destination, configure datasets, test connections, and copy data via ETL.

Create and connect datasets in Azure Data Factory, select the customer table from an Azure SQL Database, and store it as a delimited file in Blob Storage, then publish changes.

Create a new Azure Data Factory pipeline, drag a copy data activity, configure SQL source (customer) and CSV target in storage, map fields, and run debug to copy 847 records.

Explore how to automate Azure Data Factory pipelines with schedule, tumbling window, storage event, and custom event triggers, and monitor executions with detailed input, output, and success or failure notes.

Learn to process multiple files in Azure Data Factory by copying selected data from source to target using wildcard paths, datasets, linked services, and a dynamic pipeline.

Master naming patterns for Azure resources, then set up a resource group, a data factory v2, and a data lake storage with hierarchical namespace for raw data and demos.

Upload multiple sales data files with the same schema into an Azure Data Lake by creating a container with source and target folders, then access it via adf.

Create a data factory pipeline to copy a delimited April file from data lake storage to a target folder using linked services and datasets.

Copy multiple csv files in a single operation by using a wildcard file path to match sales*.csv across subfolders, and configure the copy activity to target a dataset.

Explore how wildcard patterns and sync sets copy data from source to target, then choose merge, preserve hierarchy, or flattened hierarchy for file outputs.

Use list of files to copy multiple csv files from source to target with a single copy operation, and compare flatten vs pressure hierarchy while merging as needed.

Explore how to read source file metadata in Azure Data Factory with the get metadata activity, list child items for each file, and copy conditionally.

Learn to list files and folders with get metadata, feed results to a for each activity, and execute per-item actions using dynamic content and a wait activity.

Configure an if condition in Azure Data Factory to filter start with sales and end with csv files by name and type, routing to true and false executions.

Copy multiple files with a single copy activity using a dynamic dataset, parameterizing filenames and using for each with get metadata to copy sales CSV files to the target.

Design a metadata driven approach to copy data by storing source and target details in a metadata table, then read it to drive copies with Azure Data Factory.

Design a new metadata database in Azure SQL Database to store file configuration details and prepare to copy data with Azure Data Factory using the for each activity.

Read metadata with a lookup activity in Azure Data Factory, using a dataset from an Azure SQL metadata table to retrieve all records.

Execute a metadata driven approach to copy multiple files using lookup and forEach, starting file names with sales and ending with csv, and updating results with a store procedure.

This bonus lecture guides you to download your completion certificate, share it, rate the course, and highlight Azure Data Factory Master Course and its 20 hours of on demand video.