Learn Azure Data Factory

Recap how to create a resource group and add Azure Data Factory, Storage Account, Cosmos DB, and Azure SQL DB, and how to delete resources or an entire resource group.

Use the select component in Azure Data Factory to remove first, middle, and last names, leaving the full name in output, with preview showing id, full name, age, and class.

Apply the filter component to pass only records meeting a condition, using the expression builder to set student ID equals a value. The data preview shows one record after filtering.

Learn to sort data in Azure Data Factory with the sort component, selecting the incoming stream and column, choosing ascending or descending, and noting string versus integer behavior.

Implement a conditional split in Azure Data Factory to route records by state (Odisha, Maharashtra) and by last name Panda, using first matching condition and the expression builder, with previews.

Explore how conditional split in Azure Data Factory routes data under all matching condition, using Odisha and Maharashtra streams and Panda last name to populate the others stream.

Explore using a lookup in an Azure Data Factory data flow to validate order data against the customer dataset from blob storage, using customer ID as the lookup key.

Learn to perform aggregation in Azure Data Factory by grouping by order date and summing price, with casting to integer for totals; also count orders by state to analyze volumes.

Consolidate customer data from multiple sources using union in Azure Data Factory data flow, mapping IDs and names to a unified target.

Learn to implement incremental loads in Azure Data Factory data flow by comparing source orders to a reference output, using the exist transformation to identify new records to migrate.

Explore how Alter Row in Azure Data Factory applies insert, update, delete, and offset policies on a row with priority-based conditions and a UI demonstration.

Flowlet is a reusable container of activities in azure data factory that can be created from an existing mapping data flow or from scratch, enabling reuse across multiple data flows.

Explore how parameters provide runtime values to avoid hardcoding, enabling a single linked service to access multiple databases via domain and database name, and avoid parameterizing credentials with Azure keyboard.

Learn to use parameters in Azure Data Factory datasets to create a single, generic dataset that accesses multiple blob storage files by passing container, directory, and file name at runtime.

Explore variables in Azure Data Factory pipelines, create and set a pipeline variable, modify it during a run, and verify by the set variable output showing value v1.

Learn how to use system variables in Azure Data Factory, including pipeline and trigger scopes, and reference the pipeline name with dynamic content.

Explore control flow activities in Azure Data Factory to build complex, iterative pipeline logic using append variable, set variable, and if condition.

Explore how the append variable activity adds values to an existing array variable within an Azure Data Factory pipeline, and confirm that it supports only array type variables.

Explore the wait activity in Azure Data Factory to pause a pipeline between steps; see a pipeline that runs copy data, waits five seconds, then triggers set variable.

Invoke a child pipeline from a parent pipeline using the execute pipeline activity in Azure Data Factory, coordinating a copy data step with subsequent child execution.

Explore how to apply a filter activity in Azure Data Factory to an input array, define a variable, and view the filtered results with a current item condition.

Master the if condition activity in Azure Data Factory by evaluating an expression to run true or false branches, testing cells with equals on a string parameter, and setting variables.

Learn how the switch activity in Azure Data Factory routes a pipeline using a string parameter and an expression, executing sales, finance, or a default activity with set variable steps.

Learn to configure a storage event trigger in Azure Data Factory to start pipelines when a blob is created, and compare it with the custom event trigger using event grid.

Learn how the tumbling window trigger in Azure Data Factory fires at fixed, non-overlapping, continuous one-hour windows with a start time, backdated catch-up, recurrence, and end dates.

Explore how the tumbling window trigger retry policy works, setting a retry count and a 30-second interval to re-run failed pipelines before marking them failed.

Explore how tumbling window triggers use dependency offset and window size to schedule dependent jobs, including zero and negative offsets, and apply them to daily and weekly workflows.

Compare tumbling window and scheduled trigger to understand their differences and use cases; tumbling window adds backdated processing, retry, reliability, concurrency, and window start/end variables.

Explore Azure Data Factory triggers, including schedule, storage event, and tumbling window triggers, learn how to generate and create them, set properties, schedule dependencies, and compare trigger types.