[8 Course BUNDLE]: DP-203: Data Engineering on MS Azure

Compare dtu-based and vcore-based pricing tiers for azure sql database, selecting basic, standard, premium, general purpose, hyperscale, or business critical configurations, with provisioned or serverless compute and hybrid benefit.

Enable auditing to track database activities and store audit logs in a storage account or log analytics. Configure retention days and review events like create table and select.

Learn to use the get metadata activity in a data factory pipeline, creating a dataset and link service to scan a local folder and return file metadata.

Learn how to use the if condition activity in data factory to branch between true and false activities. Use an expression and the empty function to evaluate the previous output.

Define pipeline-level variables and use append variable to add values in an if condition, then access them across the pipeline and set defaults.

Explore the dataflow source transformation in Azure Data Factory, configuring datasets from blob storage or data lake, and handling schema drift for scalable visual data integration.

Explore adding a derived column in a dataflow to create a new field, such as a hardcoded customer name, and see auto mapping populate the sink.

Enrich sales data with a lookup transformation in a data factory data flow by joining on customer id to fetch names from a reference table; publish the pipeline.

Configure a filter transformation in the data flow to select records with quantity greater than four, publish changes, and validate results across the destination tables.

Explore the exist transformation in a data flow to filter records by existence in a second source, configure exist or doesn't exist, and validate results by running the pipeline.

Implement an Azure Data Factory v2 lab that builds a pipeline to load source SQL data (customer and lead) into staging with incremental and full loads using lookup and filters.

Explore slowly changing dimension type 1, where updates replace existing records to avoid history in the dimension, using staging, incremental loads, and a surrogate key.

Learn to implement slowly changing dimension type 2 in ADFv2 with a data flow, using staging and dimension, lookups, and conditional splits to insert new records and update current flags.

Explore Azure Data Lake Storage Gen2, a cost-effective, scalable solution built on blob storage for unstructured data, supporting ELT workflows and Hadoop-compatible analytics.

Explore how to monitor Azure Data Lake Storage using classic and monitoring views, configure alert rules for latency, capacity, and ingress/egress, and review diagnostic and usage graphs for admin insights.

Load data from a local file system into Synapse Analytics tables using Azure Data Factory, staging the file in blob storage and loading with Polybius or bulk insert.

Explore Cosmos DB, a globally distributed database you can deploy worldwide with a few clicks, offering multi API support including SQL and Cassandra, across regional Azure data centers.

Learn the database container items hierarchy in cosmos db, from a database account to databases, containers, and items, and how stored procedures, user defined functions, and triggers live inside containers.

Explore how to monitor a Cosmos DB database using metrics, per-database and per-container views, region filters, throughput, requests, storage, latency, and cost.

Build a C# app to connect to Cosmos DB and read items from a container. Configure the endpoint and primary key, then read and write data from the database.

Create an Azure Databricks workspace in the portal by selecting your subscription, resource group, and workspace name in south central us, then access notebooks and tables inside the workspace.

Extract data from a data lake by mounting containers, transform into global tables, and load only new customers by joining with dim customer to avoid duplicates.

Learn to create and manage table relationships, set cardinality, and configure cross filter directions in your Power BI data model.

Apply the group by feature in the transform section to aggregate by city, state, and country using count, sum, average, max, and min, with an optional duplicate of the table.

Set the background color and add a company logo to a Power BI desktop report, while adjusting page background properties and inserting the logo image.

Build detailed, table-based reports by adding customer and sales order data—quantity, unit price, discounts, and ship method—and use conditional formatting to reveal city-level sales and product-level insights.

Explore page-level and all-pages filters, set a default city filter like London, and use slicers to synchronize visuals across a multi-page report before publishing to the cloud for sharing.

Install and configure the personal gateway for the Power BI service on your local laptop to access the Excel file and the Aja Sequel Database, enabling dataset refresh.

Create a Power BI service dashboard by pinning visuals from existing reports. Share the dashboard with others and understand its read-only snapshot nature.