Data Engineering Certification: Master Data with SQL& Azure

Learn how databases and database management systems simplify data access, and how SQL—structured query language—provides a consistent syntax across Oracle, MySQL, SQL Server, and PostgreSQL for querying applications and dashboards.

Explore different types of databases, including relational SQL databases like SQL Server and NoSQL options such as document, key-value, column-based, and graph databases with real-world examples.

Explore how a database management system is installed on a server, organized into databases, tables, and schemas, and how primary keys, constraints, and foreign keys link data.

Learn to install SQL Server and SQL Server Management Studio, choose Developer Edition, perform basic or custom setup, configure authentication, and connect to create a database.

Learn how to create and manage databases with SQL Server and SQL Server Management Studio, including creating databases, creating tables, selecting databases, executing SQL code, and dropping databases.

Learn how to alter a table by adding and dropping columns, enforce unique column names, and drop the table using a sample SQL database example.

Master data manipulation by inserting, updating, and deleting rows in a student table using insert into, while enforcing primary key and not null constraints and handling duplicate keys.

Delete data with the delete statement, use where to remove specific records or multiple rows, handle nulls with is null, and remember without where all data is deleted.

Learn the basics of sql filtering with where conditions to extract specific records, using operators such as =, not equal, >, <, and between across numbers, dates, and strings.

Master data extraction with SQL using the like operator for pattern matching, including starts with, ends with, contains with percent and underscore wildcards, and combining conditions with and, or, not.

Learn to create an Azure storage account in the portal, selecting subscription, resource group, unique name, location, redundancy, performance, access tier, and data protection settings.

Download and install Storage Explorer, sign in or attach your subscription, and use connection strings, keys, or SAS to access and manage your storage accounts.

Connect to Azure SQL from the portal and SSMS by enabling firewall rules for your current IP or this subscription, then run SQL queries in portal, SSMS, or Data Studio.

Create your first azure data factory by selecting subscription, resource group, and a unique name, choosing east us, v2, with optional git version control, then deploy a public endpoint.

Explore the Azure Data Factory interface—from Launch Studio to author, monitor, and learning center—and learn to build pipelines with linked services, integration runtimes, triggers, datasets, and templates.

Demonstrates copying data from Azure SQL DB to Azure Storage and then to Azure Data Lake Storage Gen2 using Azure Data Factory, with connections, datasets, and two copy activities.

Build an Azure Data Factory pipeline to copy data from a sql db table to a csv in storage using a copy activity, with source, sink, mapping, and run monitoring.

Add extra columns in copy activity to capture file path, file name, and UTC execution time using dynamic content and system variables. Verify updates in ADLS after running the pipeline.

Explore azure data factory by creating datasets for source and target in a demo container, using a delimited file, and publishing changes to enable data movement with adls paths.

Learn to insert and configure a wait activity in an Azure Data Factory pipeline to pause execution for a defined number of seconds, enabling controlled sequencing between data copy steps.

Learn how to use the delete activity in Azure Data Factory to remove source files after a successful copy to target. Configure with patterns and review logs for confirmation.

Configure a get metadata activity within a data factory pipeline using a source dataset. View metadata such as last modified, size, structure, and child items to filter or copy data.

Learn to implement an if condition in Azure Data Factory, routing to true or false activities. Use get metadata to check file exist and drive copy, delete, wait, or fail.

Discover how the lookup activity in Azure Data Factory reads data from datasets, outputs JSON, and retrieves single or all records, including CSV wildcards and file lists.

Learn to use parameters and variables in Data Factory to run pipelines with dynamic file names, using dataset and pipeline level parameters and dynamic content for copy activities.

Learn to use variables in Azure Data Factory, set and get metadata for last modified, format dates, and route into year month and date folders using dynamic content and concat.

Learn to loop over arrays with the for each activity in Azure Data Factory, list files with get metadata, and control sequential versus parallel execution.

Practice assignment teaches copying a file from source to target, organizing outputs into csv and json folders with year/month/date subfolders based on UTC now.