
Explore the architecture of Azure Data Factory, linking data sources via linked services, and orchestrate pipelines of activities with datasets to enable data movement and migration.
Create a copy wizard in Azure Data Factory to move data from Azure Blob Storage to Azure SQL Database, set up source and destination, and configure a new ADF instance.
Create a free subscription and a resource group, learn automation templates to deploy Azure resources across environments, and explore storage accounts with blobs, files, queues, and tables using Storage Explorer.
Create a blob in an Azure storage account, review read access geo redundant storage and endpoints, and manage access with keys, connection strings, and container types.
Explore blob storage properties in Azure, comparing block, append, and page blobs, including size limits and use cases. Learn how entity tags enable efficient caching between client and server.
Create a destination by provisioning an Azure SQL database and SQL server, configure server and database settings for ADF, set firewall rules, and verify connectivity from SQL Management Studio.
Create an Azure Data Factory instance and upload a sample csv file to blob storage using Storage Explorer, then review deployment status and basic portal steps.
Create a sql azure schema that mirrors a csv file by using a create table statement or the online query editor, then deploy via Visual Studio with sql server authentication.
Learn to create a copy activity in Azure Data Factory to copy data from blob storage to SQL, configuring source and destination, linked services, and schema preview.
Configure a copy activity in Azure Data Factory to map blob data to a SQL table, with schema and column mapping, upsert via repeatability, fault tolerant settings, and performance options.
Explore the Azure Data Factory resource explorer, review pipelines and copy activities from blob storage to Azure SQL database, and configure linked services, gateways, alerts, and monitoring dashboards.
Author and deploy Azure Data Factory pipelines to move data from blob storage to Azure SQL Database by creating datasets and linked services, with manual and scheduled runs.
Create a blob storage and Azure SQL data sets linked to their services. Build a copy pipeline that uses source and destination datasets to move data.
Fix a failed Azure Data Factory pipeline by inspecting the copy activity error, correcting the source dataset and blob container path, and redeploying until the pipeline succeeds.
Explore scheduling daily data ingestion with Azure Data Factory, handling year-month-day folders, and setting up linked services, datasets, and pipelines for repeatable ETL through copy activities.
Demonstrates setting up a daily ADF scheduling pipeline to monitor a groceries data feed in blob storage and load new CSV files into a SQL database.
Upload groceries data to Azure blob via a linked service, then set up source and destination data sets in Azure Data Factory and validate the upload in the portal.
Create and deploy a groceries pipeline in azure data factory, using a copy activity from blob storage to a SQL sink with column mappings, datasets, and a scheduled window.
Monitor and manage Azure Data Factory pipelines by inspecting consumed and produced datasets, slices, and validation states to ensure data readiness and troubleshoot missing blobs.
Explore the architecture of scheduling in Azure Data Factory, including slices, data sets, and blob storage, and how execution policies like oldest first or newest first govern retrospective processing.
See how Azure Data Factory pipelines use partition by and the slice start runtime to build dynamic folder paths from year, month, and day.
Explore how to analyze an Azure Data Factory pipeline using the diagram view to visualize activities, inputs, and outputs, and learn to use show lineage for focused data flow.
Explore the monitor and manage tool in Azure Data Factory, view activity windows and pipelines, and use the resource explorer to highlight datasets and linked services.
Learn to recover a failed data slice by manually uploading the relevant files to Azure blob storage, triggering a re-run, and verifying results via SQL counts, addressing browser cache issues.
Pause the Azure Data Factory pipeline to stop execution, inspect the underlying data source or storage, fix the issue, then resume the pipeline using the monitoring and manage tools.
Create a production ADF pipeline that transforms and aggregates blob data with Azure Data Lake Analytics using U-SQL, storing results in Azure Data Lake Store and loading into SQL Azure.
Create and configure Azure Data Lake Analytics and Data Lake Store. Use U-sql, Yarn, and Hiveql to run big data jobs with encryption and pay-as-you-go pricing.
Explore data lake analytics and data lake store, focusing on the new job option and data explorer, writing SQL code to Azure Data Factory, and preparing for blob storage uploads.
Shows creating grocery sales data set, uploading to blob storage, using a u sql script to parse space-delimited records, aggregate by date, and store summary in sql and data lake.
Build an Azure Data Factory pipeline to transform blob data with Azure Data Lake Analytics, store results in Azure Data Lake Store, and copy to SQL Azure.
Create and reuse data sets to connect blob storage, data lake store, and SQL Azure within Azure Data Factory. Manage three data sets representing each transformation and copy step.
Continue creating data sets for SQL Azure and JSON in Azure Data Factory, aligning the columns with grocery_sales, and configure data sources for Azure Blob Storage and Data Lake Analytics.
Create an Azure Data Factory pipeline that summarizes blob storage data using data lake analytics, then copies the results into SQL Azure, configuring scripts, inputs, outputs, and mappings.
Identify and fix missing inputs in Azure Data Factory pipeline by correcting dataset names, adjusting validations, deploying, and re-running pipelines until the run turns green.
Discover Azure PaaS through IaaS, PaaS, and SaaS, deployment methods, and regional hosting. Use the portal, PowerShell, or Visual Studio to manage app services, databases, and resource groups.
Explore how to provision a web app in Azure by selecting a subscription, creating a resource group, and configuring an app service plan and location.
Compare Azure App Service pricing tiers from premium to free, and learn how cores, memory, storage, autoscale, slots, backups, SSL, and custom domains shape web app capacity.
Explore web app attributes in Azure App Service, including overview, resource group, url, pricing tier, activity log, access control, and tags, then configure scaling and deployment sources.
Deploy a web app to Azure from Visual Studio using the Azure SDK for dotnet, configure app service, resource group, and app service plan, then publish via web deploy.
Publish the web app to the cloud and verify it in the portal, then provision an Azure mobile app service to connect front end UI with the cloud back end.
Provision mobile apps with an Azure back end, connect databases, and configure Android or iOS clients using Android Studio and Visual Studio, then publish to Azure App Service.
Validate the IDE–Azure connection, publish the mobile app service, and verify backend running; explore creating a storage account, connecting to Azure storage and SQL, and replication options.
Create and configure an Azure storage account, choose geo or zone redundant replication, then set up containers, file shares, and blob types (block, page, and append) for scalable data storage.
Learn how to provision an Azure SQL database as a service, creating a server and database within Azure, and configure credentials, firewall settings, and the Adventureworks sample.
Explore Azure SQL Database elastic pools and the concept of database throughput units, covering basic, standard, and premium tiers and how databases share a pool and scale.
Explore Azure App Service for hosting web apps and rest APIs across languages with auto scaling and high availability, and understand App Service plans and serverless Azure Functions.
Set up an Azure subscription and Visual Studio 2017, then build an ASP.NET core web app, and publish it to Azure App Service using a resource group.
Replace the carousel with a jumbotron in Visual Studio, redeploy to Azure via publish, and explore app service management, resource groups, and Git-based deployment in the portal.
Build a dotnet core web app with a SQL database using entity framework core code-first, then deploy both to Azure as app service and SQL server using Git deployment.
Create and configure an Azure resource group with location metadata, then provision a SQL server under that group, set admin credentials, and note the master database that comes with it.
Configure the SQL server firewall to allow Azure services so the web app talks to the database using cloud shell. Create the database on server 1158 and connect locally.
Configure an Azure sql server and test database, whitelist your client ip via firewall, generate the connection string, and deploy your app using local git with deployment credentials.
Create a free-tier app service plan in a resource group to host a web app, then enable local git deployment and set the connection string with Azure CLI.
Configure and deploy a dotnet core app to azure by setting ASP.NET Core environment to production, using SQL Server connection strings in Azure, and running database.migrate to create tables.
Push changes to the Azure remote to deploy the app on an Azure web app, and connect to the database via SQL Server Management Studio or Azure Portal.
Update the model with a confirmed field, apply ef migrations to update the database, modify views and controllers to use it, and deploy to Azure App Service via git.
Learn about the same origin policy and how cross-origin resource sharing enables a web app to securely fetch resources from other domains, including REST APIs.
Clone a dotnet core rest API from GitHub, enable CORS in Azure App Service, deploy with local git, and verify Swagger UI is accessible.
Enable the built-in core support in App Service for the REST API and test cross-origin requests. Configure CORS with allowed origins, using App Service or custom CORS, noting precedence.
Learn how a content delivery network uses edge servers and edge locations to deliver static assets close to users, cache content, reduce latency, and lower server load.
Select the Akamai CDN over Verizon to reduce endpoint propagation time, understand origin hostname and Azure edge dot net endpoints, and create a test CDN 1158 on Azure dot net.
Create and verify an Azure CDN endpoint, understand the seven-day TTL, and purge specific content like index.html to refresh assets after web app updates.
Azure caching rules let you cache every unique URL with different query strings; this setting makes v=1 and v=2 cache separately, demonstrating CDN setup and upcoming authentication guidance.
Enable Azure Active Directory authentication and authorization, and secure apps with App Service authentication; compare authentication and authorization while demonstrating multi-tier app security across front-end and back-end.
Enable frontend to backend data flow by replacing in-memory data with remote API calls, then configure App Service authentication and Azure Active Directory to issue access tokens for authorized requests.
Grant the front end access to the back end by configuring Azure Active Directory permissions and tokens, and send the access token with every API call.
This video explains securely calling apis from client code in azure app service by enabling cors and wiring a function in app.js to set the access token header.
Learn how to configure Azure App Service backup to protect app configuration, files, and databases, using manual or scheduled backups, with storage in a dedicated container and plan requirements.
Schedule automated backups by defining frequency, start time, and time zone, and set retention to balance data availability with storage costs.
Configure partial backups to exclude images and log files by creating underscore backup dot filter in the site ww w root via kudu.
Discover how to create a simple Azure Logic App that checks an RSS feed every minute and emails new items, using the Logic Apps designer in the Azure portal.
Create and run a logic app that checks an rss feed on a minute schedule, uses dynamic content like title, date, and link in the body, and emails new items.
Create a logic app with a scheduler trigger to fetch travel time via Bing Maps API, convert seconds to minutes, and email via Gmail if it exceeds a limit.
Create a Bing Maps API key and configure a logic app with a recurrence trigger to fetch a travel route and travel time, then prepare for threshold alerts.
Initialize a variable to store travel time in minutes by dividing seconds by 60, then alert via email when travel time exceeds a set limit.
Learn to create and manage Azure virtual machines via portal, PowerShell, or Azure CLI, configure Windows Server 2016, use RDP to install IIS, compare resource manager and classic deployment models.
Compare memory optimized, compute optimized, general purpose, storage optimized, and GPU optimized VMs, then consider availability zones and choosing the cheapest option where available.
Explore geography, Azure regions, and availability zones to gain redundancy, then learn region pairs, availability sets, fault domains, and update domains protecting VMs.
Explore how Azure availability sets distribute virtual machines across fault and update domains to improve uptime in multi-tier apps, and how availability zones span separate data centers within a region.
Deploy a single Azure virtual machine, set up an availability set and managed disks for high availability, and review networking, extensions, and storage options.
Deploy an Azure VM, connect via RDP, install the IIS web server with PowerShell, and open port 80 in the network security group to access the web page.
Create an Azure virtual network with two Windows Server VMs, enabling private communication between them and configure ICMP on the Windows firewall so they can ping each other.
Configure a virtual network to enable inbound icmp traffic between two vms, adjust firewall rules, and establish reliable vm-to-vm communication.
Connect two Azure virtual networks using peering to enable cross-network communication. Learn the peering setup steps, status checks, and the transit versus peering concepts explained.
Explore routing network traffic in Azure using route tables attached to subnets within a virtual network. Learn how IP addressing, subnet masks, and CIDR notation guide packet routing.
Explore creating a single vnet with public, private, and dmz subnets in Azure, and route traffic through a network appliance using a route table to enhance security.
Create a virtual network with subnets public, private, and dmz in the 10.0 address space, deploy three VMs in the subnets, and set up a route table with ip forwarding.
Create a route table, add a route to direct traffic through a network virtual appliance, and associate the route table with the subnet to route VM traffic.
Enable icmp on the public and private VMs and enable ip forwarding on the NVA. Use PowerShell to create a firewall rule that allows icmp and verify routing with traceroute.
Learn to enable a public IP on a VM by configuring the network interface's IP configurations with dynamic allocation, and test connectivity via ping and traceroute.
See how traceroute shows path from a public vm to a private vm in the Azure Virtual Network, illustrating routing through a dmz and a network appliance with ip forwarding.
Learn how Azure Service Bus enables interapplication communication with queues, topics, and relays. Discover brokered, one-direction messaging for queues and topics, subscriber-based delivery, and direct bidirectional relays.
Set up an azure service bus namespace with queues and topics, and use shared access signatures for a dotnet core app to send messages, dead letter queue, and peek lock.
Create a console app, install Microsoft Azure Service Bus, and send ten messages to queue one, then use a second app to receive all twenty messages.
Create a topic and a subscription in the service bus namespace, then publish messages to the topic using a dotnet core console app and receive them from the subscription.
Create an Azure function app in the portal, in a serverless environment, choose a hosting plan, provision storage, and build an http triggered hello world function in C#, with testing.
Create a blob-triggered function in a function app (C#, function.json) that runs on blob add or update, connecting to storage via Azure Storage Explorer and noting latency differences.
Create a timer-triggered Azure function using a cron expression in a C# function app, detailing the six fields that schedule execution by seconds, minutes, and more.
Discover how to create a generic webhook to trigger an Azure function via http post, configure an activity log alert, and test by creating a resource group.
Learn to publish messages to an Azure storage queue using an http-triggered Azure function with an output binding, sending query string data to the out queue.
Create a serverless api with Azure functions by converting an http trigger to an api, restricting to get requests, and exposing via a front end proxy to a backend endpoint.
Learn Azure Event Grid and its event-based architecture. Identify events, event sources, topics, event subscriptions, and event handlers, with examples like Azure Functions and filters.
Create and route custom events using Azure Event Grid by setting up a custom topic, subscribing to it with an Azure function endpoint, and triggering events to log data.
Learn to create and route a custom event to an Azure Event Grid topic with PowerShell, building the event body and validating data reaches subscribers and the event handler.
Explore how to deploy and scale web applications on Azure, leveraging App Service, storage, and Azure SQL databases, while weighing costs, scalability, and availability for cloud migration.
Map web application components to Azure PaaS services, from local IaaS and IIS to app service. Explore deploying front-end and back-end with Azure AD, WebJobs, and Azure SQL database.
deploy a web app using ftp deployment in the azure portal, configuring resource groups and app service plans, with basic or free tiers and optional application insights.
Create an Azure web app with a new app service plan and enable Application Insights, then deploy via FTP from Visual Studio.
Publish a web app to Azure App Service via FTP, configuring FTP host, site path, and credentials from the publish profile, and verify deployment at the Azure Websites URL.
Configure Azure Active Directory for cloud apps by setting up tenants, registering applications, managing users, enabling multi-factor authentication, and navigating classic and new portal views.
Create a new user in Azure Active Directory, set the username and profile, and configure App Service authentication with Azure Active Directory in express mode.
Learn to create an Azure AD app and configure authentication for a demo app, including sign-in permissions, tenant integration, and MVC web application setup with work accounts.
Learn to configure Azure AD authentication for web apps, register apps, manage permissions and keys, enable organizational accounts, and integrate OpenID Connect and multi-factor authentication.
Learn how Azure WebJobs power background tasks in a sample advertising bulletin board, using blobs, Azure SQL, and queues to generate thumbnails and update records, with local and cloud deployment.
Demonstrates using Azure WebJobs to generate ad image thumbnails by processing blob storage and queues, storing results in blob and SQL, with a Contoso ads context.
Configure Azure web jobs by updating web.config with local db and storage connection strings, create a storage account, and copy its keys and connection string into the app.
Explore configuring Azure web jobs for a Contoso ads app by wiring account name and key in web.config and app.config, running locally, and processing image thumbnails via a queue.
Demonstrates deploying and running an Azure web job that processes queue messages to generate image thumbnails, stores results in blob storage, and updates the SQL database.
Learn how an Azure web job processes uploaded images in blob storage to generate thumbnails and upload them back to the blob, with local configuration updates and portal verification.
Publish the Contoso Ads web app to Azure App Service using Web Deploy, with WebJobs storage, selecting a resource group and App Service plan, and configure storage and database connections.
Create an Azure sql database for image storage and map the Contoso ads context. Configure runtime connection strings and update web.config to point to the south india server.
Configure the image database connection string for Azure apps, reuse it in web.config and web jobs storage, and enable reading, uploading, and thumbnailing images in blob storage.
Troubleshoot Azure webjobs by validating configuration files and connection strings, correcting credentials, restarting the web app, and monitoring web job status (continuous or scheduled).
Explore how web jobs generate thumbnails from uploaded ads, processing input blobs and queues, monitoring with logs and function invocations, and storing results in output blobs and Azure SQL.
Connect to Azure SQL Database with SQL Server Management Studio, configure firewall to allow your client IP, and access image ads data and thumbnails in the Contoso server.
Explore programmatic access to Azure storage blobs from your application using REST APIs and the Storage Client Library, including container setup and blob operations in a console app.
Learn how to copy a text file into an Azure blob using a blob client, container, and cloud storage account, then upload from a stream and verify the blob content.
Learn to program against Azure Table NoSQL storage using cloud table client and table entity, with partition key and row key, including insert and batch operations.
Execute insert and batch operations in Azure Table storage, create a table, insert customers, and query by partition key and row key using a customer entity.
Explore Azure storage concepts, including tables, queues, and blobs, learn to handle conflicts and exceptions in batch operations, and map web apps, webjobs, and Azure SQL for cloud deployment.
Learn how to migrate a local IIS website to Azure App Service, leveraging deployment slots, Traffic Manager for geographically nearest deployment, and the Azure App Service Migration Assistant.
Learn to migrate a local web app from IIS and ASP.NET to Azure App Service using the migration assistant, including publishing settings, publish profiles, and site mapping.
Learn to migrate and publish a local web app to Azure App Service using the migration assistant, and review compatibility analysis, readiness report, and port bindings.
Use the Azure App Migration Assistant Windows tool to run readiness assessments, perform compatibility checks, and migrate a local IIS MVC app to Azure App Service.
Learn to migrate a local MySQL database to Azure SQL Database using Data Migration Assistant, performing compatibility assessment, generating migration scripts, and completing the PaaS deployment via the Azure portal.
Use the Data Migration Assistant to assess compatibility and feature parity for an Azure SQL database migration, export the assessment report, and provision an Azure SQL Database from SQL Server.
Migrate an on-premises SQL server to Azure SQL database using the data migration assistant, covering schema and data, with firewall configuration and verification of department and employee tables.
Learn to migrate dotnet web forms and mvc apps from on-premises to Azure PaaS, moving data to Azure SQL and deploying app service plans for scalable web apps.
Migrate a dotnet web form from local IIS to Azure paas, moving database to Azure SQL with Data Migration Assistant, then deploy to an Azure Web App and enable insights.
Connect to the Azure employee database from SQL Server Management Studio, configure firewall rules to allow the client IP, update the connection string to Azure SQL Server, and verify access.
Leverage the migration readiness report and readiness assessment to validate checks for migrating the employee query form, including port bindings and app service deployment steps with publish settings.
Publish the IIS-hosted employee queue form to the Azure web app by updating the original site name in the publish profile, migrate the database to Azure SQL, and verify deployment.
Migrate an MVC application to Azure web app by moving the database to Azure SQL with the Data Migration Assistant, deploy to an app service tier, and validate the configuration.
Assess a local database and migrate it to Azure SQL database via paas, checking compatibility and feature parity while noting that trace flags are not supported in an end-to-end migration.
Generate SQL scripts for the employee and department tables, deploy the schema, migrate the data to Azure SQL database, and update the IIS connection strings in the MVC application.
Migrate a local MVC application to the Azure App Service tier using App Service Migration Assistant, generating deployment settings and provisioning an employee query app on basic App Service plan.
Migrate content and database for an mvc app from local to azure app service using app migration assistant and publish settings, updating the original site name and deploying to azurewebsites.net.
Execute an end-to-end migration: migrate the database, connect your local mvc app, move the app to the cloud, then configure monitoring, alerts, and performance testing.
Course Introduction
Welcome to "Azure Advanced: Data Factory | PAAS | App Services | Application Migration." This comprehensive course is designed to equip you with advanced knowledge and practical skills to effectively utilize Microsoft Azure. Whether you're looking to deepen your understanding of Azure Data Factory, explore Platform as a Service (PAAS), manage Azure App Services, or master the nuances of migrating applications to Azure, this course covers it all. Through a blend of theoretical insights and hands-on exercises, you'll gain the expertise needed to excel in the dynamic field of cloud computing.
Section 1: Microsoft Azure - Data Factory
In this section, we delve into the essentials of Azure Data Factory. You'll start with an introduction to its capabilities and practical applications. From creating a Blob and setting up ADF in Microsoft Azure to managing pipelines and handling errors, you'll gain a comprehensive understanding of the data integration process. Key topics include creating schemas in SQL Azure, copy activities, and monitoring pipelines. By the end of this section, you’ll be proficient in setting up and managing data workflows in Azure.
Section 2: Microsoft Azure - PAAS Overview
This section offers a detailed overview of Platform as a Service (PAAS) in Microsoft Azure. You’ll learn about various Azure services, including Web Apps, mobile app deployment, Azure Storage, and SQL Databases. The section focuses on deploying web applications through Visual Studio, provisioning mobile apps, and managing Azure SQL databases. By understanding these core services, you'll be able to leverage PAAS for building and deploying scalable applications.
Section 3: Azure Practical - App Services
Here, we focus on the practical aspects of Azure App Services. Starting with an introduction to Azure App Service, you'll learn how to deploy websites, make changes, and redeploy them. The section covers key services such as Dot Net Core Web Apps, resource group creation, and database migration functions. You’ll also explore enabling CORS, deploying REST APIs, and implementing Azure CDN. This section equips you with the skills to manage and optimize Azure App Services effectively.
Section 4: Azure Practical - Developing Your Applications
This section is dedicated to developing applications on Azure. It begins with an introduction to Azure application development, including web app components and FTP deployment. You’ll learn about using Azure Active Directory, creating web jobs, and managing databases for applications. Practical examples and step-by-step guidance help you understand how to develop, deploy, and manage applications in Azure, ensuring you can handle real-world scenarios with confidence.
Section 5: Azure Practical - Website and Database Migration to Azure Platform
In this section, you’ll learn about migrating websites and databases to the Azure platform. Topics include moving local web apps, using migration tools, and configuring Azure SQL databases. The section guides you through connecting to Azure SQL servers and optimizing performance. By the end, you'll have a solid understanding of the migration process, from initial setup to final implementation, ensuring a smooth transition of your applications to the cloud.
Section 6: Azure Practical - Migrating .NET-based Web Applications to Azure PAAS
The final section focuses on migrating .NET-based web applications to Azure PAAS. You’ll explore the steps involved in migrating web applications, connecting to Azure databases, and generating SQL scripts for migration. Detailed tutorials on MVC application migration, content and database transfer, and testing configurations provide a complete guide to the migration process. This section ensures you can seamlessly migrate and manage .NET applications in the Azure environment.
Conclusion
"Azure Advanced: Data Factory | PAAS | App Services | Application Migration" is designed to provide you with in-depth knowledge and hands-on experience in using Microsoft Azure. Each section builds on the previous one, ensuring a comprehensive learning experience. By the end of this course, you’ll have the skills and confidence to manage complex Azure environments, develop robust applications, and execute seamless migrations, positioning you for success in the ever-evolving world of cloud computing.