
In this chapter we learn about the importance of data.
In this chapter we will learn about the various tool and services we are going to use in this course.
This chapter goes into using Azure as a cloud-based platform.
In this chapter we will see how to create an Azure Free account.
In this chapter we will have a quick tour around the Azure Portal.
In this chapter we will have an introduction onto Microsoft Fabric.
In this chapter we will learn about the different Microsoft Fabric terms.
In this chapter we will have a quick note on Microsoft Fabric Licensing.
In this chapter we will learn how to on-board onto Microsoft Fabric.
In this chapter we will learn how to get Microsoft Trial capacity.
In this chapter we will learn how to install Visual Studio Code.
In this chapter we will have a note on the data sets we are going to use.
This gives the code for this section.
In this chapter we will have an introduction onto the data warehouse.
In this chapter we will see how data is modelled in a data warehouse.
In this chapter we will see how to build a data warehouse.
In this chapter we will learn how to create a data warehouse.
In this chapter we will learn how to create an Azure Data Lake Gen2 storage account.
In this chapter we will learn how to ingest data via T-SQL
In this chapter we will learn how to ingest data using Data Pipelines.
In this chapter we will learn how to ingest data using Dataflow Gen2.
Build a fact table by denormalizing sales data from sources like sales order header and details, selecting columns such as customer id, unit price, discount, line total, to derive insights.
In this chapter we will view the data sets we are going to use.
In this chapter we will look into a lab onto configuring the source in Data flow Gen2.
In this chapter we will look into a lab on completing the Data flow Gen2 workflow.
In this chapter we will look into a lab into building dimension tables using Data Flow Gen2.
In this chapter we will look into a lab into building Date Dimension tables using Data Flow Gen2.
In this chapter we will look into a lab into running data flows in Data pipelines.
In this chapter we will look into a lab into running a stored procedure in Data pipelines.
In this chapter we will look into a lab into creating a new workspace in Microsoft Fabric.
In this chapter we will look into a lab into transferring data onto a data warehouse.
In this chapter we will look into a lab into building a semantic model.
In this chapter we will look into a lab into using T-SQL commands for a data warehouse.
In this chapter we will look into slowly changing dimensions.
This gives the code for this section.
This chapter goes into what is a lakehouse.
This chapter shows a lab into creating a lakehouse.
This chapter shows a lab on how to ingest files into the lakehouse.
This chapter talks about a delta lake.
This chapter shows a lab on the data setup part for ingesting data into a lakehouse via a data pipeline.
This chapter shows a lab on the running the pipeline for ingesting data into a lakehouse via a data pipeline.
This chapter just talks about running the data pipeline on a schedule.
This chapter talks about shortcuts onto Azure Data Lake.
This chapter talks about shortcuts onto AWS S3.
This chapter shows a lab for a data pipeline use case where we go through the overview.
This chapter shows a lab for a data pipeline use case where we go through creating the Azure SQL database.
This chapter shows a lab for a data pipeline use case where we go through the implementation.
This chapter goes through Apache Spark.
This chapter shows a lab on using Notebooks where we load data onto a data frame.
This chapter shows a lab on using Notebooks where we see how to detect NULL values.
This chapter shows a lab on using Notebooks where we check for duplicate rows.
This chapter shows a lab on using Notebooks where we have the initial setup for building a Fact table.
This chapter shows a lab on using Notebooks where we build the Fact table.
This chapter shows a lab on using Notebooks where we extract values.
This chapter shows a lab on using Notebooks where we build dimension tables.
This chapter shows a lab on using Notebooks where we run notebooks as part of a data pipeline.
This chapter shows a lab on using Notebooks where we merge data.
This gives the code for this section.
This chapter gives a description onto an Eventhouse.
This chapter goes into a lab on ingesting data into the eventhouse.
This chapter goes into a lab on ingesting sample data.
This chapter goes into a lab on ingesting data from Azure Data Lake.
This chapter goes into an introduction onto the Kusto Query Language.
This chapter goes into event streams in event house.
This chapter goes into a lab into creating an Azure Event hub.
This chapter goes into a lab into setting up an Eventstream.
This chapter goes into a lab into ingesting sample data into the event stream.
This chapter goes into a lab onto working with KQL queries.
This chapter goes into a lab into creating data tables.
This chapter goes into a lab into transforming data by choosing columns.
This chapter goes into a lab into transforming data by filtering data.
This chapter gives a note on OneLake availability.
This chapter gives a note on the change data capture feature.
This chapter goes into cleaning up resources.
This chapter has the code for this section.
This chapter goes into a lab for creating a new workspace.
This chapter goes into a lab into giving admin permissions over the tenant.
This chapter goes into a lab into assigning users to Microsoft Fabric.
This chapter goes into a lab onto what new users can do in Microsoft Fabric.
This chapter goes into a lab into giving access to update tables via T-SQL.
This chapter goes into a lab into giving access to update tables via Workspace permissions.
This chapter goes into a lab into column level security.
This chapter goes into a lab into row level security.
This chapter goes into data masking.
This chapter goes into a lab for data masking.
This chapter goes into sensitivity labels.
This chapter goes into endorsing items.
This chapter goes into domains in Microsoft Fabric.
This chapter goes into deployment pipelines in Microsoft Fabric.
This chapter goes into a lab into building deployment pipelines.
This chapter goes into additional notes on deployment pipelines.
This chapter goes into version control.
This chapter goes into an example on using version control.
This gives the code for this section.
This chapter goes into making decisions.
This chapter goes into environments in Microsoft Fabric.
This chapter goes into high concurrency in notebooks.
This chapter goes into lakehouse shortcuts cache.
This chapter goes into an overview of managed private connections.
This chapter goes into a lab onto Managed private connections.
Discover the admin monitoring workspace in Microsoft Fabric, a read-only environment with daily refreshed semantic models and default reports, including content sharing and feature usage and adoption.
Discover lakehouse delta table maintenance with optimize to consolidate small parquet files, a border operation for sorting and compression, and vacuum to remove older unreferenced files.
Learn best practices for the kusto query language (kql) to optimize queries on millions of events, including where filters and placing the fewest rows first in joins.
Monitor event streams by routing data from Azure Event Hub to a lakehouse delta table, and use runtime logs and data insights to track watermark delay and data conversion issues.
Learn to monitor a data warehouse with query activity, identify and kill long-running queries, and use dynamic management views for connections, sessions, and active requests.
Clean up your fabric workspaces by deleting items like data pipelines and data warehouses, respecting item dependencies; remove entire workspaces from workspace settings for a fresh slate.
Modernize the data infrastructure with Microsoft Fabric to enable real time analytics and scalable lakehouse architecture via a medallion bronze to silver design, ingesting Azure SQL data.
Introduce data for the project, illustrating migration from on premises to Microsoft Fabric using an Azure SQL database and CSV or Parquet sources, with Wide World Importers and Adventureworks datasets.
Set up an Azure SQL database server to host transactional data for Microsoft Fabric, using a backup file and Wide World Importers data in a resource group.
Create an Azure storage account with Data Lake Gen2, enable hierarchical namespace, upload the backup file to a backup container, and import the database into SQL Server using basic pricing.
Verify the source system by configuring Azure SQL firewall to allow your client IP, connect to the Wide World Importers database, and inspect core tables for the lakehouse.
Learn how the medallion architecture organizes lakehouse data into bronze, silver, and gold layers, ingested from sources like Azure SQL, then refined for business value.
Create a lake house in Microsoft Fabric to store bronze-layer data and plan for a two-layer medallion architecture with bronze and silver layers, using be Lake House naming conventions.
Ingest data from Azure SQL into the bronze lakehouse, loading tables as-is; create shortcuts to data lake or S3 to avoid duplicates, and use a dynamic for-each pipeline with parameters.
In this lab, copy data from an Azure SQL database into the lakehouse bronze layer using a for each table parameter, validating the pipeline run and creating bronze tables.
Explore mirroring Azure SQL databases in Microsoft Fabric to make a mirrored database available in Fabric, enabling SQL analytics endpoint access and a default semantic model for lakehouse bronze pipelines.
Build the silver layer in the same workspace of the medallion architecture, and construct fact and dimension tables from the bronze layer using notebooks (invoices, invoice lines, customers, stock).
Create a lakehouse and notebook, build a fact invoices table from bronze data in the silver layer, join tables, compute total amount, and save for analytics.
Develop the dimension customer table in the silver lake house by joining bronze customers with customer categories, using customer ID as natural key and creating a surrogate key for analytics.
Build the dimension stock table in the silver layer by joining stock items, stock groups, and the linking table, add a surrogate key, and write to the lake house.
Grant selective access to the lakehouse silver layer via the sql endpoint for DP scientists, without workspace-level access, using a security group and item-level permissions.
Create and share a semantic model in fabric for a DP analyst. Grant licenses and read access to Power BI and Apache Spark data, and define table relationships.
Track delta changes from the Azure SQL database invoices table as they reach lakehouse bronze layer and update the silver layer's invoices via a parquet pipeline from Azure Data Lake.
Design a data pipeline to extract delta changes from Azure SQL for 2014 and load them into the bronze layer of a lakehouse, using pipelines, variables, and dynamic queries.
Transfer delta changes from the bronze lakehouse to the silver layer using dataflow gen2, building the fact invoices table from invoices and invoice lines, then publish to the lakehouse.
Build a silver layer fact invoices table from delta changes in the bronze lakehouse and merge only new year data (2015). Verify with max invoice date.
Build a complete data pipeline from parquet-based information added to Azure SQL to a lakehouse bronze layer and delta-based silver invoices, using Fabric data pipelines, notebooks, and semantic model refresh.
Leverage notebook utilities in Microsoft Fabric to manage file systems and environment variables. Create a master notebook to run notebooks in a data pipeline from bronze to silver with PySpark.
Learn to ingest fabric workspace events into an event house by creating an event stream from workspace item events, routing to a Fabric events table, and querying counts by type.
Right here! Avail special discount coupon links for all of my Al Azure and AWS Courses
This intensive, comprehensive course is designed to prepare data professionals for the Microsoft DP-700 certification exam, focusing on data warehouse implementation and optimization using Microsoft Fabric. Participants will gain the knowledge and practical skills necessary to design, implement, and manage semantic models, data warehouses , lakehouses that leverage the full power of Microsoft's modern data analytics platform.
What are we going to learn
First we'll setup the required accounts and tools required to practice along. We will learn how to use Free trial licenses when it comes to Microsoft Fabric.
Then we will do a deep-dive into hosting data warehouses in Microsoft Fabric. There are different ways in which we can ingest data into a data warehouse. We will learn how to use Data pipelines, T-SQL and Data Flow Gen2 to ingest data into a data warehouse. We won't be done there. We will learn how to design Fact and Dimension tables within a data warehouse.
Next it will be time to perform a deep-dive into Lakehouses. We'll again use Data pipelines and Data Flow Gen2 to ingest data into a lakehouse. We'll learn on how to use Notebooks to in interact with data within a lakehouse.
Next comes the Eventhouse. How do we get data into the eventhouse. How do we stream a continuous data streams into the eventhouse. And then how to use the Kusto Query Language to query for data.
As per the exam , we need to focus on key security concepts, such as how to secure access to items in Microsoft Fabric. How do we enforce column and row level security. There's a lot to cover when it comes to security in Microsoft Fabric.