Microsoft Fabric Data Engineering: 10+ Real-Time Projects

Microsoft Fabric unifies data engineering and analytics in a single SaaS platform, bringing Power BI, Azure Data Factory, Spark, data lake, and lake house together for end-to-end analytics.

Learn how to create a Microsoft Fabric account and start a 60-day free trial with an organization email, or use Azure free credits to set up Fabric capacity.

navigate the microsoft fabric overview, manage workspaces, and explore data catalogs and real-time analytics, while reviewing workloads like data engineering, data science, data warehouse, and power bi.

Discover how one lake unifies analytics data on fabric, built on Azure Data Lake Storage Gen2, storing delta parquet data for all workloads.

Explore how lakehouse blends data lake and data warehouse in fabric, storing unstructured files and structured delta tables in one place, with SQL and PySpark queries and Power BI visualization.

Create and link data across multiple fabric workspaces by building lake houses, uploading files, and generating shortcuts for files and tables to access cross-workspace data.

Create external shortcuts in fabric to access data stored in Azure Data Lake Storage without moving it, enabling cross-lake analytics from ADLS to fabric.

Use the get metadata activity in fabric to extract file metadata, identify files in a container, view last modified dates, and filter by file type or name.

Apply the if condition activity in Microsoft Fabric to route files by name. Read metadata from ADLs, loop with for each, delete EMP files, and copy others to the lakehouse.

Learn how to schedule a pipeline in Fabric on hourly, daily, weekly, or monthly intervals, set the start date and time zone, and automate runs.

Perform a join in Microsoft Fabric by using merge queries as new to combine two data sources, selecting customer id from each table and applying left or inner joins.

Explore pools in Microsoft Fabric, comparing starter and custom pools, their setup times, performance, and billing for development versus production workloads.

Create and run your first spark notebook in fabric's data engineering workspace, writing PySpark, SQL, or R code, managing pools and clusters, and using notebooks, lake house, and environments.

Create a data frame in PySpark by reading a csv from a lakehouse. Use header true and infer schema to auto-detect types, and explore read options with spark.read.csv and spark.read.format.

Learn how to use PySpark to read a data frame, display data, select columns (id, sales, product name), and limit or drop columns to view a focused dataset.

Learn to use with column in PySpark to cast the quantity column from string to integer. Create new columns like quantity one and country, update values, and rename zip mode.

Master group by in PySpark by aggregating with sum, min, max, and count to analyze segment and city profit in a dataframe, displaying segment-wise results.

Handle nulls in pyspark on fabric by reading a csv into a dataframe, filtering with is null or is not null, and replacing missing values with df.na.fill for specific columns.

Convert a PySpark data frame into a temporary view, then perform transformations with spark sql or sql, and optionally convert back to a data frame within the session.

Learn to use notebook utils in fabric notebooks to list, copy, move files with fs, cp, mv, and run or exit notebooks from one notebook to another.

Learn how to load data into a data warehouse using three methods: a data factory pipeline or a data flow, copy into, and create table as select.

Learn how to run T-SQL directly in a notebook by connecting to your data warehouse, selecting T-SQL, adding the warehouse, and executing queries like selecting from consumer.

Learn how Power BI visualizes data in Microsoft Fabric, connecting lake house, warehouses, and SQL endpoints to build reports and visualizations using Power BI Desktop and DAX.

Power BI, a Microsoft business intelligence tool, connects to diverse data sources, cleans data, and creates interactive dashboards and visual reports for real-time insights shared with stakeholders.

Explore Power BI desktop by learning how to connect to multiple data sources, navigate the report, table, model, and DAX views, and create visualizations with the visualization tab.

Load an Excel workbook into Power BI Desktop by selecting four sheets (customer, order, people, return), adjust data types, view relationships in the data model, and begin visualizations.

Add a Power BI sales KPI to compare yearly sales against a 5 million target and quantify distance from the goal.

Derive the day of week from the order date using format, then apply a DAX switch to label each date as weekend or weekday via a new column.

Group data in Power BI by creating a region group column that maps regions into north, central, east, and south, then visualize sales by the region in a pie chart.

Explore how to use the rankx function in Power BI to rank countries by total sales, create measures, and filter ranges such as top 4 to top 9.

Master text modification in Power Query using extract, format, and parse to transform a text column, including length, first or last characters, delimiter splits, and case changes.

Learn to handle null values in Power Query by removing blank rows, and replacing nulls with a value or Na after converting numeric columns to text when needed.

Learn how to union two tables in power query by using append query. Ensure columns and data types match to successfully combine data from both tables.

Execute an end-to-end Azure data engineering project with Microsoft Fabric to build a unified customer 360 dataset for an e-commerce company, using Data Factory, PySpark, Lake House, and Power BI.

Set up an Azure ADLS storage account with hierarchical namespace, create a lakehouse with silver and gold folders, and run a PySpark notebook to clean and store data.

Create a data pipeline that reads files from Azure Data Lake Gen2, uses get metadata, and for each file copies them into the lake house as parquet.