Microsoft Fabric Masterclass - Future of Data with Fabric

Compare lakehouse and data warehouse architectures in fabric, highlighting spark and sql processing, raw versus structured data, acid compliance, and schema on read versus write.

Explore fabric licensing, including organizational licenses with premium per user or capacity, and individual licenses—free or pro—for creating, sharing, and collaborating on fabric content, including trial capacity.

Learn to sign up for fabric via a 60-day trial or purchased capacity, with admin prerequisites, enable the tenant, and provision premium capacity through Azure.

Create and configure Microsoft Fabric workspaces, organize data into domains, and manage workspace security roles (admin, member, contributor, viewer) via Azure Active Directory.

Discover how fabric lakehouse stores data as Delta parquet files, enabling Delta Lake features like transactions and time travel, while exposing an endpoint for sql queries and Power BI dataset.

Learn to load data into one lake with the One Lake File Explorer: install, log in, upload raw files, and move data from files to delta tables in fabric lakehouse.

Master authentication and authorization in Fabric by controlling access to data in lakehouses within shared workspaces, and assigning roles (admin, member, contributor, viewer) to manage access.

Discover how shortcuts in fabric unify access across clouds, workspaces, and databases, providing a single virtual data copy with security governed by source and target permissions.

Explore creating shortcuts in the fabric ecosystem to access lakehouse data from tables and files, including internal and external sources like Gen2 Azure and S3, minimizing data duplication.

Explore the monitoring hub and One Lake data hub, learning how to track job progress, view Spark jobs, and navigate data lineage across lake houses and datasets.

Understand how lakehouses serve as the center of gravity in fabric, as spark-managed, file-based storage for structured, unstructured, and semi-structured data.

Explore Apache Spark, the open source in-memory compute engine for data engineering, data science, and machine learning on single nodes or clusters, supporting lake house ETL.

Explore using VS Code notebooks for fabric data development, installing VS Code, Synapse extension, Jupyter and Conda, and configuring Java 1.8 and environment variables to run notebooks locally with Spark.

Explore fabric's warehouse designed for all skill levels, enabling no data duplication, parquet with delta lake logs, acid transactions, and cross-engine operability with Spark.

Prepare data and load it into the warehouse by using lakehouse storage, shortcuts, and tables, then import the Instacart dataset and connect notebooks.

Load data into the lakehouse by preparing folders, copying paths, and loading four tables. Verify accessibility via the lakehouse SQL endpoint and create shortcuts to organize folders.

Create a groceries warehouse and load the Instacart products into Fabric via a data pipeline, connecting to a data lake and troubleshooting a copy activity error.

Explore loading data with dataflows in dataflow gen two, replacing values in the source, connecting a storage account, and publishing to a data warehouse for a refreshed table.

Load the departments table through a data pipeline using a data flow, troubleshoot connections, configure mapping and schema, and optimize scheduling and monitoring with copy data details.

Explore cross-database queries in fabric by accessing a vehicles warehouse from groceries warehouse in the same workspace, using a view to connect to vehicles lake house without duplicating data.

Explore how fabric automatically creates statistics to inform query costs, including histogram, average column length, and cardinality statistics, and learn to monitor performance with sessions, connections, and requests DMVs.

Generate real-time environmental data with a Python script, capture it via Azure Event Hubs, feed a Fabric event stream, and store it in a database.

Create and name a Kql query set linked to a database, and explore the Json environmental events table, and enable streaming data for Power BI analysis.

Explore KSQL database concepts by creating and querying external tables, materialized views, and functions, then govern security, retention policies, and data management in Fabric.

Master real-time data analysis in Microsoft Fabric Jupyter notebooks using PySpark, loading the New York Taxi Rides dataset to query, visualize, and cluster pickup and drop-off locations.

Explore querying the New York taxi rides data in a Kusto database using notebooks and Spark in Fabric, with persistent queries and data frames.

Explore how data factory functions as a data integration service that ingests, prepares, and transforms data from disparate sources into a data lake or warehouse, using scalable workflows and transformations.

Explore end-to-end data architecture in Fabric, from bronze to gold lakehouses, data mesh, and centralized vs decentralized designs, using notebooks, data flows, and copy data activities.

Set up two workspaces in Microsoft Fabric: a data central workspace for ingestion and etl to a lake, and an operations workspace for analytics and modeling in a mesh architecture.

Move data from Gen two into one lake using a metadata copy activity with dynamic file paths to parquet files, and update the last update date with a script.

Learn how to prepare data in Data Central, ingest Instacart data into the data lake, and move from files to tables using a data flow and pipeline in Fabric.

Continue the Gen2 dataflow by expanding to more tables, moving parquet data into delta tables in the lake house, and choosing incremental or full load patterns.

Execute pipeline demonstrates running a data flow to transform Instacart data into delta tables with parquet files, and uses nested pipelines and scheduling to automate ingestion and table loading.

Learn how to move data from a bronze lake house to a silver lake house using a notebook, create delta tables, and add a date table to enable analytics.

Build the medallion architecture by creating a gold lakehouse and transforming silver data to gold using data flows, power query, and merges, building a composite key for analytics.

Create a new bronze to silver to gold pipeline, introduce notebook and data flow activities, and implement parameters to dynamically pass start and end dates into the notebook.

Review notebook activities in pipelines by inspecting input-output snapshots and logs. Open the notebook pane to view run details, parameter overrides, and data skew guidance.

Explore how Power BI now sits under fabric, unifying Azure data services, web authoring, and enterprise data modeling with Copilot, Direct Lake, and unified capacities.

Fabric brings version control to Power BI by linking reports to Azure DevOps and enabling web-based edits in the Power BI portal, with commits and tracked changes in a workspace.

Explore how Fabric enables Power BI direct lake, delivering real-time data from delta tables and parquet files with fast, import-like performance via native direct query.