Microsoft Fabric - The Complete Guide

Discover how one lake serves as a centralized, invisible data repository with shortcuts to external files, enabling seamless access in fabric through workspaces and lake houses.

Explore lake houses as a single, flexible workspace for storing delta tables and both structured and unstructured data, with a read-only SQL endpoint and easy tool integration.

Convert a csv file into a delta table for analytics, fix header issues with underscores, and load data into a new table with a preview of column types.

install the one lake file explorer, sign in, and access your fabric workspace from windows explorer to drag and drop files for lakehouse cloud sync.

Explore the SQL analytics endpoint in Microsoft Fabric, connect with SQL connection string, run queries on the sales table, and save as views or create visual queries for easier analysis.

Create shortcuts in fabric to reference data from external sources like S3 without copying it. Access the shortcut as a delta table with full querying and semantic modeling.

Learn to manage access in a Fabric workspace by assigning roles, such as admin, member, contributor, or viewer, and sharing items like a lake house with selected users.

Create a new semantic model in the lake house for Power BI using direct lake connectivity to avoid data copies and support multiple use cases.

Create measures in the semantic model to perform virtual calculations that are reusable across reports, using a simple sum-based discount example in a sales table.

Connect Power BI Desktop to Fabric lake house semantic models for live data visualization, sign in with two-factor authentication, and publish reports to a Fabric workspace.

Connect the SQL endpoint directly in Power BI using the connection string, sign in with two factor authentication, and load tables in import mode for transformation and modeling.

Update the app by editing underlying Power BI reports, publish changes, and adjust content, audience, color, and logo, and when needed unpublish or delete the app in Fabric.

Learn how Microsoft Fabric uses data pipelines for orchestration and data flow gen2 for transformation, with Power Query editor, connecting sources, shaping data, and loading to lake house or warehouse.

Create a data pipeline in fabric using the copy data activity for ELT, connect to parquet retail data, and load into a lakehouse table with mappings.

Add data flow gen 2 to a Microsoft Fabric data pipeline to transform data in the Power Query editor, connect to a lakehouse, publish, and schedule the workflow.

Learn to troubleshoot and monitor data pipelines using run history, monitoring hub, and gantt views; inspect per-activity details, input/output JSON, data read/write, and cancel or manually trigger runs.

Create your first data warehouse in Microsoft Fabric, explore its features from data engineering to a SQL perspective, use delta parquet format, and build tables with T-SQL and data pipelines.

Compare Fabric data warehouse with lake house read-only SQL endpoint, then demonstrate creating tables using T-SQL, with not null constraints, and observe table creation and basic queries.

Learn to use the copy into command to ingest data from Azure blob storage into a data warehouse, creating and populating the dimension city and fact sale tables.

Explore cloning tables in Microsoft Fabric: use the clone command to create reference-only tables that don’t copy data, enabling fast, storage-efficient updates across clones and originals.

Explore Apache Spark in Fabric as a distributed in-memory compute framework for data engineering and data science, with notebooks and multi-language support for large-scale batch and real-time processing.

Create a spark data frame in fabric notebooks by defining a python data list and a manual schema, then display results and explore loading data from files and lakehouses.

Navigate notebooks by executing cells interactively, manage variables and outputs, and control cell states with freeze and lock options while using markdown to organize content.

Upload files or reference data from anywhere, then read csv and json with pandas in notebook, with spark for larger lake house files.

Define data schemas by manually specifying types or automatically inferring them in pandas and spark, ensuring data integrity in large lakehouse datasets.

Apply data filtering with the filter method on a data frame, using single and/or multiple conditions, and between timestamps to select 2023 January high-value transactions, then display results.

Learn to aggregate and group data with PySpark SQL, applying average and sum, aliasing results, and grouping by customer to compute total sales.

Write data frames to lakehouse as csv, json, or parquet files, using overwrite or append modes, and store into delta tables for semantic models and Power BI visualization.

Learn to run sql queries in spark sql within notebooks, perform filters, joins, and aggregations, and switch between spark sql and spark python to work with tables and data frames.

Learn to schedule notebooks for interactive exploration, visualizations, and machine learning, and link them to data pipelines, with options for hourly, daily, weekly runs and spark jobs.

Learn how to integrate notebooks into data pipelines, configure pipeline activities, set dependencies, and orchestrate workflows from notebook to spark job definition for scalable production.

Explore spark settings in your workspace, configure pools and node sizes, enable auto scaling, and optimize spark workloads for data science, notebooks, and machine learning experiments.

Explore spark job definitions for production, including creating, scheduling, and running jobs that read csv data, transform it, and write parquet files to a lakehouse.

Learn to schedule and monitor spark job definitions, configure retry policies and environments, and analyze run metrics across spark jobs and notebooks.

Explore the main building blocks of real-time analytics in fabric, including the event house, Kql databases and tables, and event streams, and how the real-time hub organizes streaming data.

Create a KQL database inside a Fabric event house, load a sample stocks analytics dataset, explore the stocks table, and run basic queries to learn KQL syntax and query sets.

Explore the basics of KQL in Microsoft Fabric by building piped data operations from a table source, selecting columns with project, taking 100 rows, and considering performance with columnar storage.

Learn how to filter and sort data using KCL: apply where conditions, project specific columns (date, close, volume), and order by date descending to efficiently query and view results.

Aggregate data with summarize to compute counts, max close, and averages. Group by ticker and date, bin with week of year, and prepare visuals in queries.

Explore visualizing data with kcl using render to create pie, bar, column, and time charts from time-series data, with where filters, for interactive, high-performance insights in Fabric.

Learn to join data in Kql by combining stocks and company tables, using inner and full outer joins, projecting specific columns, limiting results, and preparing for event streams in Fabric.

Learn to create and configure event streams in a Microsoft Fabric workspace, ingest real-time data via a visual editor, apply transformations, and route results to Kql databases or other destinations.

Route a new event stream into a KQL database by configuring an event house destination, creating a new table, selecting JSON input, publishing the live data flow, and applying transformations.

Connect a KQL database to Power BI to visualize streaming data with live direct query or import. Use a semantic model optimized for Power BI to enable lake house notebooks.

Learn to ingest stream data into a lake house using a one lake shortcut from the event house, enabling table availability and optional destination for the semantic model.

Build an end-to-end fabric-based classification model from data ingestion to applying to new data, using notebooks, experiments, and mlflow to track, register, and reuse the model.

Explore the data with Data Wrangler, check for missing values, and apply cleansing steps like dropping columns, removing duplicates, and one-hot encoding to prepare for modeling.

Set up experiments with MLflow auto logging to train and compare models like LightGBM and random forest on prepared, split data, then evaluate results to select the best model.

Split the data into train and test sets, using a 20% test split with a reproducible seed, oversample the minority class to balance the imbalanced data before training models.

Train a random forest classifier within the experiment using balanced data. Track metrics, generate a report and confusion matrix, and prepare for comparing runs in the next lecture.

Register models in an experiment with auto logging, then evaluate and compare runs using accuracy, precision, recall, and F1 score, aided by a confusion matrix.

Apply the trained model via wizard or code, map input data to the model, output predictions to a lake house table, and schedule notebooks for end-to-end automation.

Explore fabric activator, a real-time intelligence component that detects events such as threshold crossings and data changes, and automatically triggers actions like alerts or emails, with Power BI examples.

Learn to build and monitor data triggers from objects in Microsoft Fabric, using event streams, predefined properties, and reflexes to activate alerts and test email notifications.