DP-420 : Microsoft Azure Cosmos DB Exam Guide [Hands-on]

In this lecture, we take the first step in our journey to mastering Azure Cosmos DB by understanding its basics. We cover its history, core features, and why it stands out as a fully managed, distributed NoSQL and relational database. You'll learn about its architecture, key capabilities, and the advantages it offers for modern application development.

By the end of this lecture, you will have a clear understanding of what Azure Cosmos DB is and why it is a leading choice for scalable, low-latency, and serverless database solutions.

Key Highlights:

• Historical Evolution:

  • Originated as DocumentDB in 2015 with SQL support.

  • Rebranded as Azure Cosmos DB in 2017 with added cloud features and API support.

• Core Features:

  • Fully managed NoSQL and relational database.

  • Serverless architecture with no provisioning or maintenance required.

  • Multi-region distribution ensures low response time and high availability.

• Performance and Scalability:

  • Single-digit millisecond response time guaranteed.

  • Scales seamlessly to terabyte or petabyte levels.

  • Unlimited theoretical scalability for storage and compute.

• Developer Convenience:

  • Schema-less and no pre-defined indexing required.

  • SDKs available for popular languages like C#, .NET, Java, and Python.

• Advanced Capabilities:

  • Integration with Azure Synapse Link for analytics.

  • Supports multiple consistency levels (e.g., strong, eventual, bounded staleness).

• Enterprise-grade Reliability:

  • 99.999% SLA with ultra-low latency.

  • High business continuity and enterprise-level security.

• Open-source and API Support:

  • Supports APIs like MongoDB, Cassandra, Gremlin, and more, making it easy to lift and shift existing workloads.

In this lecture, we dive into the diverse APIs offered by Azure Cosmos DB, enabling developers to leverage familiar tools and frameworks while enjoying the scalability and performance of Cosmos DB. You'll learn how to choose the right API based on your application requirements and how to migrate legacy applications seamlessly to Cosmos DB.

We'll also explore real-world scenarios, such as working with different APIs like NoSQL, MongoDB, PostgreSQL, Cassandra, Gremlin, and Azure Table Storage, and how to set up your Cosmos DB account for optimal performance. By the end of this lecture, you'll be equipped with a clear understanding of API selection and its implications for development and migration.

Key Highlights:

• APIs Supported by Azure Cosmos DB:

  • Core API (NoSQL):

    • Flexible schema support with SQL-like query language.

    • Ideal for developers with SQL background for fast, flexible development.

  • MongoDB API: For document-oriented workloads.

  • PostgreSQL API: For relational and distributed applications.

  • Cassandra API: For column-family-based database applications.

  • Gremlin API: For graph-based database applications.

  • Azure Table Storage API: For legacy workloads using Azure Table Storage.

• Performance and Features:

  • Automatic scaling for storage and compute.

  • Disaster recovery and high availability with distributed architecture.

  • Low operational overhead and easy migration.

• Migration Scenarios:

  • Lift-and-shift approach for applications using MongoDB, Cassandra, Table Storage, or Gremlin.

  • Seamless migration by updating connection strings without rewriting backend code.

• API Selection Guidance:

  • For new applications: Use Azure Cosmos DB Core API (NoSQL) for flexibility and performance.

  • For legacy applications: Choose the API that matches your existing backend database.

In this lecture, we delve into Azure Cosmos DB's NoSQL capabilities and explore its relevance in modern application development. From understanding the need for NoSQL databases to examining the advantages of Azure Cosmos DB for NoSQL, this session offers a comprehensive overview of its document-based data model, JSON support, and globally distributed architecture. You’ll gain insights into how Azure Cosmos DB meets the demands of high-volume, high-velocity data processing and unpredictable application traffic.

By the end of this lecture, you'll understand why Azure Cosmos DB for NoSQL is a preferred choice for building scalable, real-time, and mission-critical applications.

Key Highlights:

• Why NoSQL Databases?

  • Handle high-volume, real-time, and diverse data from various sources.

  • Dynamic schema to accommodate evolving application needs.

  • Horizontally scalable for applications with rapid user growth.

• NoSQL Data Models Supported by Azure Cosmos DB:

  • Key-Value Pair: Ideal for simple, key-based lookups (e.g., Azure Table Storage).

  • Document-Based: Stores data in JSON format (e.g., MongoDB).

  • Graph-Based: Captures relationships between entities (e.g., Apache Gremlin).

  • Column-Family: Designed for column-oriented data (e.g., Apache Cassandra).

• Azure Cosmos DB for NoSQL:

  • First API supported by Cosmos DB.

  • Natively supports JSON documents for flexible, schema-less data storage.

  • Distributed database spanning multiple regions for global reach and low latency.

  • High-performance, scalable, and reliable with guaranteed SLAs.

  • Seamless integration with modern development environments.

• Advantages of Azure Cosmos DB for NoSQL:

  • Guaranteed speed at any scale with ultra-low latency.

  • Serverless, cost-effective, and fully managed architecture.

  • Ideal for mission-critical applications with high business continuity (99.999% SLA).

  • Supports unpredictable traffic spikes and dips with automatic scaling.

  • Enables fast, flexible app development with minimal operational overhead.

• When to Use Azure Cosmos DB for NoSQL:

  • Applications with unpredictable traffic patterns.

  • Scenarios involving real-time data generation and processing.

  • Projects requiring high business continuity and low latency.

In this lecture, we explore the key components of Azure Cosmos DB for NoSQL and understand the hierarchical structure of accounts, databases, containers, and items. These components form the building blocks of Azure Cosmos DB, each serving a unique purpose in scalability, organization, and data management.

By the end of this lecture, you’ll gain a clear understanding of these components and their roles, setting the stage for practical exercises in creating accounts, databases, and containers, and managing items within the Cosmos DB environment.

Key Highlights:

• Core Components of Azure Cosmos DB for NoSQL:

  1. Account:

     • The fundamental unit of distribution and high availability.

     • Required as the starting point for creating and managing databases.

  2. Database:

     • Logical grouping of containers, akin to a database in traditional SQL systems.

     • Useful for organizing and managing related containers, e.g., a "Student" database grouping multiple "tables."

  3. Container:

     • The fundamental unit of scalability and partitioning in Cosmos DB.

     • Comparable to a "table" in relational databases.

  4. Items:

     • The individual records stored in containers.

     • Stored in JSON format for schema-less and flexible data storage.

• Hierarchy Overview:

  • Account → Database → Container → Item

  • Accounts host databases, databases group containers, and containers store items/documents.

In this lecture of the DP-420: Microsoft Azure Cosmos DB Exam Guide [Hands-on] course, we explore the diverse use cases and scenarios where Azure Cosmos DB excels. By diving into practical examples and real-world architectures, you’ll understand how Azure Cosmos DB can be applied in various domains to handle modern-day data challenges. Below are the key points covered:

Key Use Cases of Azure Cosmos DB

1. Internet of Things (IoT) and Telemetry Applications

   • IoT applications generate massive amounts of real-time data from various sensors.

   • Architecture includes:

     • Azure IoT Hub: Collects real-time sensor data.

     • Azure Stream Analytics: Processes streaming data.

     • Azure Cosmos DB: Stores structured or unstructured data efficiently.

     • Azure Synapse Analytics: Performs advanced analytics on stored data.

   • Azure Cosmos DB acts as a highly scalable, low-latency storage solution in this pipeline.

2. Retail and Marketing Applications

   • Example: A web application hosted in Azure App Service with two types of backends:

     • Blob Storage: For serving unstructured data.

     • Azure Cosmos DB: For storing NoSQL data.

   • Azure Cosmos DB integrates seamlessly with other Azure services, enabling data retrieval for analytics, recommendations, or inventory management.

3. Web and Mobile Applications with Global Distribution

   • Azure Cosmos DB supports global distribution to ensure low latency and high availability.

   • Architecture includes:

     • Applications deployed in multiple regions (e.g., West US, East US).

     • Azure Cosmos DB instances in each region syncing with each other to form a single logical database.

     • Azure Traffic Manager: Distributes traffic across regions for optimal performance.

In this lecture of the DP-420: Microsoft Azure Cosmos DB Exam Guide [Hands-on] course, you will learn the step-by-step process of creating an Azure Cosmos DB account configured for NoSQL. This hands-on tutorial provides detailed insights into navigating the Azure portal and configuring key settings for Azure Cosmos DB. Below are the key points covered:

What You’ll Learn

• Navigating to the Azure Cosmos DB service in the Azure portal.

• Understanding and selecting key configuration options during account creation.

• Differences between capacity modes: Serverless and Provisioned Throughput.

• Exploring the Azure Cosmos DB free-tier benefits.

Key Steps Covered

1. Accessing Azure Cosmos DB

   • Multiple navigation options: Left navigation bar, Azure search bar, or Marketplace.

   • Exploring documentation and learning resources available within the Azure portal.

2. Selecting the API

   • Overview of available APIs: Core (SQL), MongoDB, Cassandra, Table, Gremlin, etc.

   • Choosing the Core (SQL) API for NoSQL operations.

3. Configuring Key Settings

   • Subscription: Associating the account with a specific subscription (e.g., Pay-As-You-Go).

   • Resource Group: Creating or selecting a resource group to organize resources logically.

   • Account Name: Ensuring a globally unique name for the account.

   • Region Selection: Provisioning in specific Azure regions (e.g., East US, Southeast Asia).

4. Understanding Capacity Modes

   • Provisioned Throughput:

     • Best for predictable workloads with fixed resource allocation.

     • Pricing based on pre-defined Request Units (RUs).

   • Serverless:

     • Ideal for unpredictable workloads, with pay-as-you-go pricing based on actual usage.

     • Comparison of serverless and provisioned throughput to identify the best fit for different scenarios.

5. Enabling Free Tier

   • Benefits of Azure Cosmos DB free-tier:

     • First 1,000 RUs free.

     • 25 GB of storage at no cost.

   • How to enable and utilize the free-tier benefits.

6. Applying Throughput Limits

   • Setting a cap on account throughput to avoid unexpected charges.

   • Flexibility to update or remove this limit post-creation.

In this lecture of the DP-420: Microsoft Azure Cosmos DB Exam Guide [Hands-on] course, you will delve into advanced configurations for setting up an Azure Cosmos DB account. This step-by-step guide covers crucial options such as global distribution, networking, backup policies, and encryption settings, providing a thorough understanding of how to tailor your Azure Cosmos DB environment to your specific needs. Below are the key topics discussed:

Key Topics Covered

1. Global Distribution Options

   • Geo-Redundancy:

     • Ensures high availability by replicating data to paired regions (e.g., East US to West US).

     • Replicated data in secondary regions is read-only.

   • Multi-Region Write:

     • Enables write operations in multiple regions.

     • Ensures data is consistent across all selected regions.

   • Availability Zones:

     • Replicates data across zones within a single region for improved resiliency.

2. Networking and Security

   • Connecting Azure Cosmos DB securely using:

     • Public Network Access: Allow connections from all networks.

     • Virtual Networks (VNet): Restrict access to specific VNets.

     • Private Endpoints: Enhance security by connecting privately.

   • Configuring firewalls to allow access only from trusted sources.

3. Backup Policy Configuration

   • Periodic Backups:

     • Configure backup intervals (e.g., every 6 hours) and retention periods (e.g., 15 days).

     • Retain multiple backup copies based on configuration.

   • Continuous Backups:

     • Options for 7-day (free) and 30-day (paid) continuous backups.

     • Restore data to any point within the backup window.

   • Storage redundancy options:

     • Locally Redundant Storage (LRS): Store backups within the same data center.

     • Zone-Redundant Storage (ZRS): Store backups across zones in the same region.

     • Geo-Redundant Storage (GRS): Store backups across regions.

4. Encryption Settings

   • Service-Managed Keys: Encryption handled automatically by Azure.

   • Customer-Managed Keys: Users provide and manage their encryption keys for added security.

5. Tagging Resources

   • Adding key-value tags to organize and manage resources effectively (e.g., environment: learning).

In this hands-on lecture, we demonstrate the process of creating an Azure Cosmos DB account with a Pay-As-You-Go subscription. By walking you through the key steps and configurations, you'll gain practical knowledge to set up your account effectively. This tutorial is ideal for learners preparing for the DP-420 certification or those new to Azure Cosmos DB.

Key Pointers Covered in This Lecture:

• Subscription and Billing:

  • Verifying your Pay-As-You-Go subscription.

  • Ensuring all outstanding bills are cleared before proceeding.

• Validation and Configuration:

  • Reviewing successful validation before account creation.

  • Examining the configurations, including resource scope, location, account name, and API selection.

• API Selection:

  • Highlighting the importance of API choice (e.g., SQL API, Gremlin API, Cassandra API).

  • Understanding that the API cannot be changed post-creation.

• Capacity Mode:

  • Configuring provisioned throughput for optimized performance.

• Tags and Connectivity:

  • Defining the environment tag for learning purposes.

  • Configuring connectivity options to allow access from all networks.

• Backup Policy:

  • Choosing a periodic backup policy for data protection.

• Deployment Process:

  • Observing the real-time deployment progress, which typically takes around 2 minutes.

In this lecture, we dive deep into the Azure Cosmos DB portal to explore the account you created in previous videos. This session provides an in-depth understanding of the key features, configurations, and tools available in the Azure Cosmos DB platform. By the end of this lecture, you’ll be well-versed with the essential elements of the Cosmos DB interface and how to navigate it effectively.

Key Pointers Covered in This Lecture:

• Account Overview:

  • Reviewing the configurations used during account creation, including resource group, subscription, API, and backup policy.

  • Observing the account status, location, and capacity mode.

• Activity Logs:

  • Understanding how to track actions and changes within your Azure Cosmos DB account.

  • Viewing logs for account creation, updates, and administrative activities.

• Access Control:

  • Assigning access permissions to other users via Azure Active Directory.

• Tags and Diagnostics:

  • Exploring assigned tags for easy management (e.g., environment = learning).

  • Utilizing diagnostic tools for troubleshooting and performance monitoring.

• UI Navigation:

  • Exploring key areas in the portal, including:

    • Data Explorer: Manage and view your data (to be covered in-depth later).

    • Notifications: Stay informed of alerts, recommendations, and updates.

    • Cost Management: Overview of expenses (detailed discussion in future sessions).

• Advanced Features:

  • Highlighting options like geo-redundancy, manual failover, and service-managed failover.

  • Understanding default consistency levels (e.g., Strong, Eventual).

• Backup and Networking:

  • Reviewing backup frequency and policy (e.g., periodic backup every 720 minutes).

  • Configuring network access options (e.g., allowing access from all networks).

• Integration Options:

  • Integrating with other Azure services such as Cognitive Search, Azure Functions, Synapse, and Power BI.

• Monitoring Tools:

  • Understanding database performance metrics, query insights, and request tracking.

This lecture provides a hands-on demonstration of creating your first database in Azure Cosmos DB. Learn how to configure database settings, manage provisioned throughput, and understand the implications of different capacity modes. By the end of this session, you'll have a thorough understanding of the database creation process and key options available in the Azure Cosmos DB portal.

Key Pointers Covered in This Lecture:

• Navigating to Data Explorer:

  • Accessing your recently created Azure Cosmos DB account.

  • Exploring the data exploration interface and its key functionalities.

• Connection Strings:

  • Understanding the primary and secondary connection strings.

  • Choosing between read-only and read-write modes for connecting to your database.

• Database Creation:

  • Creating databases with different configurations, such as:

    • Student Database (Student DB): Includes containers like StudentInfo, Courses, and Grades.

    • Library Database (Library DB): Includes containers like BookInfo and AuthorInfo.

    • Faculty Database (Faculty DB): Includes containers like Profile and Courses Taught.

• Provisioned Throughput:

  • Configuring provisioned throughput at the database level.

  • Understanding the implications of the free-tier limit (1000 RU/s).

  • Handling errors related to throughput exceeding the limit.

• Throughput Modes:

  • Manual Throughput: Assigning a fixed RU/s value for predictable performance.

  • Auto-Scale Throughput: Allowing throughput to scale dynamically based on database traffic.

• Cost Implications:

  • Understanding the costs associated with provisioned throughput.

  • Comparing hourly rates for manual and auto-scale configurations.

• Database Configuration Options:

  • Assigning capacity (e.g., CPU and IOPS) to databases and containers.

  • Differentiating between database-level and container-level throughput.

This lecture focuses on creating multiple databases in Azure Cosmos DB with different throughput configurations, explaining the implications of these settings on performance and cost. We demonstrate how to configure databases with provisioned throughput, shared throughput, and auto-scaling, providing a comprehensive understanding of their practical applications.

Key Pointers Covered in This Lecture:

• Database Creation Process:

  • Step-by-step walkthrough for creating databases using the Azure Cosmos DB UI.

  • Examples include creating databases for specific use cases like Student DB, Library DB, and Faculty DB.

• Provisioned vs. Shared Throughput:

  • Provisioned Throughput: Assigning fixed RU/s to a database for consistent performance.

  • Shared Throughput: Creating databases without dedicated throughput to utilize account-level RU/s.

• Auto-Scale Throughput:

  • Enabling auto-scaling to adjust throughput dynamically based on traffic, with minimum and maximum RU/s thresholds.

• Cost Management and Limits:

  • Understanding the free-tier limit of 1000 RU/s for the entire account.

  • Adjusting account-level throughput to accommodate additional database requirements.

  • Estimating costs based on throughput configurations.

• Database Examples:

  • Student DB: Configured with 1000 RU/s provisioned throughput.

  • Library DB: Created without provisioned throughput to leverage shared account-level throughput.

  • Faculty DB: Configured with provisioned throughput (e.g., 1000 RU/s) after increasing the account-level limit.

• Error Handling:

  • Addressing errors related to exceeding account-level throughput limits.

  • Adjusting account-level throughput dynamically to support additional database requirements.

• Key Observations:

  • Differences in functionality for databases with and without provisioned throughput.

  • Reviewing throughput allocation in the Azure portal under cost management.

In this lecture, we demonstrate how to create containers (equivalent to tables in relational databases) in Azure Cosmos DB and explore the implications of assigning provisioned throughput at various levels—account, database, and container. This hands-on session covers real-world configurations, addressing performance, cost, and scalability considerations.

Key Pointers Covered in This Lecture:

Container Creation: Basics and Concepts

• Navigating the Portal:

  • Accessing the Azure Cosmos DB account and exploring existing databases.

  • Using the "New Container" option to create containers.

• Understanding Key Components:

  • Container ID: Naming containers based on their use case (e.g., StudentInfo, Courses).

  • Partition Key: Specifying partition keys (e.g., sID or cID) for optimal data distribution and performance.

  • Unique Key: Maintaining data integrity by ensuring unique document entries.

Provisioned Throughput Configurations:

• Database-Level vs. Container-Level Throughput:

  • Sharing provisioned throughput across containers at the database level.

  • Assigning dedicated provisioned throughput at the container level for specific workloads.

• Auto-Scale and Manual Throughput:

  • Auto-scale throughput dynamically adjusts between 10% and the maximum assigned RU/s based on traffic.

  • Manual throughput assigns a fixed RU/s value, ensuring consistent performance and cost predictability.

Container Examples:

• Student DB Containers:

  • Creating containers such as StudentInfo, Courses, and Grades.

  • Exploring shared and dedicated throughput configurations.

• Library DB Containers:

  • Demonstrating container creation in a database without provisioned throughput.

  • Assigning throughput directly at the container level.

• Faculty DB Containers:

  • Balancing shared and dedicated throughput across multiple containers, such as FacultyProfiles and CoursesTaught.

Cost Management and Resource Allocation:

• Observing throughput allocation and usage across databases and containers.

• Managing the total provisioned throughput at the account level.

• Addressing errors related to exceeding throughput limits and adjusting account-level settings accordingly.

In this lecture, we dive into the practical aspects of inserting, managing, and querying items in Azure Cosmos DB containers. You will learn how to add new items, update existing records, delete items, and filter data using simple queries. This session also explains how Azure Cosmos DB auto-generates IDs and manages system properties, offering insights into effective data handling.

Key Pointers Covered in This Lecture:

Adding Items to Containers:

• Accessing the Data Explorer:

  • Navigating to the container (e.g., StudentInfo) within the database.

  • Understanding the layout of the Data Explorer for inserting and managing items.

• Inserting Items:

  • Adding new items with fields such as ID, Name, Age, and custom properties like StudentID (sID).

  • Observing auto-generated unique IDs (ID) by Azure Cosmos DB for each document.

• Partition Key Usage:

  • Using partition keys (e.g., sID) to optimize data distribution and indexing.

  • Differentiating between partition keys and unique document IDs.

Managing Items:

• Updating Items:

  • Editing existing records by modifying properties (e.g., changing Age from 23 to 18).

  • Saving changes and verifying updates in the container.

• Deleting Items:

  • Removing specific items from the container using the delete option.

Querying Data:

• Filtering Data:

  • Performing simple queries to filter documents (e.g., WHERE sID = '05').

  • Understanding how queries return results based on the specified filter criteria.

Key Observations:

• System-Generated Properties:

  • Exploring additional system-generated fields like etag, self, and timestamps.

  • Leveraging auto-generated IDs for document management and ensuring uniqueness.

• Error Handling:

  • Addressing issues while creating or updating items with incorrect or missing fields.

Cost Management:

• Limiting Provisioned Throughput:

  • Setting the account throughput to 1000 RU/s for cost-effective usage.

  • Monitoring throughput usage and ensuring the operations stay within the free tier.

This lecture explains the concept of Request Units (RUs) in Azure Cosmos DB, the fundamental measure of system resources consumed by database operations. Learn how RUs represent a combination of CPU, memory, and IOPS, how they are used across different APIs, and their importance in configuring and managing Azure Cosmos DB accounts efficiently. The session provides practical examples, analogies, and calculations to solidify your understanding.

Key Pointers Covered in This Lecture:

What Are Request Units (RUs)?

• Definition of RUs:

  • A unit of measurement representing system resources (CPU, memory, and IOPS) required for database operations.

  • Abstracted to simplify resource management in Azure Cosmos DB.

• Analogy for Understanding RUs:

  • Similar to measuring length in kilometers, meters, or centimeters; RUs measure resources consumed for operations like reads, writes, and queries.

RUs Across APIs:

• Applicable across all Azure Cosmos DB APIs, such as:

  • NoSQL API

  • MongoDB API

  • Gremlin API

  • Table Storage API

• Regardless of the API, resource usage is always measured in RUs.

Operational Insights:

• RUs for Basic Operations:

  • Reading a 1 KB item consumes 1 RU.

  • Insert, update, and delete operations consume variable RUs based on complexity.

  • Complex queries require higher RUs depending on filters, sorting, and aggregations.

• Configuring RUs:

  • Configurable at account, database, and container levels.

  • RUs cannot be assigned to specific regions, but they can be scaled across containers and databases.

Examples and Calculations:

• Write Operation:

  • Each write request may consume 10 RUs.

  • For 10,000 write requests per second, the total RU requirement is 100,000 RUs per second.

• Complex Queries:

  • Example of a query consuming 100 RUs per request with 700 requests per second:

    • Total RUs = 700 × 100 = 70,000 RUs per second.

• Application-Wide Calculation:

  • Combining multiple operations and queries to estimate total RU requirements for an application.

Managing RUs in Azure Cosmos DB:

• Free Tier Limit:

  • The free tier provides 1000 RUs and 25 GB of storage.

  • Users can configure limits to avoid exceeding the free tier.

• Scaling RUs:

  • Scaling options include manual or auto-scale configurations at the container and database levels.

In this lecture, "Understanding Throughput in Azure Cosmos DB", we dive into the concept of throughput, its significance, and how to provision it effectively for your Azure Cosmos DB accounts. Through this lecture, you'll gain a clear understanding of throughput's role in scaling and managing performance for your applications.

Key Points Covered:

• What is Throughput?

  • Throughput as the speed or velocity of handling requests per second.

  • Defined as a unit of measurement for consumed system resources.

• Relationship Between Throughput and Application Needs:

  • Connection with CPU and memory capacity.

  • Dependency on request processing power.

• Provisioning Throughput in Azure Cosmos DB:

  • At the Database Level:

    • Total RU/s (Request Units per second) shared among all containers.

  • At the Container Level:

    • Individual RU/s allocation for each container, enabling specific scalability.

  • Mixed Strategy:

    • Combining database-level provisioning with dedicated container-level throughput.

• Scalability in Azure Cosmos DB:

  • Scaling throughput by increasing RUs at the container or database level.

  • Real-life scenarios to optimize throughput allocation.

• Hands-On Demonstration Recap:

  • How to define throughput at database, container, or mixed levels using Azure's interface.

  • Overview of Azure's Capacity Calculator for estimating RU/s needs.

In this lecture, "Cost Management and Capacity Planning in Azure Cosmos DB", we explore how to effectively estimate and manage throughput costs for Azure Cosmos DB using the Cosmos DB Capacity Calculator. This lecture provides practical insights into defining throughput requirements and optimizing costs while configuring your Cosmos DB accounts, databases, and containers.

Key Points Covered:

1. Understanding Throughput and Cost Management:

   • Introduction to throughput as a measure of system resources consumed per second.

   • Relation between throughput, application performance, and cost.

2. Cosmos DB Capacity Calculator:

   • Overview of the Calculator:

     • Located at cosmos.azure.com/capacitycalculator.

     • Allows estimation of required Request Units (RUs) based on workload.

   • Input Parameters:

     • Storage size (e.g., 10 GB).

     • Read, write, delete, update operations per second.

     • Item size (data size per operation).

   • Output:

     • Estimated RU requirements and associated costs.

     • Storage cost per GB and throughput cost per second.

3. Hands-on Example:

   • Workload simulation with inputs like 50 reads/sec, 10 writes/sec, etc.

   • Example scenarios:

     • Small data items (~1 KB): Minimum throughput required ~400 RUs.

     • Larger data items (~2 KB): Adjusted throughput requirements.

4. Throughput Configuration Options:

   • Account Level:

     • Free tier: 1000 RUs/sec with 25 GB storage.

     • Option to increase or remove throughput limits.

   • Database Level:

     • Shared throughput across containers within the database.

   • Container Level:

     • Dedicated throughput configuration for specific containers.

5. Practical Considerations:

   • Differentiating between account-level, database-level, and container-level configurations.

   • Strategies to balance cost and performance:

     • Use shared throughput for cost-saving.

     • Opt for dedicated throughput for critical containers requiring consistent performance.

In this lecture, "Horizontal Scalability in Azure Cosmos DB", we explore how Cosmos DB efficiently handles data growth through horizontal scaling. This lecture delves into the mechanics of data distribution across multiple physical machines and highlights how Cosmos DB ensures seamless scalability without manual intervention.

Key Points Covered:

1. Understanding Horizontal Scalability:

   • Horizontal scalability refers to the ability to distribute data across multiple physical machines as storage capacity grows.

   • Key benefit: Seamless handling of large datasets without impacting performance.

2. How Horizontal Scaling Works in Cosmos DB:

   • Data Distribution:

     • When a container reaches the maximum capacity on one physical machine, additional data is automatically stored on another machine.

     • Azure Cosmos DB takes responsibility for distributing and managing data across machines.

   • Example Scenario:

     • Initial storage: 1 million records on a single machine.

     • Additional 1 million records: Automatically stored on a new physical machine.

   • Scalability is transparent to the user, requiring no manual configuration.

3. Microsoft's Role in Scaling:

   • Azure Cosmos DB ensures data consistency and redundancy by maintaining copies of data across machines.

   • Users focus on inserting data at the container level while Azure handles underlying infrastructure.

4. Key Advantages:

   • Unlimited storage scalability for containers.

   • Reduced operational complexity for database scaling.

   • Consistent performance, even as data grows.

5. Takeaways for Developers:

   • Horizontal scaling in Cosmos DB is automatic and efficient.

   • Users need to focus on container-level configurations, and Azure handles the rest.

   • Ideal for applications with high data growth requirements.

In this lecture, "Understanding Partitions and Partition Keys in Azure Cosmos DB", we explore the concept of partitions and partition keys, their significance, and how they improve query performance in Cosmos DB. By understanding these core elements, learners will be equipped to design scalable and efficient database structures.

Key Points Covered:

1. What is a Partition?

   • A mechanism to divide data into smaller, manageable chunks.

   • Improves query performance by limiting the search scope to relevant data.

2. How Partitions Work:

   • Data is divided into logical partitions based on a partition key.

   • Logical partitions group related data together, e.g., all records with city = London in one partition.

   • Logical partitions reside on one or multiple physical machines, ensuring scalability.

   • A single logical partition cannot span multiple machines.

3. What is a Partition Key?

   • A partition key is a criterion for dividing data into partitions.

   • Example: Choosing city as the partition key groups all records with the same city into one partition.

   • Helps optimize queries by narrowing the search scope to relevant partitions.

4. Benefits of Using Partitions:

   • Enhanced query performance: Queries target specific partitions based on the partition key.

   • Improved scalability: Data is distributed across machines seamlessly.

   • Efficient data organization: Logical grouping of related data simplifies management.

In this lecture, "Single Partition vs. Cross Partition Queries in Azure Cosmos DB", we examine the impact of partitioning on query performance. The discussion focuses on single-partition queries, cross-partition queries, and how the choice of partition key influences database efficiency.

Key Points Covered:

1. What is a Single Partition Query?

   • A query that targets a specific logical partition.

   • Example: Using username as the partition key, all data for John is stored in a single partition.

   • Advantages:

     • Faster query performance.

     • Reduced resource consumption since only one partition is searched.

2. What is a Cross Partition Query?

   • A query that spans multiple logical partitions.

   • Example: A query filtering by location when the partition key is username.

     • Data for John, Sara, and others must be searched across all partitions.

   • Challenges:

     • Higher resource usage due to the need to "fan out" across all partitions.

     • Slower query performance compared to single-partition queries.

3. Impact of Partition Key Selection:

   • Selecting an optimal partition key is critical for query performance.

   • A poorly chosen partition key can lead to more cross-partition queries, increasing costs and reducing efficiency.

   • Best Practices:

     • Choose a key that aligns with frequent query patterns.

     • Ensure even data distribution across partitions to avoid hotspots.

4. Comparing Single vs. Cross Partition Queries:

   • Single Partition Query:

     • Efficient and resource-friendly.

     • Ideal for queries targeting a specific partition.

   • Cross Partition Query:

     • Involves searching multiple partitions, consuming more Request Units (RUs).

     • Should be minimized wherever possible.

5. Practical Example:

   • Single Partition Query: Searching all data for username = John.

   • Cross Partition Query: Searching data where location = United States when the partition key is username.

In this lecture, "Understanding and Avoiding Hot Partitions in Azure Cosmos DB", we discuss the concept of hot partitions, their negative impact on query performance, and strategies to design effective partition keys to prevent uneven data and query distribution.

Key Points Covered:

1. What is a Hot Partition?

   • A hot partition occurs when a significant portion of data or queries is concentrated in a single logical partition.

   • Leads to:

     • Overutilization of assigned throughput for one partition.

     • Underutilization of other partitions, resulting in wasted resources.

2. Impact of Hot Partitions on Performance:

   • Query Performance:

     • Overloaded partitions experience delays in query processing.

   • Throughput Imbalance:

     • Example: A container with 10,000 RUs/sec evenly distributes 2,500 RUs/sec per partition.

     • If the partition for London requires 5,000 RUs/sec, while others are underused, the system suffers from inefficiency.

3. Causes of Hot Partitions:

   • Poor partition key choice:

     • Example: Using current time as a partition key for a shopping cart.

       • Most data goes to recent time partitions, creating a hot partition.

   • Uneven data distribution:

     • Example: A city-based key (London, Paris) where one city has a disproportionate number of records or queries.

4. Characteristics of an Ideal Partition Key:

   • Distributes data evenly across partitions to balance storage and throughput.

   • Aligns with application query patterns to avoid cross-partition queries.

   • Examples of good partition keys:

     • User ID or Product ID in e-commerce applications.

     • Geographical region for location-based queries.

5. Best Practices to Avoid Hot Partitions:

   • Analyze workload patterns:

     • Understand data distribution and query frequency.

   • Domain expertise:

     • Leverage domain knowledge to select partition keys that align with application usage.

   • Test and iterate:

     • Use real-world scenarios to validate the effectiveness of your partitioning strategy.

   • Avoid bad keys:

     • Avoid time-sensitive or frequently accessed static values as partition keys.

6. Examples and Scenarios:

   • Poor Partition Key: Using current time in a shopping cart results in most activity focusing on recent time partitions.

   • Good Partition Key: Using User ID ensures distributed activity across partitions.

In this lecture, "Time-to-Live (TTL) in Azure Cosmos DB", we explore the TTL property, which allows you to manage the lifecycle of data in Cosmos DB by automatically deleting expired items. This feature is essential for maintaining efficient storage usage, adhering to data retention policies, and reducing query costs.

Key Points Covered:

1. What is Time-to-Live (TTL)?

   • TTL defines the lifespan of documents in a database.

   • Automatically deletes items after a specified duration since their last modification.

2. Where Can TTL Be Configured?

   • Container Level: Applies to all items within the container.

   • Item Level: Configurable individually for specific items, overriding container-level TTL.

3. How TTL Works:

   • A countdown begins after the last modification of an item.

   • Example: If TTL is set to 10 seconds:

     • After 7:00:00, with no further changes, the item will be purged at 7:00:10.

   • Deletion occurs as a background operation, consuming available RUs (Request Units).

4. Configuring TTL:

   • Off: No TTL is enforced.

   • On (No Default): Enabled but requires manual configuration at the item level.

   • On (With Default): TTL is defined at the container level and applies to all items.

5. Advantages of TTL:

   • Automated Data Management:

     • Removes obsolete data, ensuring the database remains uncluttered.

   • Cost Optimization:

     • Saves on storage costs by automatically purging unnecessary data.

   • Data Retention Compliance:

     • Enforces retention policies for regulatory or business requirements.

   • Improved Query Performance:

     • Smaller datasets result in faster queries.

6. Practical Scenarios for TTL:

   • Temporary data, such as session states or cache entries, can be automatically deleted.

   • Regulatory compliance, e.g., ensuring data is retained only for a specific duration.

7. Hands-on Demonstration:

   • Steps to Configure TTL:

     • Navigate to the Data Explorer in the Azure portal.

     • Select the container and go to Settings.

     • Choose one of the three TTL options:

       • Off.

       • On (No Default): Enables item-level TTL configuration.

       • On (With Default): Define a default TTL value for the container (e.g., 10 seconds).

     • Save and observe items being deleted after the specified duration.

In this lecture, "Serverless Mode in Azure Cosmos DB", we explore the serverless capacity mode, which offers a pay-as-you-go model for database operations. This lecture highlights the benefits, use cases, and practical steps to configure serverless mode in Cosmos DB, ensuring learners understand its suitability for various application scenarios.

Key Points Covered:

1. What is Serverless Mode?

   • A consumption-based model: Pay only for what you use.

   • Eliminates the need to pre-provision throughput (RUs) in advance.

   • Billing is based on the total Request Units (RUs) consumed by database operations.

     • Example: Use 3,000 RUs → Pay for 3,000 RUs.

     • Use 10 million RUs → Pay for 10 million RUs.

1. How to Configure Serverless Mode:

   • While creating a Cosmos DB account:

     1. Select the Serverless Capacity Mode during account setup.

     2. No additional configuration is needed for throughput.

   • Serverless mode is simplified: No pre-defined throughput levels, scaling happens automatically.

1. When to Use Serverless Mode:

   • Unpredictable Traffic Patterns:

     • Ideal for applications where user activity varies widely (e.g., holiday surges, event-driven traffic).

   • Prototype Applications:

     • Best for testing or developing new apps without predefined load requirements.

   • Low Traffic Applications:

     • Cost-effective for workloads with minimal or occasional traffic.

   • Uncertain RU Forecasts:

     • Suitable when it’s hard to estimate usage patterns or database load.

1. When to Avoid Serverless Mode:

   • High-Traffic Applications:

     • Provisioned throughput is more cost-efficient for predictable, high-volume workloads.

   • Applications with consistent, stable traffic patterns.

1. Advantages of Serverless Mode:

   • Simplifies cost management with pay-per-use billing.

   • Reduces waste: No unused or over-provisioned RUs.

   • Ideal for applications with spiky or seasonal workloads.

1. Practical Example:

   • A website with fluctuating traffic:

     • December holiday season → High traffic → Automatic scaling of RUs.

     • January off-season → Minimal traffic → Reduced RU consumption and costs.

1. Comparing Serverless and Provisioned Throughput:

   • Serverless Mode:

     • Flexible, pay-as-you-go model.

     • No upfront throughput provisioning.

   • Provisioned Throughput:

     • Predefined RU levels offer predictability and cost efficiency for high, stable traffic.

In this lecture, we dive deep into the differences between Provisioned Throughput Mode and Serverless Mode in Azure Cosmos DB. Understanding these modes is critical for optimizing cost and performance based on your workload requirements. This lecture covers:

Key Points:

1. When to Choose Serverless Mode:

   • Ideal for unpredictable traffic patterns.

   • No need for advance throughput planning or provisioning.

   • Automated and dynamic handling of resource allocation.

   • Limited to single Azure region.

   • Maximum storage limit of 50 GB per container.

2. When to Choose Provisioned Throughput Mode:

   • Best suited for predictable workloads where traffic and usage are well understood.

   • Advance provisioning of Request Units (RUs) is required.

   • Supports global distribution with replication across multiple Azure regions.

   • Unlimited storage capacity.

3. Configuration Differences:

   • Provisioned throughput needs explicit configuration at the container level, database level, or account level.

   • Serverless mode simplifies this process by automatically scaling resources based on demand.

4. Practical Demonstration:

   • A walkthrough on how to configure and compare these modes using the Azure portal.

   • Demonstration includes:

     • Viewing an existing Cosmos DB account with provisioned throughput.

     • Navigating the Data Explorer and scaling settings.

     • Creating a new Cosmos DB account and selecting the appropriate mode.

5. Impact on Pricing:

   • A preview of how the choice between these modes affects pricing and cost optimization (detailed pricing to be covered in a subsequent lecture).

This lecture explains the two modes of provisioning throughput in Azure Cosmos DB's Provisioned Throughput Mode: Auto Scale and Manual (Standard). Understanding these modes is essential to optimize cost and performance for workloads with varying traffic patterns.

Through hands-on demonstrations in the Azure portal, this lecture covers:

Key Points:

1. Provisioned Throughput Modes Overview:

   • Auto Scale Mode:

     • Dynamically adjusts throughput based on traffic demand.

     • Starts with a base billing of 10% of the maximum provisioned RU/s.

     • Ideal for less predictable workloads where usage fluctuates below 66% of the max throughput.

   • Manual Mode (Standard):

     • Requires explicit configuration of throughput at database or container level.

     • Billing is fixed at the configured RU/s, irrespective of utilization.

     • Suitable for steady workloads where utilization exceeds 66% of the provisioned capacity.

2. Key Differences Between Auto Scale and Manual Mode:

   • Flexibility: Auto Scale adjusts dynamically, whereas Manual mode remains static.

   • Cost Efficiency:

     • Auto Scale minimizes cost for underutilized workloads by billing only for what’s used.

     • Manual mode incurs costs for the full provisioned capacity, regardless of usage.

   • Use Cases:

     • Auto Scale: Ideal for workloads with unpredictable traffic patterns.

     • Manual Mode: Best for highly predictable traffic with consistent usage patterns.

3. Scenarios for Selecting Modes:

   • If your workload is steady and predictable, choose Manual Mode with the appropriate throughput to match your usage.

   • If your workload is dynamic or fluctuating, opt for Auto Scale Mode to handle demand spikes without overpaying for unused capacity.

4. Hands-On Demonstration:

   • How to configure Auto Scale and Manual throughput modes in the Azure portal.

   • Setting up throughput at database level and adjusting the RU/s limits.

   • Understanding billing implications at container and database levels.

   • Practical examples of creating a database with Auto Scale mode starting at 10% billing, and Manual mode with fixed throughput.

5. Best Practices:

   • For workloads below 66% of max RU/s utilization, Auto Scale provides better cost-efficiency.

   • For workloads consistently above 66% utilization, Manual provisioning ensures optimal performance at predictable costs.

In this lecture, we explore the pricing models in Azure Cosmos DB, a critical topic for architects and data engineers aiming to optimize costs. The lecture provides an in-depth comparison of the Serverless Mode and Provisioned Throughput Mode pricing structures, along with practical demonstrations using the Azure Pricing Calculator.

Through detailed scenarios and calculations, you will learn to forecast and manage your Azure Cosmos DB costs effectively.

Key Points:

1. Azure Cosmos DB Pricing Models Overview:

   • Serverless Mode:

     • Pay-as-you-go model, perfect for low and unpredictable traffic patterns.

     • Costs are calculated per 1 million Request Units (RUs), e.g., $0.25 per 1M RUs in East US.

     • Limited to a single Azure region.

     • Example: 10M RUs = $2.50/month.

   • Provisioned Throughput Mode:

     • Reserved capacity with predictable costs, ideal for steady or high traffic workloads.

     • Two sub-modes:

       • Manual Provisioning: Static RUs billed regardless of utilization.

       • Auto Scale Provisioning: Dynamically adjusts based on traffic, starting at 10% of max RUs.

     • Example: 400 RUs reserved for 730 hours = $23.36/month.

2. Storage Costs:

   • Uniform across both modes:

     • $0.25 per GB/month for storage.

     • Example: 10 GB = $2.50/month.

3. Backup Costs:

   • Periodic backups: $1.20 per copy.

   • Costs vary based on the number of copies and backup type.

4. Practical Demonstration:

   • Using the Azure Pricing Calculator to simulate costs:

     • Adding Azure Cosmos DB as a service.

     • Configuring parameters like RUs, regions, and storage.

     • Comparing pricing for Serverless, Manual, and Auto Scale modes.

5. Use Cases and Recommendations:

   • Serverless Mode:

     • Best for applications with low or sporadic traffic.

     • Ideal when traffic spikes are rare or unpredictable.

   • Provisioned Throughput (Manual):

     • Suitable for steady workloads with consistent usage above 66% of provisioned capacity.

   • Provisioned Throughput (Auto Scale):

     • Perfect for workloads with variable traffic requiring dynamic scaling.

     • Cost-effective when traffic consistently fluctuates below 66% of maximum RUs.

6. Cost Optimization Best Practices:

   • Use the Azure Pricing Calculator to forecast monthly costs based on your workload.

   • Select Serverless Mode for low traffic to save costs.

   • For steady, high-volume workloads, choose Manual Provisioning with appropriate capacity.

   • Opt for Auto Scale Mode when traffic varies significantly but stays within a defined range.

This lecture provides a comprehensive walkthrough on how to download, install, and set up Visual Studio 2022 on a Windows operating system. The process includes selecting the right edition, configuring installation settings, and preparing the environment for development.

Whether you are a beginner or an experienced developer, this guide ensures you get started with Visual Studio quickly and efficiently.

Key Points:

1. Choosing the Correct Edition:

   • Visual Studio offers three editions:

     • Community Edition: Free and suitable for individual developers and small teams.

     • Professional Edition: Paid version for small-to-medium-sized teams.

     • Enterprise Edition: Paid version for large-scale teams with advanced features.

   • For this lecture, the Community Edition was selected for demonstration.

2. Downloading the Installer:

   • Navigate to the Visual Studio download page.

   • Select the desired edition (Community Edition in this case) and download the setup file.

3. Starting the Installation:

   • Run the downloaded setup file.

   • Configure the installation by selecting the required workloads (e.g., ASP.NET and web development).

   • Verify available disk space on the installation drive (C Drive in this case).

4. Installation Options:

   • Choose between:

     • Install while downloading: Starts installation as files are downloaded.

     • Download all and install: Downloads all files first and then begins the installation.

5. Installation Process:

   • The installation includes downloading prerequisites and the selected workloads (e.g., 11.23 GB for the selected components).

   • Depending on your internet speed, downloading may take 5–10 minutes or more.

   • Once downloaded, installation packages are processed, which may take additional time.

6. Post-Installation Setup:

   • Launch Visual Studio after installation is complete.

   • Customize settings:

     • Choose a development theme (e.g., Dark Theme).

     • Select a personalized development environment (e.g., General Development Settings).

7. Launching Visual Studio:

   • Open Visual Studio 2022 from the Start Menu.

   • Skip optional sign-in for now and proceed to the main interface.

8. Cross-Platform Availability:

   • Visual Studio is also available for Linux and macOS. Users of these platforms can follow a similar installation process with platform-specific installers.

In this lecture, we demonstrate how to connect an Azure Cosmos DB account to a .NET application using Visual Studio. This hands-on session covers everything from creating a new .NET console application to establishing a secure connection with Azure Cosmos DB via SDKs.

By the end of this lecture, you'll have a functional .NET application that interacts with Azure Cosmos DB and retrieves account details, setting the foundation for further operations like database and container creation.

Key Points:

1. Setting Up the .NET Project:

   • Create a new Console Application in Visual Studio.

   • Configure the project location and set the framework to .NET 6.0 (LTS).

   • Adjust font size for better visualization in Visual Studio.

2. Installing Required Packages:

   • Use NuGet Package Manager to install the Azure Cosmos DB SDK.

   • Verify successful installation of the required libraries.

3. Preparing Azure Cosmos DB:

   • Log in to the Azure Portal and navigate to the Cosmos DB account.

   • Retrieve the Primary Connection String and Endpoint from the Keys section for authentication.

4. Writing Connection Code:

   • Define variables for endpoint and key in the project.

   • Create a Cosmos DB client using the CosmosClient class.

   • Authenticate with Azure Cosmos DB using the endpoint and key.

5. Reading Account Information:

   • Use asynchronous methods (e.g., ReadAccountAsync) to fetch account properties.

   • Print properties like Account ID and Readable Region to the console.

   • Example output:

     • Account ID: CosmosDBAccount

     • Region: East US

6. Handling Asynchronous Operations:

   • Use the await keyword for asynchronous calls to handle network latency effectively.

   • Wrap asynchronous calls in error-handling logic to debug issues.

7. Testing the Connection:

   • Run the application to verify that it successfully connects to Azure Cosmos DB.

   • Ensure the account details are retrieved and printed correctly.

In this lecture, you'll learn how to create a new database in Azure Cosmos DB programmatically using .NET code. This hands-on session demonstrates a step-by-step approach, including the necessary code snippets, to efficiently manage database creation.

What You'll Learn:

• How to delete an existing database to free up provisioned throughput.

• How to use the Azure Cosmos DB Data Explorer to verify database changes.

• Writing .NET code to create a new database using CreateDatabaseIfNotExistsAsync.

• Understanding the database provisioning process and validating the output in the Azure portal.

Key Points Covered:

1. Setting Up the Environment:

   • Navigate to the Azure portal and access the Cosmos DB account.

   • Review the available provisioned throughput and manage database limits.

2. Deleting Existing Database:

   • Use Azure Data Explorer to delete an existing database.

   • Free up resources to create a new database.

3. Writing Code for Database Creation:

   • Use the .NET SDK client to programmatically create a database.

   • Implement CreateDatabaseIfNotExistsAsync to ensure idempotent operations.

   • Understand the significance of the Database ID returned by the operation.

4. Verifying Database Creation:

   • Run the application to confirm successful database creation.

   • Validate the results using the Azure portal's Data Explorer.

In this lecture, you’ll learn how to create a container inside an Azure Cosmos DB database programmatically using .NET code. This step-by-step tutorial will guide you through writing efficient code to set up containers, define partition keys, and configure throughput—all crucial components for optimizing your Cosmos DB setup.

What You’ll Learn:

• How to create containers inside an Azure Cosmos DB database programmatically.

• Using CreateContainerIfNotExistsAsync to ensure idempotent container creation.

• Setting up partition keys and throughput manually via code.

• Verifying container creation and settings through the Azure portal.

Key Points Covered:

1. Prerequisite – Database Setup:

   • Ensure the database (e.g., Cosmic Works) is already created.

   • Reuse or initialize the database object using the CreateDatabaseIfNotExistsAsync method.

2. Container Creation Code:

   • Use the database object to call CreateContainerIfNotExistsAsync.

   • Specify the container name (e.g., Products) and the partition key (e.g., Category ID).

   • Define the throughput settings, such as manual provisioning of 400 RUs.

3. Running the Code:

   • Execute the application to create the container.

   • Confirm successful execution by checking for the returned container ID.

4. Verification in Azure Portal:

   • Navigate to the Azure portal and confirm the Products container has been created.

   • Review the container’s Scale and Settings to validate the defined throughput and partition key.

In this lecture, you’ll learn how to manage documents in Azure Cosmos DB using .NET. This includes creating, reading, updating, and deleting (CRUD) documents programmatically. You'll follow a hands-on approach to implement these operations using simple and effective code, as well as validate changes through the Azure portal.

What You’ll Learn:

• Create: Insert new documents into a Cosmos DB container.

• Read: Retrieve documents based on partition keys and document IDs.

• Update: Modify existing documents with updated data.

• Delete: Remove documents from a container.

Key Points Covered:

1. Prerequisite – Database and Container Setup:

   • Ensure the Cosmos DB account, database, and container (e.g., Products) are already created.

   • Understand how to reuse container objects for CRUD operations.

2. Creating a Document:

   • Define a product object with fields such as ID, CategoryID (partition key), Name, Price, and Tags.

   • Use CreateItemAsync to insert the product into the container.

   • Validate the document creation in the Azure portal.

3. Reading a Document:

   • Retrieve a document using ReadItemAsync by specifying the Partition Key and Document ID.

   • Display key details such as ID, Name, and Price in the console.

4. Updating a Document:

   • Modify properties of an existing document (e.g., update Price and Name).

   • Use UpsertItemAsync to apply updates.

   • Confirm the changes by refreshing the Azure portal.

5. Deleting a Document:

   • Remove a document from the container using DeleteItemAsync.

   • Provide the Document ID and corresponding Partition Key for deletion.

   • Verify deletion by refreshing the portal and checking the container.

In this lecture, you’ll learn how to insert multiple documents into an Azure Cosmos DB container using a Transactional Batch in .NET. This hands-on session demonstrates how to efficiently manage batch operations for documents with the same partition key, along with a discussion of potential limitations when working with multiple partition keys.

What You’ll Learn:

• Create and insert multiple documents into a Cosmos DB container.

• Use Transactional Batch to perform batch operations programmatically.

• Understand partition key requirements for transactional batch operations.

• Handle errors such as mismatched partition keys during batch insertion.

Key Points Covered:

1. Setting Up the Environment:

   • Reuse the container object from the previous lecture for batch operations.

   • Ensure the container and database are already created.

2. Document Creation:

   • Define multiple objects (e.g., Saddle and Handlebar) with common partition keys.

   • Implement constructors in the Product class to simplify object creation.

3. Using Transactional Batch:

   • Initialize the transactional batch using the CreateTransactionalBatch method.

   • Add multiple objects (e.g., products) to the batch using CreateItemAsync.

   • Execute the batch operation asynchronously to insert multiple items in one transaction.

4. Validating the Results:

   • Verify successful insertion by checking the HTTP Status Code (OK).

   • Refresh the Azure portal to confirm the documents were added to the container.

5. Handling Partition Key Constraints:

   • Explore a scenario with different partition keys for the documents.

   • Understand why Transactional Batch operations require all items to share the same partition key.

   • Observe and handle errors (e.g., Bad Request) caused by partition key mismatches.

6. Preparing for Bulk Insertion:

   • Discuss limitations of inserting a small number of documents manually.

   • Introduce the concept of generating random data for bulk insertion (covered in the next lecture).

In this lecture, you’ll learn how to perform bulk insertion of documents into an Azure Cosmos DB container using .NET. The lecture focuses on efficiently inserting thousands of records asynchronously using the Bulk Execution feature, along with tools like the Bogus library for generating sample data.

What You’ll Learn:

• How to enable Bulk Execution for Azure Cosmos DB client.

• Using the Bogus library to generate random data for bulk insertion.

• Implementing bulk insertion using asynchronous operations and concurrent tasks.

• Verifying bulk data insertion using queries in the Azure portal.

Key Points Covered:

1. Enabling Bulk Execution:

   • Configure the CosmosClientOptions to set AllowBulkExecution to true.

   • Pass the configured options while initializing the Cosmos DB client to enable bulk operations.

2. Generating Sample Data:

   • Use the Bogus library to create realistic sample data for testing.

   • Define a Product class with fields like ID, Name, Price, and CategoryID.

   • Generate 1,000 sample products dynamically.

3. Bulk Insertion Process:

   • Prepare a list of tasks for inserting documents using CreateItemAsync.

   • Use concurrent tasks to batch the insertion process for efficiency.

   • Execute all tasks using Task.WhenAll to perform bulk insertion asynchronously.

4. Handling Errors:

   • Resolve common issues such as missing parameterless constructors.

   • Debug and refine the insertion code to ensure smooth execution.

5. Validating Bulk Insertion:

   • Navigate to the Azure portal to verify the inserted documents in the container.

   • Run a count query in Data Explorer to confirm the total number of records (e.g., SELECT COUNT(1) FROM c).

6. Performance Insights:

   • Understand how the Bulk Execution feature optimizes the insertion process.

   • Learn how asynchronous operations improve performance by processing multiple documents in parallel.

In this lecture, you'll learn how to interact with data stored in Azure Cosmos DB containers by writing SQL queries. This session covers basic query operations, aliasing, filtering, and retrieving specific fields to extract meaningful insights efficiently from your database.

What You’ll Learn:

• Navigate the Azure Cosmos DB Data Explorer for query execution.

• Write SQL queries to retrieve data from Cosmos DB containers.

• Apply filters and conditions to retrieve specific records.

• Use aliasing for simplified and efficient querying.

Key Points Covered:

1. Setting Up the Environment:

   • Access the Data Explorer in Azure Cosmos DB via the Azure portal.

   • Create a new container (e.g., Products) with a defined partition key (CategoryID).

   • Insert sample documents into the container for query testing.

2. Basic Query Execution:

   • Retrieve all documents using SELECT * FROM <container_name>.

   • Understand query statistics such as Request Units (RUs) and document size.

3. Selective Field Retrieval:

   • Query specific fields using SELECT <field1>, <field2> FROM <container_name>.

   • Learn to reduce query results by fetching only required fields like ID, Name, Price, and CategoryName.

4. Using Aliases for Simplification:

   • Use shorter aliases for containers in queries (e.g., SELECT p.ID FROM Products p).

   • Understand the flexibility of using custom alias names.

5. Applying Filters:

   • Use WHERE clauses to filter data based on conditions.

   • Examples of conditions:

     • Retrieve records where Price > 300 AND Price < 400.

     • Handle queries with no matching results (e.g., no records satisfying Price > 300 AND Price < 330).

6. Understanding Query Statistics:

   • Observe the Request Unit (RU) consumption for each query.

   • Learn how even queries with no results consume RUs for computation.

In this lecture, you'll learn how to manipulate and project query results in Azure Cosmos DB using SQL. This session demonstrates how to use aliases to rename fields, create nested objects, and customize the format of query outputs to meet specific data presentation requirements.

What You’ll Learn:

• Use aliases (AS) to rename fields in query results.

• Create nested JSON structures in query outputs.

• Customize query projections for improved data presentation.

• Understand how output manipulation can optimize data usage.

Key Points Covered:

1. Understanding Default JSON Output:

   • Learn how query results are returned as JSON arrays by default.

   • Recognize how individual items are represented in the output when dealing with multiple or single records.

2. Renaming Fields with Aliases:

   • Use the AS keyword to assign a new name to fields in the query result.

   • Example: Rename CategoryName to Category in the output.

   • Observe how renaming affects the field labels while retaining the original data.

3. Creating Nested Objects:

   • Structure query outputs to include nested JSON objects.

   • Example:

     • Original: "Price": 100

     • Transformed: "ScannerData": { "Price": 100 }

   • Modify field names inside nested objects using custom aliases.

4. Customizing Field Names and Values:

   • Change the labels for specific fields dynamically.

   • Example: Rename "Price" to "Price123" while maintaining the field's data structure.

5. Practical Demonstration:

   • Execute a sample query with nested JSON formatting and observe changes in the output.

   • Validate how different projections can be tailored to specific application requirements.

In this lecture, you'll explore how to analyze and manipulate specific fields in Azure Cosmos DB documents using built-in query functions. This session demonstrates how to work with distinct values, retrieve unique data, and efficiently query specific fields to streamline data analysis.

What You’ll Learn:

• How to query specific fields in Cosmos DB documents.

• Using the DISTINCT function to retrieve unique values.

• Leveraging the VALUE keyword to simplify query results.

• Combining DISTINCT and VALUE for concise and meaningful output.

Key Points Covered:

1. Setting Up Sample Data:

   • Delete unnecessary documents from the container for a clean workspace.

   • Insert five sample documents with fields like ID, CategoryName, CategoryID, and Price.

   • Verify data insertion using the query SELECT * FROM Products.

2. Querying Specific Fields:

   • Retrieve a single field (e.g., CategoryName) instead of all document properties.

   • Understand the benefits of reducing query result size for better performance and readability.

3. Using the DISTINCT Function:

   • Retrieve unique values for a specific field (e.g., CategoryName).

   • Example:

     • Original: Apple, Banana, Cherry, Apple, Orange

     • Result: Apple, Banana, Cherry, Orange

4. Leveraging the VALUE Keyword:

   • Simplify query results by returning only the values of a specific field.

   • Example:

     • Default: { "CategoryName": "Apple" }

     • With VALUE: "Apple"

5. Combining DISTINCT and VALUE:

   • Use DISTINCT with VALUE to retrieve only unique values in a simplified format.

   • Example:

     • Original: Apple, Banana, Cherry, Apple, Orange

     • Result: "Apple", "Banana", "Cherry", "Orange"

6. Analyzing Query Performance:

   • Monitor Request Unit (RU) consumption for each query.

   • Learn how smaller, targeted queries can reduce RU costs.

In this lecture, you'll learn how to utilize type-checking functions in Azure Cosmos DB queries. These functions help identify whether certain fields in your documents exist, whether their values are of specific types, and how to manipulate data dynamically during query execution.

Key Points Covered:

• Introduction to Type Checking Functions

  • Purpose: To check if fields exist or match specific data types (e.g., string, number, array).

• Using the IS_DEFINED Function

  • Learn how to verify whether a particular field or key exists in a document.

  • Example: Check for the existence of fields like tags, price, or custom keys.

• Using the IS_NULL Function

  • Identify if a field has a null value.

  • Example: Determine if a field like tags is null or holds a value.

• Handling Arrays with IS_ARRAY

  • Check whether a field contains an array.

• Validating Numeric Fields with IS_NUMBER

  • Ensure values are numeric before performing operations like multiplication or addition.

• Validating Strings with IS_STRING

  • Verify if a field is a string type.

• Data Manipulation in Queries

  • Perform calculations directly in the query without modifying the source document.

  • Example: Add a tax to a price field and display updated values with an alias for clarity.

In this lecture, you'll learn how to leverage built-in functions in Azure Cosmos DB to manipulate and transform data directly within your queries. Specifically, we'll cover the use of the CONCAT function for concatenating field values and the LOWER function for transforming text to lowercase.

Key Points Covered:

1. Introduction to Built-in Functions

   • Overview of built-in functions that can enhance query flexibility and efficiency.

2. Using the CONCAT Function

   • Purpose: Joins two or more field values or strings into a single output.

   • Example: Concatenate the name field with the tags field, separated by a custom delimiter (e.g., a pipe symbol |).

   • Alias Usage: Assign an alias to the concatenated output for clarity (e.g., ConcatOutput).

3. Using the LOWER Function

   • Purpose: Converts text values in fields to lowercase.

   • Example: Transform the SKU or CategoryID fields from uppercase to lowercase while displaying both original and transformed values side by side.

4. Combining Functions in Queries

   • Use functions like CONCAT and LOWER together in a single query to manipulate data dynamically.

In this lecture, you'll learn how to execute Azure Cosmos DB queries programmatically using the Cosmos DB SDK in Visual Studio 2022. This hands-on session demonstrates how to transition from running queries in the Azure portal to running them through source code, focusing on retrieving and iterating over data from a Cosmos DB container.

Key Points Covered:

1. Introduction to Programmatic Query Execution

   • Understand the shift from portal-based query execution to using the SDK.

   • Overview of prerequisites: Visual Studio 2022 and a console application for Cosmos DB.

2. Setting Up the Environment

   • Reviewing existing project setup with Cosmos DB connection configurations.

   • Ensuring compatibility of document structure (e.g., fields like ID, CategoryID, Name, and Price).

3. Writing and Executing a Query

   • Defining a SQL query (SELECT * FROM product P) as a string in the source code.

   • Creating a QueryDefinition object to encapsulate the query.

   • Using the FeedIterator to fetch query results.

4. Iterating and Displaying Results

   • Iterating through results using the FeedIterator.

   • Displaying specific fields (e.g., ID, CategoryID, and Name) in a structured format.

5. Enhancing Output Presentation

   • Adding line breaks for clarity and better readability of query results.

   • Ensuring efficient execution to minimize resource consumption.

In this hands-on lab, we explore the default indexing policy in Azure Cosmos DB, emphasizing its practical implications and functionality. Participants will gain a deep understanding of how indexing works and how to customize it for optimal performance.

What You'll Learn in This Lecture:

• Understanding Default Indexing Policy:

  • How Azure Cosmos DB automatically applies a default indexing policy.

  • Observation of indexed and excluded fields in containers.

• Practical Demonstration:

  • Creating a new Azure Cosmos DB account and container.

  • Preparing code for bulk data insertion using Visual Studio.

  • Verifying the impact of default indexing on container items.

• Hands-On Exercise:

  • Bulk inserting thousands of records into the container.

  • Examining the indexing behavior using the IncludePath and ExcludePath settings.

  • Measuring resource consumption (RU/s) with and without indexing for specific queries.

Key Pointers for the Lab:

1. Default Indexing Policy:

   • By default, all fields in container items are indexed.

   • System-generated fields like _etag are excluded from indexing.

2. Lab Setup:

   • Create a Cosmos DB account with NoSQL API.

   • Use default settings for throughput and redundancy during account creation.

3. Code Preparation:

   • Utilize .NET code in Visual Studio for bulk data insertion.

   • Update connection strings and keys as required.

4. Bulk Data Insertion:

   • Insert 1,000+ records into the container for testing.

   • Generate random data for fields like ID, Name, CategoryID, and Price.

5. Indexing Policy Analysis:

   • Observe default indexing configurations in the container settings.

   • Execute queries to measure RU/s with the default indexing policy.

6. Future Exploration:

   • Modify indexing policies for specific fields in subsequent exercises.

   • Compare performance impacts between default and custom indexing policies.

This lecture focuses on exploring and modifying the indexing policies in Azure Cosmos DB. You will learn how to optimize query performance by selectively including or excluding fields for indexing, demonstrating the impact of indexing on query resource consumption (RU/s).

What You'll Learn in This Lecture:

• Default Indexing Policy Overview:

  • Azure Cosmos DB indexes all fields by default except system-generated fields like _etag.

  • Benefits of default indexing in accelerating query execution and minimizing resource utilization.

• Hands-On Demonstration:

  • Querying indexed fields and analyzing query statistics.

  • Observing the impact of indexing policies on Request Units (RUs).

  • Using SQL-like queries to filter data based on indexed and non-indexed fields.

• Customizing Indexing Policy:

  • Excluding specific fields (e.g., name) from indexing.

  • Retaining selective fields (e.g., price) for indexing.

  • Saving changes to indexing policies and observing their impact on query performance.

This lecture demonstrates how to configure and manage the indexing policy for Azure Cosmos DB containers using the Azure SDK. Unlike the previous lab, where the changes were made through the Azure portal, this session focuses on programmatic configuration using code, offering a more automated and scalable approach.

What You'll Learn in This Lecture:

• Default Indexing Policy Overview:

  • Understanding the default indexing policy when creating a container via SDK.

  • The default policy includes all fields (/*) and excludes the system-generated _etag field.

• Hands-On Exercise:

  • Programmatically creating a database and container using the Azure SDK.

  • Observing default indexing policies in a container created via SDK.

  • Modifying the default indexing policy by including specific fields and excluding others.

• Implementation Steps:

  • Using Azure Cosmos DB SDK in Visual Studio to create and manage resources.

  • Assigning a custom indexing policy to a container during creation.

  • Verifying changes to the indexing policy in the Azure portal.

In this lecture, we conclude the indexing section by emphasizing the importance of managing resources efficiently in Azure Cosmos DB. This session focuses on understanding the implications of provisioning throughput and its impact on performance and billing.

What You'll Learn in This Lecture:

• Provisioned Throughput Management:

  • How provisioned throughput settings affect billing.

  • The importance of limiting total provisioned throughput at the account level to manage costs.

• Resource Optimization:

  • Best practices for managing resources in Azure Cosmos DB to avoid unnecessary charges.

  • Steps to delete unused databases and containers to optimize account performance.

• Practical Insights:

  • How provisioning unlimited throughput might impact your billing.

  • Tips for better resource and cost management in the Azure cloud environment.

This lecture explores how to monitor and handle changes in Azure Cosmos DB using the Azure SDK. The session focuses on leveraging the Change Feed feature to detect and respond to changes such as inserts, updates, and deletions in your Cosmos DB containers, providing a practical, hands-on experience for real-world applications.

What You'll Learn in This Lecture:

• Change Feed Basics:

  • Understand how the Change Feed in Azure Cosmos DB tracks changes in a container.

  • Learn the benefits of using Change Feed for monitoring and responding to database transactions.

• Lab Implementation:

  • Creating a database and two containers: one for storing data and another for logging changes.

  • Writing SDK-based code to detect changes in a container and take appropriate actions.

• Practical Use Cases:

  • Examples of actions triggered by Change Feed, such as stopping a virtual machine, creating a storage account, or logging transaction details.

This lecture demonstrates how to integrate Azure Cosmos DB with Azure Functions to automate the processing of database changes. The session builds upon the previous lab by replacing SDK-based monitoring with a serverless, event-driven approach using Azure Functions and Cosmos DB triggers. This hands-on experience highlights real-world applications of monitoring and responding to changes in a Cosmos DB container.

What You'll Learn in This Lecture:

• Overview of Azure Cosmos DB Integration:

  • Introduction to using Azure Functions for event-driven monitoring.

  • How to process changes in Azure Cosmos DB containers using a Cosmos DB trigger.

• Lab Implementation:

  • Creating and configuring an Azure Function App.

  • Setting up a Cosmos DB trigger to detect changes in container data.

  • Testing change detection by inserting and modifying records.

• Practical Applications:

  • Automating tasks such as stopping virtual machines, creating storage accounts, or logging changes when a database transaction occurs.

  • Demonstrating the serverless architecture for real-time data processing.

Key Pointers for the Lab:

1. Resource Setup:

   • Create a database (CosmicWorks) and a container (Products) in Azure Cosmos DB.

   • Remove unnecessary resources to keep the setup minimal and cost-effective.

2. Azure Function App Configuration:

   • Deploy a Function App with a Cosmos DB Trigger.

   • Configure the trigger to monitor changes in the Products container and log them in a lease container (e.g., ProductList).

3. Azure Function Deployment:

   • Use .NET 6 runtime for the Azure Function.

   • Automatically create a lease container if it does not already exist.

4. Testing the Integration:

   • Insert and update records in the Products container through the Azure portal.

   • Observe changes detected and logged by the Azure Function in real-time.

5. Best Practices:

   • Clean up resources post-execution to avoid unnecessary costs.

   • Regularly monitor the resource group for unused services and delete them if no longer needed.

6. Advantages of Azure Functions:

   • Simplifies integration and reduces the need for custom SDK code.

   • Supports serverless execution, improving scalability and cost efficiency.

This lecture explores how to enable advanced data search capabilities in Azure Cosmos DB by integrating it with Azure Cognitive Search. The session focuses on using free-form text search for intelligent data retrieval, moving beyond traditional SQL-based queries, and leveraging AI-powered search capabilities.

What You'll Learn in This Lecture:

• Azure Cosmos DB Data Search Basics:

  • Understanding how to search and filter data in Cosmos DB using SQL queries.

  • Exploring the limitations of SQL-based searches for free-form text queries.

• Integration with Azure Cognitive Search:

  • Provisioning Azure Cognitive Search to index data stored in Azure Cosmos DB.

  • Preparing Cosmos DB for integration by inserting and managing sample data.

• Hands-On Implementation:

  • Setting up a new Cosmos DB account, database, and container.

  • Populating the container with bulk data using the CosmicWorks tool.

  • Configuring throughput settings to support bulk data operations.

Key Pointers for the Lab:

1. Resource Setup:

   • Create a new Azure Cosmos DB account with provisioned throughput.

   • Set up a database (CosmicWorks) with a container (Products) and a partition key (CategoryID).

2. Bulk Data Insertion:

   • Install the CosmicWorks .NET tool to generate and insert sample data.

   • Use the primary key and endpoint from Cosmos DB to configure the tool.

   • Populate the Products container with a large dataset for testing.

3. Throughput Configuration:

   • Adjust throughput settings for efficient bulk data insertion:

     • Enable "No Limit" at the account level to handle high RU consumption.

     • Set throughput to 4,000 RUs at the database level for optimal performance.

4. Advanced Search Capabilities:

   • Introduce Azure Cognitive Search for free-form text search, allowing intelligent querying without relying on SQL syntax.

   • Discuss indexing Cosmos DB data into Cognitive Search for enhanced searchability.

5. Preview of Next Steps:

   • Provision Azure Cognitive Search.

   • Configure indexing for Cosmos DB data in Cognitive Search.

   • Perform free-form text searches to demonstrate AI-powered data retrieval.

This lecture demonstrates the integration of Azure Cognitive Search with Azure Cosmos DB to enable intelligent, free-form text search capabilities. The session focuses on indexing Cosmos DB data using Azure Cognitive Search and configuring search schemas for optimized data retrieval.

What You'll Learn in This Lecture:

• Azure Cognitive Search Overview:

  • Introduction to Cognitive Search as a tool for enabling AI-powered, full-text search.

  • Differences between traditional SQL-based queries and free-form text search.

• Lab Implementation:

  • Creating and configuring an Azure Cognitive Search resource.

  • Connecting Azure Cognitive Search to an Azure Cosmos DB container for data indexing.

  • Setting up a custom index schema to define searchable, sortable, and filterable fields.

• Hands-On Experience:

  • Indexing data stored in Azure Cosmos DB into Cognitive Search.

  • Running basic queries on the indexed data to demonstrate search capabilities.

Key Pointers for the Lab:

1. Resource Setup:

   • Create an Azure Cognitive Search resource using the free pricing tier for testing.

   • Configure region and resource group settings for the Cognitive Search resource.

2. Connecting to Cosmos DB:

   • Use the Products container in the CosmicWorks database as the data source.

   • Provide the Cosmos DB connection string and configure the indexing query.

3. Defining the Index Schema:

   • Customize the schema by selecting fields to be indexed, such as CategoryID, Name, and Price.

   • Define attributes for each field:

     • Searchable: Enable free-form text search (e.g., on Name).

     • Filterable: Allow filtering (e.g., on CategoryID).

     • Sortable: Enable sorting (e.g., by Price).

4. Indexer Configuration:

   • Set the indexer to run once for this lab.

   • Optionally configure periodic or continuous indexing for dynamic datasets.

5. Testing the Search Index:

   • Query the Cognitive Search index to verify the indexed data.

   • Monitor the indexing process and validate results in the Azure portal.

6. Indexing Time:

   • Allow sufficient time for the indexer to process and index data.

   • Test the functionality by running free-form text queries after the indexing is complete.

In this lecture, you'll learn how to perform search queries on indexed data from Azure Cosmos DB using Azure Cognitive Search. This hands-on session demonstrates step-by-step techniques to query and analyze data effectively, leveraging the indexing capabilities of Azure Cognitive Search.

Key Takeaways:

• Understand how to navigate and use the Search Explorer in Azure Cognitive Search.

• Execute query strings on indexed Cosmos DB data.

• Retrieve specific datasets based on filter criteria like string matches or counts.

• Use Red and Blue data filters to query and analyze indexed records.

• Explore the top results functionality for targeted data insights.

• Gain insights into free-form text search for flexible querying.

Pointers:

1. Indexer Validation:

   • Indexer processes ~295 documents from Cosmos DB.

   • Monitor and confirm index creation progress.

2. Search Explorer:

   • Utilize the Search Explorer to execute queries on the indexed data.

   • Select specific indexes for targeted querying.

3. Query Examples:

   • Query for records containing a specific string (e.g., "Red" or "Blue").

   • Retrieve counts for matching records (e.g., "Red" name field results in 38 records).

   • Filter and limit the top results (e.g., top 6 results with "Blue" in the name).

4. Free-Form Text Search:

   • Enable flexible data retrieval using free-form queries for comprehensive insights.

In this lecture, you'll learn the best practices for cleaning up Azure resources after completing hands-on activities. This step is crucial to prevent unnecessary charges and manage resource limits effectively in Azure Cosmos DB and Azure Cognitive Search.

Key Takeaways:

• Understand the importance of resource cleanup to avoid additional costs.

• Learn how to delete resource groups and individual resources in Azure.

• Configure and manage provisioned throughput in Azure Cosmos DB to optimize usage.

Pointers:

1. Resource Group Deletion:

   • Navigate to the resource group (e.g., cog-rg) created for Azure Cognitive Search.

   • Delete the resource group to ensure no additional costs are incurred.

2. Throughput Configuration in Cosmos DB:

   • Reduce provisioned throughput for cost management:

     • Attempted to limit total provisioned throughput to 1000 RU/s.

     • Resolved issues where 4000 RU/s was already set at the database and container levels.

3. Database Cleanup:

   • Navigate to the Data Explorer in Azure Cosmos DB.

   • Delete databases (e.g., cosmicwork) to reset the environment.

   • Plan to recreate databases if needed for future activities.

4. Free Trial Adjustment:

   • Convert account settings to use the free trial tier (e.g., 1000 RU/s) after cleanup.

In this lecture, we delve into configuring high availability and replication in Azure Cosmos DB. Learn how to distribute your data across multiple availability zones and regions to ensure reliability, scalability, and fault tolerance.

Key Takeaways:

• High Availability: Understand the importance of replicating data across regions and availability zones for uninterrupted application performance.

• Replication Options: Learn about local and global distribution:

  • Local Distribution: Replicate data within the same region across multiple availability zones.

  • Global Distribution: Replicate data across different geographic regions for global accessibility.

• Configuration Methods:

  • Enable global distribution and geo-redundancy during account creation.

  • Add multiple regions to an existing account using the "Replicate Data Globally" option.

• Manual vs. Automatic Failover:

  • Configure manual failover for controlled operations.

  • Enable automatic failover for seamless region switching.

• Multi-Region Write Capabilities: Configure multi-region write options for concurrent updates across regions.

• Cost Evaluation:

  • Calculate the cost of replication as a multiple of the single-region cost relative to the number of replicas.

  • Example: A region with 1,000 RU/s replicated to five regions will result in 6,000 RU/s.

Pointers:

1. Local Distribution:

   • Data is replicated within multiple availability zones in the same region (e.g., East US).

   • Ensures high availability in case of local failures.

2. Global Distribution:

   • Data is replicated across multiple regions (e.g., East US, West US, Japan East).

   • Provides redundancy and low-latency access for global users.

3. Configuration Steps:

   • During account creation:

     • Navigate to the Global Distribution tab.

     • Enable Geo-Redundancy and configure multi-region writes.

   • Post-account creation:

     • Use the Replicate Data Globally option to add or manage regions.

4. Failover Mechanisms:

   • Manual failover: Allows planned region changes.

   • Automatic failover: Ensures continuity during unplanned outages.

5. Cost Calculation:

   • Total RU/s cost = (Single-region RU/s) x (Number of regions).

   • Example: A base of 1,000 RU/s in one region replicated to five regions results in 6,000 RU/s.

In this lecture, you will explore the concepts of geo-redundancy and failover mechanisms in Azure Cosmos DB. Learn how to enable and configure global distribution, manage replication across regions, and handle failovers for high availability and resilience in your database.

Key Takeaways:

• Understand geo-redundancy and its role in ensuring data availability.

• Learn about paired regions and how to configure them during and after account creation.

• Differentiate between manual failover and service-managed failover.

• Configure multiple regions for data replication and observe region-specific read/write capabilities.

• Evaluate cost implications of adding regions and managing throughput limits.

Pointers:

1. Geo-Redundancy Configuration:

   • Enable geo-redundancy during account creation using the Global Distribution tab.

   • Automatically replicate data to paired regions (e.g., West US and East US, South India and Central India).

   • Add custom regions beyond paired regions (e.g., Australia, Europe).

2. Region Types:

   • Primary Write Region: The region where all writes occur.

   • Read-Only Regions: Additional regions for read access to replicated data.

3. Manual and Service-Managed Failover:

   • Manual Failover:

     • Controlled switch of the primary region during planned maintenance.

   • Service-Managed Failover:

     • Automatic transition to another region in case of a region outage.

4. Adding Regions Post-Creation:

   • Use the Replicate Data Globally option to add new regions.

   • Update throughput settings to accommodate additional regions and avoid limits.

5. Cost Implications:

   • Additional regions incur costs based on replication and storage.

   • Calculate costs as a multiple of the single-region cost relative to the number of replicas.

6. Global Distribution Options:

   • Enable multi-region writes for concurrent updates across regions.

   • Ensure data redundancy and fault tolerance through paired and custom regions.

7. Practical Demo:

   • Adding regions (e.g., South Central US, West US 3).

   • Configuring manual failover and preparing for service-managed failover.

   • Observing data accessibility and availability across multiple regions.

In this lecture, you will gain a comprehensive understanding of failover mechanisms in Azure Cosmos DB, including service-managed failover for automated resilience and manual failover for simulating region failure scenarios. You will also learn how to prioritize regions effectively to ensure high availability in distributed environments.

Key Takeaways:

• Learn the difference between service-managed failover and manual failover.

• Configure and prioritize regions for failover scenarios.

• Understand the role of primary write regions and read-only regions during failover.

• Observe real-time updates to region priorities and simulate failover scenarios.

• Prepare for multi-region write configurations and further hands-on demonstrations.

Pointers:

1. Service-Managed Failover:

   • Automated failover mechanism where Azure Cosmos DB switches to a secondary region if the primary write region goes down.

   • Configure failover priorities for regions:

     • Assign higher priority to regions likely to handle critical operations.

     • Drag-and-drop regions in the Azure portal to update failover priorities.

   • Example:

     • Primary region: West US.

     • Secondary region (priority 1): West US 3.

     • Tertiary region: South Central US.

   • Changes take a few minutes to propagate, and updates are reflected in the failover configuration.

2. Manual Failover:

   • Simulate a primary region failure to test how failover works.

   • Select a new read region to act as the primary write region.

   • Example:

     • Original write region: West US.

     • Simulated failover write region: West US 3.

     • Observe the updated configuration in Replicate Data Globally to verify the changes.

3. Key Configuration Steps:

   • Enable Service-Managed Failover in the Azure portal.

   • Update region priorities for seamless automatic failover.

   • Test manual failover by simulating a region outage and observing how other regions assume primary write capabilities.

4. Practical Observations:

   • Verify updated region priorities in Overview and Replicate Data Globally.

   • Snapshot configurations before and after failover for documentation.

5. Benefits of Failover Mechanisms:

   • Ensure data availability during planned maintenance or unexpected outages.

   • Minimize downtime and maintain seamless application performance.

In this hands-on lecture, you will learn how to connect to Azure Cosmos DB across multiple regions using the NoSQL SDK. We will explore how to read data from a preferred region, configure region priorities, and optimize data access using Visual Studio 2022. Additionally, we'll ensure cost management by cleaning up resources after the lab.

Key Takeaways:

• Configure multi-region read and write capabilities in Azure Cosmos DB.

• Learn how data is replicated from the primary write region to other regions.

• Use the NoSQL SDK to connect to and interact with Cosmos DB from a preferred region.

• Analyze JSON responses to verify data retrieval from the specified region.

• Perform resource cleanup to stay within free-tier limits and optimize costs.

Pointers:

1. Setting Up Multi-Region Cosmos DB:

   • Configure multiple regions during Cosmos DB account creation.

   • Assign write region and read regions:

     • Example: Primary write region – West US 3, Read regions – West US, South Central US.

2. Creating and Managing Database:

   • Create a database (e.g., Cosmic Works) and a container (e.g., Product).

   • Set throughput to 400 RU/s for the primary region, which replicates across regions (total: 1200 RU/s for three regions).

3. Connecting with NoSQL SDK:

   • Use Visual Studio 2022 to configure and run the NoSQL SDK.

   • Update keys and endpoints in the code to match the new Cosmos DB account.

   • Specify preferred read regions and the order of priority:

     • Example: 1st – West US, 2nd – South Central US, 3rd – West US 3.

4. Reading Data Across Regions:

   • Insert data into the primary write region and verify replication.

   • Fetch data using the NoSQL SDK and analyze the JSON response to determine the region from which data was read.

   • Verify the response in a JSON editor to observe key fields like HTTP Response Status and region metadata.

5. Cost Management and Cleanup:

   • Optimize costs by removing unused regions and databases:

     • Retain only one write region and one read region.

     • Delete the Cosmic Works database to reduce RU/s usage.

   • Verify cost settings in the Cost Management section and ensure limits are within free-tier eligibility.

In this lecture, you will gain a deep understanding of consistency models in Azure Cosmos DB, a critical concept in distributed databases. Learn how consistency impacts data replication across regions, explore the five consistency levels provided by Cosmos DB, and understand how to configure these levels to meet your application's requirements.

Key Takeaways:

• Understand the five consistency levels in Azure Cosmos DB:

  • Strong Consistency

  • Bounded Staleness

  • Session Consistency (Default)

  • Consistent Prefix

  • Eventual Consistency

• Learn the trade-offs between consistency, latency, and availability.

• Configure consistency levels in the Azure Cosmos DB portal.

• Recognize use cases for each consistency model, from highly sensitive applications to those tolerant of eventual consistency.

Pointers:

1. Consistency Levels Overview:

   • Strong Consistency:

     • Guarantees data is the same across all regions with minimal latency.

     • Ideal for applications like banking and stock markets where absolute data consistency is critical.

   • Bounded Staleness:

     • Ensures data consistency within a defined lag or version limit.

     • Suitable for applications requiring near-real-time data.

   • Session Consistency:

     • Provides consistency within a session, ensuring users read their own writes.

     • Default level, balancing performance and consistency.

   • Consistent Prefix:

     • Guarantees no out-of-order data but allows delays between updates.

     • Useful for less critical real-time applications.

   • Eventual Consistency:

     • Data will eventually be consistent but may arrive out of order or with significant delays.

     • Ideal for social media or other applications where latency is less critical.

2. Portal Demonstration:

   • Navigate to the Default Consistency settings in the Cosmos DB account.

   • Observe and modify the default consistency level (default: Session Consistency).

   • Switch to Strong Consistency for hands-on demonstration:

     • Configuration impacts data replication across primary and secondary regions.

     • Application scenarios like banking require this high level of consistency.

3. Trade-Offs:

   • Stronger consistency ensures accuracy but increases latency and reduces availability.

   • Weaker consistency (e.g., Eventual) improves performance and scalability but sacrifices real-time accuracy.

In this lecture, you will learn how to configure and compare consistency levels in Azure Cosmos DB using both the Azure portal and the NoSQL SDK. This hands-on demonstration highlights the differences in performance and RU consumption between Strong Consistency and Eventual Consistency, giving you insights into how consistency settings impact data retrieval across regions.

Key Takeaways:

• Understand the steps to configure consistency levels in the Azure Portal.

• Use the NoSQL SDK in Visual Studio 2022 to interact with Cosmos DB and modify consistency levels programmatically.

• Compare the RU consumption and performance of Strong Consistency vs. Eventual Consistency.

• Recognize the implications of consistency settings on distributed databases for various application scenarios.

Pointers:

1. Configuring Consistency in the Azure Portal:

   • Navigate to the Default Consistency settings in Cosmos DB.

   • Change the default setting from Session Consistency (default) to Strong Consistency.

   • Strong Consistency ensures data replication with minimal latency across all regions.

2. Database and Container Setup:

   • Create a database (e.g., Cosmic Works) and a container (e.g., Products).

   • Configure the partition key (e.g., Category ID) and set throughput at the container level (e.g., 400 RU/s).

3. Using the NoSQL SDK:

   • Retrieve data from Cosmos DB using Strong Consistency:

     • Set up keys, endpoints, and container details in the Visual Studio project.

     • Fetch data using the SDK and observe RU consumption (e.g., 2 RUs for strong consistency).

   • Modify the code to switch to Eventual Consistency:

     • Add a RequestOptions parameter to specify consistency at the SDK level.

     • Compare RU consumption (e.g., 1 RU for eventual consistency).

4. Key Observations:

   • Strong Consistency:

     • Reads the latest, consistent data from all regions.

     • Consumes higher RUs due to validation across regions.

   • Eventual Consistency:

     • Reads data from the nearest available region without strict consistency checks.

     • Consumes fewer RUs, making it more cost-effective for less critical data.

5. Use Cases:

   • Strong Consistency: Financial transactions, stock markets, or any scenario requiring real-time data accuracy.

   • Eventual Consistency: Social media applications, content delivery, or scenarios where slight delays are acceptable.

6. Cost Management:

   • Delete unnecessary databases and regions to optimize costs.

   • Reset throughput settings to remain within the free-tier limits.

In this lecture, you will learn how to configure and utilize multi-region writes in Azure Cosmos DB to enhance performance, reduce latency, and increase write availability. Through a hands-on demonstration, we’ll explore how to enable multi-region writes, insert data using the NoSQL SDK, and observe the impact of this configuration on cost and performance.

Key Takeaways:

• Understand the difference between read-only regions and multi-region writes.

• Learn how to enable multi-region writes during and after account creation.

• Use the NoSQL SDK to insert data into specific regions in a multi-region write setup.

• Observe cost implications and performance differences between single-region and multi-region writes.

• Clean up resources to optimize costs after the lab.

Pointers:

1. Understanding Multi-Region Writes:

   • Multi-region writes allow data to be written to all configured regions, removing the concept of a single primary write region.

   • This setup enhances availability and enables faster writes by reducing latency for globally distributed applications.

2. Enabling Multi-Region Writes:

   • During account creation:

     • Select the Global Distribution tab and enable multi-region writes.

   • Post-creation:

     • Navigate to Replicate Data Globally in the Azure portal.

     • Add regions (e.g., Central US, Canada Central) and enable multi-region writes.

3. SDK Integration for Multi-Region Writes:

   • Configure the NoSQL SDK in Visual Studio to interact with a multi-region Cosmos DB account.

   • Specify the target region for writes using a variable (e.g., Canada Central).

   • Insert data programmatically into the specified region.

4. Observations on Multi-Region Writes:

   • Cost Implications:

     • Multi-region writes incur higher RU consumption as data is replicated across all write-enabled regions.

     • Example: Writing to three regions with 400 RU/s costs approximately three times the RUs of a single region.

   • Performance:

     • Reduced latency for globally distributed users due to region-specific writes.

   • SDK Output:

     • Status codes (e.g., 201 Created) confirm successful data insertion into the targeted region.

5. Resource Cleanup:

   • Delete the Cosmos DB account and associated resources to avoid unnecessary costs.

   • Verify that all regions and containers are removed from the Azure portal.

In this lecture, we explore indexing policies in Azure Cosmos DB and learn how to optimize them for different workloads, such as read-heavy or write-heavy operations. By default, Cosmos DB indexes all fields in a container, but this default behavior can be customized to improve performance and reduce costs. This session includes a step-by-step demonstration of setting up a serverless Cosmos DB account, preparing data for insertion, and understanding the default indexing policy.

Key Takeaways:

• Learn the default indexing behavior of Azure Cosmos DB.

• Understand how indexing affects performance and RU consumption for read and write operations.

• Explore how to optimize indexing policies to suit specific workload requirements.

• Prepare a sample dataset and application for hands-on demonstrations in upcoming lectures.

Pointers:

1. Setting Up the Environment:

   • Create a serverless Cosmos DB account:

     • Subscription: Pay-as-you-go model.

     • Capacity Mode: Serverless.

   • Create a database (Cosmic Works) and container (Products):

     • Partition Key: Category ID.

     • Indexing Policy: Default (indexes all fields).

2. Understanding Default Indexing:

   • The default indexing policy includes all fields (indicated by /).

   • This behavior ensures all fields are searchable and queryable but can increase RU consumption for write-heavy operations.

3. Cloning the Sample Code:

   • Clone the sample index optimization repository from GitHub:

     • Contains classes for product and tag definitions, along with a script to insert data.

   • Prepare the environment using Visual Studio Code:

     • Open the cloned repository and set up the workspace.

   • Key files include:

     • Product.cs: Defines the product schema, including fields like ID, Category ID, Tags, etc.

     • Scripts.cs: Handles data insertion and logs RU consumption.

4. Preparing for Indexing Policy Optimization:

   • Use a sample JSON file with a large dataset for testing.

   • Observe the RU charges for inserting data with the default indexing policy.

   • Plan to modify the indexing policy to exclude certain fields or apply a custom configuration.

In this hands-on lecture, we explore how to optimize indexing policies in Azure Cosmos DB for common database operations such as read-heavy and write-heavy workloads. You'll see how modifying the default indexing policy can significantly reduce RU consumption and improve performance by tailoring the indexing to your application's specific needs.

Key Takeaways:

• Understand the default indexing policy in Cosmos DB and its impact on RU consumption.

• Learn to create custom indexing policies for optimized performance.

• Observe how different indexing configurations affect RU consumption for data insertion.

• Gain insights into when to use no indexing for write-intensive scenarios.

Pointers:

1. Default Indexing Policy:

   • By default, all fields (/) in a container are indexed.

   • Ensures that every field is searchable but increases RU consumption for write operations.

   • Example: Inserting a sample JSON record consumed 48.38 RUs with the default indexing policy.

2. Customizing Indexing Policy:

   • Include only the fields required for queries to reduce indexing overhead.

   • Example: Indexing only Name and Category Name reduced RU consumption to 8.38 RUs.

   • Steps to update indexing policy:

     • Navigate to the container’s Indexing Policy in the Azure portal.

     • Modify the Included Paths to specify required fields.

     • Exclude unnecessary fields using Excluded Paths.

3. No Indexing Policy:

   • Disable indexing entirely for scenarios where data retrieval is not required.

   • Example: Setting indexing mode to None reduced RU consumption further to 7.05 RUs.

   • Useful for write-intensive applications where querying is not performed.

4. Practical Demonstration:

   • Clone and use the provided GitHub repository with a sample JSON file.

   • Insert data using the NoSQL SDK with different indexing policies:

     • Default Policy

     • Optimized Policy (specific fields only)

     • No Indexing Policy

   • Observe the impact of each policy on RU consumption.

5. Reverting to Default Policy:

   • After experimenting with custom policies, revert to the default policy for further labs:

     • Update the Indexing Policy to include all fields (/).

In this hands-on lecture, you will explore how to optimize complex queries in Azure Cosmos DB using composite indexes. You’ll learn to identify scenarios where composite indexes are required and configure them to improve query performance, particularly for operations involving sorting or filtering on multiple fields.

Key Takeaways:

• Understand the limitations of default indexing policies for complex queries.

• Learn how to create and configure composite indexes in Cosmos DB.

• Optimize queries with sorting and multi-field filtering using composite indexes.

• Observe how RU consumption changes with and without composite indexes.

Pointers:

1. Setup:

   • Use the existing serverless Cosmos DB account and database (Cosmic Works).

   • Insert a large dataset using the Cosmic Works tool:

     • Configure the endpoint and keys in the tool.

     • Generate sample data for testing.

2. Default Indexing Policy Observations:

   • Execute basic queries (SELECT * FROM c) and measure RU consumption:

     • Example: A basic query consumed 8.41 RUs.

   • Observe similar RU consumption for specific field queries (e.g., retrieving Name, Category Name, Price).

3. Introducing Composite Indexes:

   • Scenarios requiring composite indexes:

     • Queries involving sorting or multi-field operations (e.g., sorting by Category Name in descending order and Price in ascending order).

     • Without composite indexes, such queries result in errors.

   • Update the Indexing Policy to include a composite index:

     • Example: Add a composite index for Category Name (descending) and Price (ascending).

4. Query Optimization with Composite Indexes:

   • Execute queries with the composite index:

     • Example: Sorting by Category Name (descending) and Price (ascending).

     • Observe successful execution and RU consumption (e.g., 12.25 RUs).

   • Add additional fields (e.g., Name) to the composite index and re-run the queries:

     • Include all fields in the composite index to handle multi-field operations.

     • Validate the optimized query performance and results.

5. Key Observations:

   • Queries involving sorting and filtering across multiple fields require composite indexes.

   • Composite indexes improve query execution by reducing errors and optimizing RU consumption.

6. Best Practices:

   • Analyze query patterns to identify fields requiring composite indexing.

   • Avoid over-indexing to minimize write costs and maintain performance balance.

7. Cleanup:

   • Delete the Cosmos DB account after completing the lab to avoid unnecessary costs in the serverless model.