High-level System Design Bootcamp

Model scale with practical estimations to choose a database, using 500 million daily active users and 2 TB per day of chat data to compare Cassandra with MySQL and MongoDB.

Estimate downtime and throughput for large-scale systems; compare caches and databases, apply sharding and consistent hashing, and master latency, units, and population baselines for scalable design.

Estimate twitter-scale metrics by assuming 20% of the world population, 10% daily active, and 2 queries per day, yielding 3.2k rps, 32 pb storage, and 200 ms latency.

Understand high-level design decisions and trade-offs in system design, choosing between the grind of the countryside and amenities of urban life, and between home and office work environments.

Compare performance and scalability: measure system speed for individual requests versus maintaining constant response times under increasing load, using a vending machine analogy.

Explore the trade-off between latency and throughput in system design, where reducing latency forces compromises in throughput. Most applications aim for high throughput with acceptable latency.

Explore the cap theorem in distributed systems, balancing consistency, availability, and partition tolerance, and decide between cp for atomic banking transactions or ap for eventual social apps.

Explore why scalability matters by comparing a limited drink bottle to a growing crowd, and learn how to provision resources to meet increasing demand in system design.

Explore horizontal vs vertical scaling in system design. Compare adding more servers to handle traffic with upgrading a single server's resources, and discuss when each is appropriate.

Understand how load balancing distributes client requests across multiple servers to prevent overload, avoid unhealthy servers, and reduce single points of failure, using strategies like cycling through available servers.

Leverage caching to speed up applications by storing responses near users, producing cache hits and avoiding database loads during peak traffic. Handle cache misses by refreshing from the main database.

Explore caching across browser, proxy, and gateway layers to reduce network traffic and latency, including browser, data, and application caching, plus CDN seeding with Google Cloud CDN and CloudFront.

Explore four cache strategies, including cache-aside and refresh-ahead, and analyze how TTL, cache misses, and asynchronous updates affect latency, data staleness, and reliability.

Explore cache eviction policies, including least recently used and least frequently used, to manage limited memory by evicting the appropriate element based on access time (timestamps) and frequency.

Explore the core database choices for systems, compare relational and NoSQL options, and evaluate throughput, pros, and cons to select the best fit for a given use case.

Explore relational databases through atomic transactions, consistency, isolation, and persisted data, then compare master-slave and master-master replication for scaling, availability, and latency trade-offs.

Compare relational and NoSQL databases to balance base properties, consistency, and availability; overview key value, document, column store, and graph databases with examples like Redis, MongoDB, CouchDB, and DynamoDB.

Compare relational and nosql databases by evaluating db parameters to choose sql for structured data and nosql for flexible data, using management scenarios to illustrate joins and consistency versus availability.

Explore sharding as a horizontal partitioning strategy for distributing data across multiple database servers to improve performance, availability, and load balancing, including hash-based and consistent hashing approaches.

Outline the core functional requirements of a ridesharing app, including two portals for drivers and customers, real-time location updates to show nearby drivers, and ride requests.

estimate scale by converting population and adoption assumptions into active riders, drivers, and daily requests, then derive requests per second and database updates, including spike scenarios.

Design a driver location database by persisting driver id, latitude, and longitude in a simple MySQL or Postgres cluster, planning for 60 MB and 500k updates every 3 seconds.

Design APIs for customer and driver actions, including update location, get driver location, trip requests and accepts, map lookup, address lookup, and request IDs, with rest, gRPC, and UDP considerations.

Scale the system with geospatial indexing using grid-based partitioning and a grid tree, implement dynamic sharding and consistent hashing, and enable live driver updates, pub-sub broadcasting, and gRPC-based matching.

Design a nearby driver search workflow using a load balancer and aggregation server to return top ten drivers with live updates and fault-tolerant grid index support.

Explore scalable ride-hailing system design with batched writes every 15 seconds, grid-based proximity, top-ten matching with pagination, and kafka-driven monitoring for reliability.

Explore the Google search engine architecture, from web crawler and index to the type-ahead suggestion service, and compare priorities and query formation in search design.

Identify problems and features for a web search system that queries HTML over desktop protocol, displays results, and offers a ten-term typeahead, emphasizing scalability, an extensible crawler, and minimal latency.

Estimate crawl capacity for a billion websites in four weeks, translating pages per site into throughput and per-second queries. Project storage needs into petabytes and outline a scalable database design.

Explore scalable database design for a web crawler, covering hostname-based partitioning, distributed hash tables with consistent hashing, checksum techniques to avoid re-crawling, and trie- or hash-based word partitioning.

Design a scalable web crawler and search stack with DNS resolution, checksum-based deduplication, distributed queues, document indexing, trie-based prefix suggestions, and adult content filtering.

Design the search engine workflow: resolve and cache URLs via DNS, batch documents, deduplicate with checksums, index content, extract crawl URLs, and filter out adult content and malware.

Design a scalable web crawler to fetch and extract documents, distribute urls across servers, and build a distributed structure for real-time top ten search terms with reduced latency.

Explore the engineering challenges of building a scalable video streaming service like YouTube, addressing how to handle billions of viewers and massive video uploads daily.

Define the core functional requirements for a video platform, including upload, watch, share, search, comments, likes, views, and recommendations, while outlining scalability, high availability, eventual consistency, and low buffering.

Explore capacity estimations for a scalable video platform, calculating target users, throughput in requests per second, video uploads, storage needs, and bandwidth to withstand spikes.

Design a scalable database to store user data, video metadata, and video files; evaluate MySQL for user data, Cassandra for metadata, and file storage like S3 for videos.

Design the platform's major APIs, including a video upload API with title and tags, a search API with pagination tokens, and a streaming API using video ID, resolution, and offset.

Scale the platform to billions of users by orchestrating asynchronous video uploads via Kafka, chunking, encoding, storage, and country-based partitioning, with global CDN delivery and search result aggregation.

Build a scalable video platform by connecting authentication, storage, encoding, thumbnails, streaming, and search through microservices, Kafka queues, and caching.

Designs video upload, streaming, and search on distributed servers, with asynchronous uploading and chunked processing to reduce buffering, and MySQL and Bigtable use for metadata, thumbnails, analytics, and video recommendation.

In the high-level system design bootcamp, learn to approach the system design of a shared messaging app like Facebook, Messenger, or WhatsApp, focusing on chat components.

Identify exact chat app requirements, including one-on-one and group chats, active status, persistence, push notifications, image sharing, message blocking, low latency, strict consistency, and high availability.

Explore capacity estimations for a global messenger app, calculating users, daily active users, messages per day, throughput, storage, and bandwidth needs.

Compare relational and NoSQL options for a 3.6 petabyte, 200,000-leads-per-second workload, evaluate buffering with Apache Express, and design hashed partitioning and shard strategy for scalable messaging.

Scale a chat system by shifting from polling to push delivery with sockets to reduce latency; use a session manager with a hash table and consistent hashing across servers.

Designs a chat system with a load balancer, session manager, distributed hash table to deliver messages, persist data, and support private and group chats with privacy considerations and image processing.

Design private one-on-one conversations and group chats via web sockets and long polling, with scalable text and image message databases, push notifications, privacy, edge space, and caching.