Big Data in the AWS (Amazon Web Services) Cloud

Discover how AWS functions as a cloud services platform, enabling big data work with managed and non-managed services, across regions and availability zones to minimize latency.

Explore big data on aws through compute, storage, warehousing, real-time analytics, ai and ml, and data movement with services like Kinesis, Glue, and QuickSight.

Explore object storage for big data on AWS with Amazon S3, buckets, and EMR, highlighting scalable, separate storage and compute, and eventual consistency tradeoffs.

Compare SQL and NoSQL databases, covering relational schemas, typing, normalization and denormalization, and flexible models such as document stores and key-value stores.

Explore relational databases on AWS, including MySQL, PostgreSQL, MariaDB, Oracle, Microsoft SQL Server, and Amazon Aurora, and compare with DynamoDB for scalable big data workloads, including global tables.

See how Duolingo uses DynamoDB to scale for millions of users and billions of exercises with a high throughput table.

Use a big data analytic framework to distribute processing across many computers, enabling parallel work on large data sets; frameworks like Hadoop simplify cluster management with a single programming interface.

Explore AWS big data frameworks: Amazon EMR to spin up scalable Hadoop clusters, Amazon Elasticsearch Service for scalable document search, and Amazon Athena for serverless SQL queries on S3.

Yelp migrates from on-premises Hadoop cluster to Amazon EMR to support big data processing for features like location recommendations, top searches, and ads, scaling to handle over 30 terabytes daily.

Understand why data warehouses enable analytics with online analytical processing, and how AWS redshift offers a scalable, columnar, massively parallel data warehouse for business intelligence and dashboards.

Pinterest uses Amazon Redshift to meet its data warehousing and analytics needs in the cloud, delivering 25 to 100 times faster queries with a managed, scalable solution.

Learn how real-time streaming data flows from many sources into a data stream, enabling real-time dashboards, aggregations, and stock price insights as events occur.

Explore real-time big data services on AWS, including Kinesis data streams, firehose, video streams, and data analytics, to scale throughput and analyze streaming data with managed services.

Netflix uses Amazon Kinesis Data Streams to analyze VPC flow logs in real time, enriching data with application metadata to improve efficiency and reduce latency across microservices.

Explore artificial intelligence and machine learning, how machines learn from data to perform tasks, and apply these concepts to predictive applications, recommendation engines, and positive or negative classifications on AWS.

Explore how aws ai services analyze images and video with Amazon recognition and build outcome predictions with SageMaker. Analyze text with Comprehend, Translate, Transcribe, Polly, and Lex via APIs.

Automate video content indexing for c-span with amazon rekognition, boosting searchability and discoverability across eight networks while reducing indexing time to a third of manual effort.

Explore how business intelligence on AWS uses Amazon QuickSight to integrate disparate data sources, create scalable dashboards with drag-and-drop visuals, and leverage SPICE for fast analytics and automatic data replication.

Explore big data computation on AWS using easy to, provisioning scalable virtual machines with infrastructure as code, auto scaling, and tailored instance types for compute, memory, and graphics intensive workloads.

Shell built a scalable Splunk-based cyber threat monitoring solution on AWS EC2, using compute-optimized instances for data collection, indexing, and search, integrated with S3 and EMR for advanced analytics.

Move data into and around the AWS cloud using Direct Connect, Snowball and Snowmobile, plus Storage Gateway, then orchestrate ETL with AWS Glue and maintain a data catalog.

Discover serverless big data computations on AWS with Lambda triggering image processing and thumbnail creation from S3. Learn memory and time limits and the breadth-over-depth approach using many small functions.