ServerlessMicroservice with AWS - A Complete Guide!: 3-in-1

This video gives an overview of the entire course. 

There are many architectures that exist before Microservices. In  this video, we will see what are they, what are the benefits and  drawbacks,  how do they compare to microservices.           

• See the overview of monolithic multi-tier architecture and Service Orientated Architecture (SOA)

• See the overview of micsoservices architecture

• See the benefits and drawbacks of monolithic and micsoservices architecture

In this video, we will show where does serverless computing fit in with other public cloud offerings.

• See the overview of virtual machines benefits and drawbacks

• See the overview of container benefits and drawbacks

• See the overview of serverless computing  benefits and drawbacks

In this video, we will see what the serverless computing offerings available in AWS are and how these service s fit together.

• See the overview of the serverless managed services

• See the example: how these can fit together to solve a data transformation task

In this video, we will set up a new user. Also, we will see how  can I ensure that the user access is secure and how  I setup my local  serverless environment.           

• Use the AWS Management Console to setup a new user

• Set up multi-factor authentication for a user

• Set up your local environment credentials, AWS Python SDK and CLI

In this video, we will see that with data breaches and compliance requirements security is a key consideration.           

• See the security Examples and impacts on an organization

• Explain the security at rest, in transit, authentication, and authorization

• Learn that AWS has a shared security responsibility model that should be followed

In this video, we will see how can we configure AWS security to help secure user authorization and access to resources.           

• Explain AWS Identity and Access Management (IAM)

• Learn IAM policies, users, groups to help secure your users

• Explain IAM roles to  help secure your trusted entity

In this video, we will see how can we ensure that your microservice is secure and what are some of the best practices.           

• Explain Lambda security

• Learn API gateway and DynamoDB security

• Explain how to monitor and alert

In this video, we will see what is the architecture of serverless data provider API.

• Make use of API gateway, Lambda, and DynamoDB

• Overview of JSON and time series data

• Explain about request/response data flows

In this video, we will see how can I create my first Lambda function that parses the given URL parameters.

• Setup the IAM policies and role

• Write the Lambda code that parses the given URL parameters

• Test the Lambda function works

In this video, we will configure the API gateway to call a Lambda function.

• Create the API gateway resources and method

• Integrate the GET method with Lambda

• Test the API invoked the Lambda function correctly

In this video, we will see how can I create, add data to, and query my DynamoDB table.

• Create a DynamoDB table using boto3

• Insert data into DynamoDB

• Query DynamoDB by partition and sort keys

In this video, we will see how can I query DynamoDB from within a Lambda function.

• Add the previous Lambda code that parses the given URL parameters

• Add the previous code that queries DynamoDB by Partition and sort keys

• Test the Lambda function with  sample request data

In this video, we will see how  the API Gateway, Lambda and DynamoDB integrate together and is it all working.

• Understanding the integration

• Test the data with a request URL

• Understand the overall serverless microservices API

In this video, we will see how we can make sure that the code is  still running correctly and be more productive writing Lambda code. 

• See that testing allows better collaboration, improves product  quality, and leads to shorter release cycles

• Explain how unit tests help  to verify that our functions work as expected

• Mock to replace parts of the system under test

In this video, we will see how can I simulate and debug a Lambda locally with real data without deploying it to AWS.

• Use sample event data

• Pass it into the Lambda function

• Debug the Lambda code step by step

Once the Lambda function is deployed to AWS, this video explains how I can test it to make sure it’s running correctly.

• Use sample event data

• Submit the data via the AWS CLI

• Make sure we get the expected response

In this video, we will see how I know if my API is performing as  expected with a low latency and how can I improve the latency.   

• Use Python Locust tool

• Load test for concurrent  users

• See that the latency can be improved by changing DynamoDB and Lambda settings

In this video, we will get to know that there are many challenges  of provisioning infrastructure manually which include the cost, effort,  lack of repeatable processes and limited scalability.

• Use Infrastructure as a Code

• Learn that serverless infrastructure and resources can be deployed using Serverless Application Model (SAM)

• Learn that serverless infrastructure and resources can be deployed using the AWS CLI other frameworks

In this video, we will get to know that resource and infrastructure need to be provisioned.

• Create an S3 bucket for the Lambda deployment package

• Deploy IAM policies for the Lambda function

• Deploy IAM roles for the Lambda function

In this video, we will see what are the steps needed to prepare and deploy the serverless stack.

• Create a Lambda deployment package

• Create API gateway, lambda, and dynamo using SAM

• Load data into DynamoDB

In this video, we’ll summarize what we have covered in this course.

In this video, we’ll have a quick look at what is going to be covered in the next course. 

This video gives an overview of the entire course. 

In building serverless microservices how can we benefit from what  is available from leading companies and open source communities.                         

• Make use of design patterns

• Use software design patterns and principles

• Use and understand serverless  microservice pattern categories

How do we ensure efficient communication between microservices? How are microservices created and designed?                         

• Understand synchronous versus asynchronous communication

• Understand one-to-one and many-to-many communication microservice patterns

• Understand  decomposition pattern by business capability

What are the serverless microservice patterns that can leverage in AWS to reduce development, maintenance, and running costs?                         

• Understand lambda event sources types

• Understand lambda stream event sources

• Leverage AWS managed services and serverless offerings in each category and use case

Lambda functions are stateless, so you need to store state  externally which is best practice. What is important in distributed  systems and what are the best ways to store state?                         

• Understand that CAP Theorem stands for  Consistency, Availability, and Partition tolerance and is important for  data storage in distributed systems

• Understand the shared  database and database per service patterns, and how they are used in distributed systems

• Understand how these patterns can be implemented with DynamoDB, RDS, Aurora or Aurora serverless

How can we expose data in DynamoDB externally?                         

• Use API Gateway and a lambda function as a proxy

• Create API methods such as GET and POST

• Use the lambda function to shape the request into a DynamoDB query and parse the returned data for the API

Is there a simpler way to expose DynamoDB data externally?                         

• Use API gateway direct DynamoDB integration

• Use API gateway to  shape the query in the integration request step

• Use API gateway to shape the response in the integration response step

How can I implement the transaction log trailing pattern?                         

• Use DynamoDB streams

• Use a lambda function to consume  the stream as an event source

• Use a lambda to write the steam data to another table in DynamoDB

How can I implement the saga pattern as a serverless stack?                         

• Use SNS and lambda combination

• Use step functions to build up a state machine in JSON using the Amazon states language

• Use step functions states to invoke lambda functions

How can I ensure that my DynamoDB database is secure?                         

• Enable encryption at rest

• Enable the VPC endpoint

• Use IAM policies that are locked down as much as possible to a specific table and with minimum permissions

What are the main differences between relational and non-relational databases?                         

• Understand the BASE versus ACID consistency models

• Understand the  scalability for non-relational databases

• Understand SQL flexibility for relational databases

What is a VPC and why is it needed?                         

• Understand VPC is a virtual network isolated from other networks

• Understand that a VPC contains one or more Subnets

• You have to use a VPC to access RDS or Aurora from a Lambda function

What are the main steps for setting up a VPC?                         

• Create VPC and subnets

• Attach an internet gateway and Egress only gateway

• Configure the to route tables and associate it with the subnets

What are the main steps to look out for when accessing RDS from your local network?                         

• Configure you corporate or local firewall

• Configure the route tables with your IP

• Configure the VPC security groups with your IP

Why and how can a lambda access an RDS instance?                         

• Access MySQL RDS from API gateway via a lambda function

• Understand the code and configuration to set it up

• Ensure the lambda role has a policy that allows it to attach and detach network interfaces within a VPC

What are the main differences for a lambda to access Aurora over RDS MySQL?                         

• Use the same code to access MySQL RDS, MySQL Aurora,  and Aurora Serverless

• Understand that Aurora is clustered by default

• Understand the  additional read replica database endpoint in Aurora

What are the main steps to ensure your RDS or Aurora databases communication is secure?                         

• Understand RDS and Aurora encryption in transit code and configuration

• Set up RDS or Aurora IAM database authentication

• Use IAM database authentication to generate an authorization token rather than use a password

We need a way to create an efficient API that can be consumed by third parties and abstract away the underlying complexities.                         

• Understand API Gateway Pattern

• Understand API Composition Pattern

• Understand the benefits and drawback of API Composition Pattern

What can we use to implement the API gateway and API composition patterns?                         

• Understand Amazon Route 53 and use it as a sclable cloud DNS

• Understand implement Amazon API Gateway and AWS Lambda

• Use Amazon CloudFront and Amazon S3

How can we overcome the database persistence limitation such as  lack of full history on row changes, and efficient distributed  synchronisation/replication challenges?                         

• Use the event sourcing pattern where every action is an event appended to an event store

• Use the Command Query Responsibility Segregation (CQRS) pattern to split write and read methods

• Understand that in CQRS, commands are methods that mutates state and queries are methods that return data

What AWS managed services are core to architect the serverless event sourcing pattern?                         

• Use Amazon Kinesis streams

• Use Amazon DynamoDB

• Use Amazon SQS

How can we implement the serverless event sourcing pattern?                         

• Have API Gateway with or without a lambda proxy write to DynamoDB

• Have API Gateway with or without a lambda proxy write to Kinesis Streams

• Have API Gateway with or without a lambda proxy write to SNS or SQS. The lambda the uses CloudWatch trigger

What AWS managed services are core to architect a serverless CQRS pattern?                         

• Use Aurora, Aurora Serverless, or DynamoDB

• Use SQS / SNS

• Use Kinesis Streams and Kinesis Firehose

How can we implement the serverless CQRS pattern command and event processor?                         

• Have API gateway with or without a lambda proxy write to DynamoDB. Then a lambda writes to DynamoDB and Kinesis Firehose

• Have API gateway with or without a lambda proxy write to SQS/SNS. Then a lambda writes to DynamoDB and Kinesis Firehose

• Have API gateway with or without a lambda  proxy write to Kinesis Streams. Then a Lambda writes to DynamoDB and  Kinesis Streams write to Kinesis Firehose

How can we secure the serverless resources?                         

• Use roles specific to the resource

• Attach policies to the role

• Specify the permitted actions on the resources in the policy

Why is it important to monitor your microservices and what patterns are available to do so?                         

• 1000s of microservices, governance, compliance, operational auditing, risk auditing, and so on

• Use application metrics and health checks patterns

• Use centralized logging, audit logging, and distributed tracing patterns

How can serverless application metrics and health check API patterns be implemented?                         

• Use Amazon CloudWatch

• Use a lambda triggered by CloudWatch to run health checks

• Use a lambda to write the results of the  check to CloudWatch metrics and set an Alarm that sends an email  notification upon failure

What are the different ways to send logs to CloudWatch logs?                         

• Use API Gateway built-in logging or custom logging

• Use Lambda print function, native logging or external JSON logging packages

• For non-lambda microservice (e.g. traditional EC2 or container-based) use CloudWatch logs agent

How can I monitor my AWS account activity?                         

• Enable AWS CloudTrail

• Push logs to CloudWatch logs, optionally have a lambda scan them for suspicious activity

• Create Alarms that trigger on suspicious activates

How can I configure Lambda, DynamoDB and other microservices to send traces to X-Ray?                         

• Enable built-in Lambda X-Ray integration with the correct policy

• For Lambda and DynamoDB  in Python use aws-xray-sdk via xray_recorder begin and end, or decorators

• For non-lambda services use aws-xray-daemon

How can I architect a serverless discovery and catalogue service?                         

• For batch discovery use a lambda function that lists all resources, and persists results to DynamoDB

• For near-real-time use AWS CloudTrail that sends logs to CloudWatch logs that a lambda scans, and persists results to DynamoDB

• Expose the DynamoDB catalogue using API gateway via a Lambda proxy

What are the focus and differences of Continuous Integration, and Continuous Delivery, and Continuous Deployment?                         

• Use continuous Integration to focus on automating build and testing to detect issues early

• Use Continuous Delivery to focus on automated release process with a human approval step for the production release

• Use Continuous Deployment to focus on a fully automated release process without a human approval step for the production release

What AWS managed services are used in a serverless CI/CD pipeline and what are they for?                         

• Use IAM roles and polices for security and access

• Use AWS CodeCommit for code version control. Use AWS CodeBuild to  manage build service

• Use AWS CodePipeline for  continuous delivery, and as a service that helps you build a CI/CD  pipeline

How can you access AWS CodeCommit?                         

• With Git credentials use HTTPS or SSH Keys

• Generate them in IAM

• Attach the CodeCommit access policy to the IAM User

How do you setup a CodeBuild project?                         

• Use  source code from CodeCommit or GitHub

• Use an IAM role for CodeBuild to access the AWS resources or code

• Generate a personal access token to access GitHub over HTTPS

How can you create a CodePipeline?                         

• Use An IAM role for CodePipeline to access the AWS resources or code

• Setup CodePipeline stages

• Configure each CodePipeline stage with one or more actions such as source, build, deploy, and approve

How can you setup other CI/CD tools for serverless?                         

• Use their YAML files

• Use a Makefile

• Use CloudFormation with SAM to build and deploy you serverless stack

When is it best to use serverless computing versus containers?                         

• Look at your non-functional requirements

• Use containers for very low latency <  50ms, large number of concurrent requests > 3000, large deployments  packages and fixed costs

• Use serverless when you require latency > 50ms, focus on business logic code, PAYG costs, and you are already on AWS

How can we estimate the serverless costs?                         

• See the number of concurrent and average length of the requests

• See data sizes and read/write capacities

• Monitor costs in billing dashboard

How do you scale database and event streaming resources?                         

• Scale DynamoDB using GSI, creating a batch API and using DAX

• Scale Aurora using read-replicas, DB optimization. Runload testing with production loads and monitor performance

• Scale Kinesis Streams look at the number of consumers required, and load test by replying full production volumes

How do I ensure Lambda functions work at web scale?                         

• Set lambda concurrency limits

• Use lambda pre-warming to reduce their launch time

• Use lambda code optimizations such as reducing package size, logging and declare objects outside  lambda_handler()

This video concludes the entire course.