Implementing Serverless Microservices Architecture Patterns

Lambda functions are stateless, so you need to store state externally which is best practice. What is important in distributed systems and what are the best ways to store state?

   •  Understand that CAP Theorem stands for Consistency, Availability, and Partition tolerance and is important for data storage in distributed systems

   •  Understand the shared  database and database per service patterns, and how they are used in distributed systems

   •  Understand how these patterns can be implemented with DynamoDB, RDS, Aurora or Aurora serverless

How can we expose data in DynamoDB externally?

   •  Use API Gateway and a lambda function as a proxy

   •  Create API methods such as GET and POST

   •  Use the lambda function to shape the request into a DynamoDB query and parse the returned data for the API

Is there a simpler way to expose DynamoDB data externally?

   •  Use API gateway direct DynamoDB integration

   •  Use API gateway to  shape the query in the integration request step

   •  Use API gateway to shape the response in the integration response step

How can I implement the transaction log trailing pattern?

   •  Use DynamoDB streams

   •  Use a lambda function to consume  the stream as an event source

   •  Use a lambda to write the steam data to another table in DynamoDB

How can I implement the saga pattern as a serverless stack?

   •  Use SNS and lambda combination

   •  Use step functions to build up a state machine in JSON using the Amazon states language

   •  Use step functions states to invoke lambda functions

How can I ensure that my DynamoDB database is secure?

   •  Enable encryption at rest

   •  Enable the VPC endpoint

   •  Use IAM policies that are locked down as much as possible to a specific table and with minimum permissions

What are the main differences between relational and non-relational databases?

   •  Understand the BASE versus ACID consistency models

   •  Understand the  scalability for non-relational databases

   •  Understand SQL flexibility for relational databases

What is a VPC and why is it needed?

   •  Understand VPC is a virtual network isolated from other networks

   •  Understand that a VPC contains one or more Subnets

   •  You have to use a VPC to access RDS or Aurora from a Lambda function

What are the main steps for setting up a VPC?

   •  Create VPC and subnets

   •  Attach an internet gateway and Egress only gateway

   •  Configure the to route tables and associate it with the subnets

What are the main steps to look out for when accessing RDS from your local network?

   •  Configure you corporate or local firewall

   •  Configure the route tables with your IP

   •  Configure the VPC security groups with your IP

Why and how can a lambda access an RDS instance?

   •  Access MySQL RDS from API gateway via a lambda function

   •  Understand the code and configuration to set it up

   •  Ensure the lambda role has a policy that allows it to attach and detach network interfaces within a VPC

What are the main differences for a lambda to access Aurora over RDS MySQL?

   •  Use the same code to access MySQL RDS, MySQL Aurora,  and Aurora Serverless

   •  Understand that Aurora is clustered by default

   •  Understand the  additional read replica database endpoint in Aurora

What are the main steps to ensure your RDS or Aurora databases communication is secure?

   •  Understand RDS and Aurora encryption in transit code and configuration

   •  Set up RDS or Aurora IAM database authentication

   •  Use IAM database authentication to generate an authorization token rather than use a password

We need a way to create an efficient API that can be consumed by third parties and abstract away the underlying complexities.

   •  Understand API Gateway Pattern

   •  Understand API Composition Pattern

   •  Understand the benefits and drawback of API Composition Pattern

What can we use to implement the API gateway and API composition patterns?

   •  Understand Amazon Route 53 and use it as a sclable cloud DNS

   •  Understand implement Amazon API Gateway and AWS Lambda

   •  Use Amazon CloudFront and Amazon S3

How can we overcome the database persistence limitation such as lack of full history on row changes, and efficient distributed synchronisation/replication challenges?

   •  Use the event sourcing pattern where every action is an event appended to an event store

   •  Use the Command Query Responsibility Segregation (CQRS) pattern to split write and read methods

   •  Understand that in CQRS, commands are methods that mutates state and queries are methods that return data

What AWS managed services are core to architect the serverless event sourcing pattern?

   •  Use Amazon Kinesis streams

   •  Use Amazon DynamoDB

   •  Use Amazon SQS

How can we implement the serverless event sourcing pattern?

   •  Have API Gateway with or without a lambda proxy write to DynamoDB

   •  Have API Gateway with or without a lambda proxy write to Kinesis Streams

   •  Have API Gateway with or without a lambda proxy write to SNS or SQS. The lambda the uses CloudWatch trigger

What AWS managed services are core to architect a serverless CQRS pattern?

   •  Use Aurora, Aurora Serverless, or DynamoDB

   •  Use SQS / SNS

   •  Use Kinesis Streams and Kinesis Firehose

How can we implement the serverless CQRS pattern command and event processor?

   •  Have API gateway with or without a lambda proxy write to DynamoDB. Then a lambda writes to DynamoDB and Kinesis Firehose

   •  Have API gateway with or without a lambda proxy write to SQS/SNS. Then a lambda writes to DynamoDB and Kinesis Firehose

   •  Have API gateway with or without a lambda proxy write to Kinesis Streams. Then a Lambda writes to DynamoDB and Kinesis Streams write to Kinesis Firehose

How can we secure the serverless resources?

   •  Use roles specific to the resource

   •  Attach policies to the role

   •  Specify the permitted actions on the resources in the policy

Why is it important to monitor your microservices and what patterns are available to do so?

   •  1000s of microservices, governance, compliance, operational auditing, risk auditing, and so on

   •  Use application metrics and health checks patterns

   •  Use centralized logging, audit logging, and distributed tracing patterns

How can serverless application metrics and health check API patterns be implemented?

   •  Use Amazon CloudWatch

   •  Use a lambda triggered by CloudWatch to run health checks

   •  Use a lambda to write the results of the check to CloudWatch metrics and set an Alarm that sends an email notification upon failure

What are the different ways to send logs to CloudWatch logs?

   •  Use API Gateway built-in logging or custom logging

   •  Use Lambda print function, native logging or external JSON logging packages

   •  For non-lambda microservice (e.g. traditional EC2 or container-based) use CloudWatch logs agent

How can I monitor my AWS account activity?

   •  Enable AWS CloudTrail

   •  Push logs to CloudWatch logs, optionally have a lambda scan them for suspicious activity

   •  Create Alarms that trigger on suspicious activates

How can I configure Lambda, DynamoDB and other microservices to send traces to X-Ray?

   •  Enable built-in Lambda X-Ray integration with the correct policy

   •  For Lambda and DynamoDB  in Python use aws-xray-sdk via xray_recorder begin and end, or decorators

   •  For non-lambda services use aws-xray-daemon

How can I architect a serverless discovery and catalogue service?

   •  For batch discovery use a lambda function that lists all resources, and persists results to DynamoDB

   •  For near-real-time use AWS CloudTrail that sends logs to CloudWatch logs that a lambda scans, and persists results to DynamoDB

   •  Expose the DynamoDB catalogue using API gateway via a Lambda proxy

What are the focus and differences of Continuous Integration, and Continuous Delivery, and Continuous Deployment?

   •  Use continuous Integration to focus on automating build and testing to detect issues early

   •  Use Continuous Delivery to focus on automated release process with a human approval step for the production release

   •  Use Continuous Deployment to focus on a fully automated release process without a human approval step for the production release

What AWS managed services are used in a serverless CI/CD pipeline and what are they for?

   •  Use IAM roles and polices for security and access

   •  Use AWS CodeCommit for code version control. Use AWS CodeBuild to  manage build service

   •  Use AWS CodePipeline for  continuous delivery, and as a service that helps you build a CI/CD  pipeline

How can you access AWS CodeCommit?

• With Git credentials use HTTPS or SSH Keys
• Generate them in IAM
• Attach the CodeCommit access policy to the IAM User

How do you setup a CodeBuild project?

• Use  source code from CodeCommit or GitHub
• Use an IAM role for CodeBuild to access the AWS resources or code
• Generate a personal access token to access GitHub over HTTPS

How can you create a CodePipeline?

• Use An IAM role for CodePipeline to access the AWS resources or code
• Setup CodePipeline stages
• Configure each CodePipeline stage with one or more actions such as source, build, deploy, and approve

How can you setup other CI/CD tools for serverless?

• Use their YAML files
• Use a Makefile
• Use CloudFormation with SAM to build and deploy you serverless stack

When is it best to use serverless computing versus containers?

• Look at your non-functional requirements
• Use containers for very low latency < 50ms, large number of concurrent requests > 3000, large deployments packages and fixed costs
• Use serverless when you require latency > 50ms, focus on business logic code, PAYG costs, and you are already on AWS

How can we estimate the serverless costs?

• See the number of concurrent and average length of the requests
• See data sizes and read/write capacities
• Monitor costs in billing dashboard

How do you scale database and event streaming resources?

• Scale DynamoDB using GSI, creating a batch API and using DAX
• Scale Aurora using read-replicas, DB optimization. Runload testing with production loads and monitor performance
• Scale Kinesis Streams look at the number of consumers required, and load test by replying full production volumes

How do I ensure Lambda functions work at web scale?

• Set lambda concurrency limits
• Use lambda pre-warming to reduce their launch time
• Use lambda code optimizations such as reducing package size, logging and declare objects outside  lambda_handler()

This video concludes the entire course.