Code Along - AWS Lambda, Kinesis and Serverless Framework

Build a dynamic, server-driven university search platform with flexible filters for degree level, field of study, tuition, and scores, delivering real-time, low-latency, high-availability results under a tight budget.

Explore the structure of data as an array of university objects, each containing courses and fields, and learn how to query by graduate programs while considering fixed schemas and latency.

Examine the traditional architecture with virtual machines running 24/7, API servers behind a load balancer, a compact database schema, plus security and caching layers, and the high maintenance costs.

Explore a serverless architecture built with lambda functions, triggering data processing and pushing results to analytics and a hosted search database, all exposed via api gateways for front-end search.

Install the serverless framework, sign up for AWS, set up identity and access management, and configure credentials with the AWS key and secret to enable Lambda and Kinesis development.

See how the serverless framework helps manage multiple lambda functions, including creating, deploying, changing, and removing them within a project.

Create a new project by making a directory, diving into it, and setting up the codebase for an AWS Lambda and Kinesis workflow using the Serverless Framework.

select a Node.js runtime that matches AWS Lambda's supported versions, focusing on using the same version (such as 6.1.0) to ensure compatibility during development.

Create and deploy your first AWS Lambda function using the serverless framework, configure the provider and region, and test the hello function stored in S3 and managed by CloudFormation.

Understand how a Lambda function is invoked with event and context objects, how the callback returns results after processing, influenced by the event queue, remaining time, and the default timeout.

Create a function that talks to third-party APIs to fetch university data, use the sample data from a provided link, and create a file in the project.

Fetch data with a Lambda function, test via the console, and push the data to Kinesis from the command prompt.

Introduction to Kinesis streams, where producers push data and consumers read it; data stays 24 hours to seven days and is split across shards, not in a single buffer.

Attach university information to each course by incorporating meta information, creating a final data structure that links courses with their universities.

Transform university and course data into objects, assign a partition key, and push records to a kinesis stream with lambda and the serverless framework. Monitor the flow via logs.

Create a kinesis consumer with aws lambda and the serverless framework to read stream records, decode data, and push results to the designated destination via callbacks.

Push course data to Algolia with AWS Lambda and Kinesis for search. Create a unique object id from zip code, course name, and level to update records instead of duplicating.

Explore extracting and categorizing course release levels from the data object to produce a unique list of levels such as undergraduate, graduate, and associate.

Discover how faceting extracts unique course levels, like computer science and electrical engineering, without loops, then push the results to the frontend via API.

Code along with AWS Lambda, Kinesis, and the Serverless Framework; embrace break and feedback to iteratively refine cloud workflows.

Develop a lambda-based API function to fetch course categories (graduate and undergraduate), expose a REST endpoint, and deliver a menu list to the frontend for dynamic dropdowns.

Learn to fetch a university's data by a unique object ID via an API function, configure which fields are searchable, and test by deploying and inspecting the function's path parameters.

Explore how serverless functions are hosted: Lambda triggered by a scheduler or Kinesis, with code in an S3 bucket, CloudFormation, and API Gateway endpoints managed by the Serverless Framework.

Develop an api function to filter university data by name, level, and scores, using explicit, searchable parameters; test deployment and refine indexing for accurate, front-end filtering.

Measure API latency across endpoints using lambda functions, monitor performance with the browser network tool, and confirm consistently fast response times.

Contrast the 90s data-centric approach with a modern serverless design using the serverless framework to create a decoupled, scalable, cost-effective system with searchable data and a high speed cache.

Explore how architectural design thinking shapes practical skills, as students share how the course influenced them, and provide feedback to shape future levels.