Microservices with NodeJS, React, Typescript and Kubernetes

Understand monolithic architecture as a traditional single-unit model where client, server, and database share one codebase. Emphasize easy deployment and debugging while noting complexity and potential cascading failures.

Explore microservices architecture, where each subdomain is a separate service with its own database, communicating through an API gateway via REST, GraphQL, WebSockets, and AMQP.

Explore microservices communication types, comparing synchronous request–response patterns using http/https with api gateways, and asynchronous messaging via events or queues like RabbitMQ or Kafka for 1-to-1 and 1-to-many patterns.

Identify the three main microservices communication styles: synchronous request–response via http rest apis through an api gateway; push and real-time via websockets; and asynchronous event-driven publish-subscribe with message queues.

Build a marketplace where buyers and sellers create and buy gigs and custom gigs, leveraging microservices with api gateway, gig, chat, order (stripe), notification, authentication, user, and reviews via elasticsearch.

Define and implement functional requirements for the app, including user authentication, profiles, search and filters, messaging, ratings, payments with Stripe, and order viewing with seller-initiated cancellations.

Non-functional requirements describe how the system performs, focusing on scalability, reliability, loading time, look and feel, availability, and usability for a strong user experience.

Design decisions route all client requests through an api gateway, use http and socket io, and employ rabbitmq for event-driven inter-service communication, with token management centralized at the gateway.

Understand the project’s microservices architecture, featuring an api gateway, one instance of Elasticsearch, Kibana, RabbitMQ, and services for auth, user, chat, gig, order, review, and notification, with docker compose.

Explore inter-process communication within microservices, contrast http-based messaging with event-driven patterns, and learn how RabbitMQ enables producers and consumers across auth, user, gig, order, review, chat, and notification services.

Explore local development tools and run them with Docker Compose, avoiding local installs, using VS Code to manage Elasticsearch, Kibana, MongoDB, MySQL, PostgreSQL, Redis, and RabbitMQ.

Install and configure Docker on your local machine, sign in to Docker Hub, and set up VS Code to prepare for Docker Compose and Kubernetes deployment.

Set up a Redis service with Docker Compose, using the official Redis alpine image, mapping port 6379, configuring a local volume at ./docker-volumes/cache/data for persistent storage.

Configure and run a MongoDB service with Docker Compose, including image selection, environment variables, port mapping, and volume persistence to a local Docker volumes folder.

Set up MySQL and Postgres services with Docker Compose using official images and the latest tag; configure root password, database, and user credentials, and map ports and volumes.

Deploy a rabbitmq service with the official management alpine image, expose amqp and management ports, skip data volumes, and prepare to set durable later if needed.

Set up Elasticsearch with a docker compose service, using the elastic official image for a single-node cluster version 8 or newer, and disable security for local development.

Learn to deploy Kibana with Docker, connect it to Elasticsearch, configure kibana.yaml, and visualize data from Elasticsearch via a dashboard.

Run multiple services with docker compose in detached mode using service names, manage image downloads, and handle Elasticsearch startup delays while monitoring logs in Docker desktop.

Learn how to set up and access Elasticsearch and Kibana, view indices, and explore dev tools and observability features like logs, metrics, and uptime for microservices.

Publish an npm library by creating a GitHub repository, adding code and exported properties, and configuring a GitHub actions pipeline, then install it with npm.

Create a helper library to house interfaces, cloudinary upload methods, error handlers, API gateway middleware, across microservices; publish with GitHub actions to npm and route logs to Elasticsearch and Kibana.

Set up the helper libraries project, configure package.json and tsconfig, install babel and TypeScript tooling, add express and jsonwebtoken types, and prepare build scripts for publishing to npm via GitHub.

Create a private GitHub repository for the project, copy the editorconfig contents, initialize git, configure .gitignore to exclude node_modules, logs, and build, then push and set up a token.

Configure a node npmrc file to securely authenticate to the npm registry using a GitHub token, exporting an npm_token environment variable for use in GitHub actions.

Learn how to organize helper interfaces for a NodeJS microservices stack, creating a server and interfaces folder, augmenting Express request with current user, and exporting nine interfaces via index.ts.

Learn to implement a Cloudinary-based upload helper in NodeJS, enabling image and video uploads with optional public id, automatic resource types, and chunked uploads, all returned as a promise.

Build a custom abstract error, an error response interface, and error classes (bad request, not found, not authorized, payload too large, server error) using http-status-codes.

implement gateway middleware to verify api gateway requests using json web token, checking a gateway token in headers and validating the payload id against allowed service tokens.

Set up a Winston-based logger to export logs to Elasticsearch and visualize them in Kibana, configuring console and Elasticsearch transports and a local node URL.

Add helpers.ts file with functions to convert first letter to uppercase, lowercase and uppercase, validate emails and data URLs, export for package.json versioning and GitHub actions.

Set up a GitHub actions workflow to publish your npm library to the GitHub registry by installing dependencies, building, configuring npmrc, and running npm publish on the main branch.

Publish a private npm package to the GitHub package registry by npm login with a token, pushing to main, and validating via GitHub Actions, then update version in package.json.

Move interfaces to the source root, update exports and build scripts, and configure babel presets for es modules and es5 to ensure correct builds and TypeScript types export.

Create the notification service and connect it to Elasticsearch and Kibana to view logs on the Kibana dashboard. Set up RabbitMQ queues and test message consumption, and develop email templates.

Discover how the notification service manages email delivery, including verification, forgot password, password reset success, order alerts, offers, receipts, extension requests, and delivery notices between buyers and sellers.

Set up the notification service folder, initialize npm and TypeScript configurations, and integrate eslint and prettier, along with editorconfig and gitignore, to streamline service development.

Learn to set up a TypeScript node service with path aliasing, install nodemon and tsconfig paths, configure a transform for imports, and add dev and build scripts with logs.

Set up a src folder for the notification service, adding server.ts, app.ts, config.ts, routes.ts, helpers.ts, and Elasticsearch.ts, plus emails and queues folders, with express and a health route for Kubernetes.

Set up a root dot env and define environment variables for the notification service, then create a config class to expose them for connecting to RabbitMQ and Elasticsearch.

Set up a node server with express, env configuration, and error handling, configure a Winston logger connected to Elasticsearch, and start the server, queues, and Elasticsearch integration.

Set up a notification service health route with an express router to monitor service health, bypassing the API gateway and returning 200 or 500 for Kubernetes monitoring.

Learn to connect the Elasticsearch notification service using the JavaScript client, configure docker-compose and cloud or kubernetes deployment options, and implement a check connection that logs cluster health to Kibana.

Configure and run a notification service by wiring up an Elasticsearch client, health checks, and a Winston logger, then initialize an Express app and verify startup and logs in Kibana.

Set up a RabbitMQ connection in the notification service using the Amqp library, connect to the configured endpoint, create a channel, and handle errors with logging and graceful shutdown.

Connect producers, queues, exchanges, and consumers with RabbitMQ as a message broker, routing messages via routing keys and exchange types like direct and fanout.

Learn RabbitMQ channel methods for the notification service: publish to exchanges, assert and bind queues, set up consumers, and acknowledge messages, covering direct, fanout, topic, and headers exchanges.

Develop an amqp-based email consumer that processes auth service events to trigger verification emails via the notification service, using a durable direct exchange, routing key, and queue.

Use the auth email consumer method to wire a RabbitMQ channel in server.ts, assert the exchange, publish and consume messages, and acknowledge delivery.

Build an order email consumer that processes order and chat messages from an order notification exchange, routing to the order email queue for eventual email delivery.

Build a mail transport using node mailer for development and Kubernetes deployment, wiring in email templates and a mail.transport.ts module to send emails with configurable locals.

Develop and test email templates using the ejs templating engine with express, including subjects, links, and icons, and validate via ethereal with node mailer transport.

Learn to send templated emails by calling email.send with a template, receiver, and locales such as app link, app icon, and username, and specify the template path and name.

This lecture shows how to use the send email method inside email consumers, destructure properties, set locals, and send templates with verify and reset links.

Implement a send email method to consume messages from the chat service, sending an order placed to the seller and an order receipt to the buyer when payment succeeds.

Install and configure jest with ts-jest for a NodeJS microservices stack, set up npm test and code coverage, and plan one unit test per service for a Jenkins pipeline.

Create a unit test for the email consumer using jest, mocking the connection, exchange, and queue to verify the consume workflow without a real broker.

Unit test an email consumer in a Node.js microservices stack by mocking amqp channels, asserting exchange, queue bindings, and consumption behavior using Jest.

Create a local development dockerfile and use docker compose to run services, adding a dot docker ignore file and npm install with nodemon. Note the production dockerfile will differ.

Add a notification service to docker compose with a build context to the notification folder using dockerfile.dev, map port 4001, and set the env file and elasticsearch and rabbitmq dependencies.

Learn to build a production-ready multi-stage dockerfile for node apps, using npm ci --production, stage-based builds, and pm2 to run the final image efficiently.

Add package.json scripts for lint and prettier checks with fix options. Start the app in production with PM2, and include stop and delete commands.

Fix the notification service by copying the emails templates into the build/source folder during npm build, using a ts script with shelljs to ensure emails render in deployed environments.

Push code to GitHub to enable deployment to Kubernetes and CI/CD pipelines; initialize git, respect gitignore rules, create separate repos per service, and integrate with docker compose and Jenkins.

Discover API gateway architecture for a microservice, routing all client requests through the gateway via HTTP and socket.io, connecting to Elasticsearch and Kibana, with object oriented controllers, routes, and services.

Set up the gateway service folder for the api gateway initial setup, reuse notification service configs, and configure port 4000, tsconfig, and package scripts while planning future route additions.

Start building the API gateway by creating a gateway server class in server.ts, configure a cookie-based JWT session, and apply security middlewares like helmet and cors.

Gateway server file part 2 adds compression and json/urlencoded middleware with size limits, wires routes and start Elasticsearch, and implements a private error handler with custom error serialization.

Implement start server and start http server in the gateway file, wire express and http, initialize the application, and confirm the gateway starts on port 4000 with a process id.

Set up gateway environment variables and create config.ts, including gateway and jwt tokens, service base URLs, redis, and elasticsearch settings, then integrate config into the gateway server.

Create a gateway service Elasticsearch connection with a class-based client, initialize the client from a configured URL, and implement a check connection and health status with logs.

Create the initial api gateway health route using Express in a Node.js TypeScript project, organize controllers and routes, and surface a gateway health endpoint for Kubernetes liveness/readiness probes.

Implement a TypeScript auth middleware that verifies the client and gateway tokens via JWT, sets request.currentUser from the payload, and enforces authentication for protected routes.

Create an Axios service in the API gateway to attach a JWT-signed gateway token to request headers for microservice calls, using base URL, service name, and withCredentials.

Duplicate and configure the gateway service in Docker Compose, set port 4000, adjust the EMV file, and use Redis and Elasticsearch containers with the JWT token for the gateway.

Initialize a git repo for the gateway service, create a private global gateway on GitHub, and push the existing code with dockerignore and editorconfig.

Explore the authentication service, implementing signup, sign-in, password reset, email verification, token refresh, and current user checks, plus a search feature using Elasticsearch and MongoDB.

Set up the auth service project by duplicating gateway service configurations, configuring docker and npm, and preparing environment variables for API gateway URL, Elasticsearch, RabbitMQ, and Cloudinary.

Set up a MySQL database using Sequelize ORM, create a database connection with an async function, enable multiple statements, and log the connection status for Kubernetes deployment.

Fix the sequelize database setup by adding the MySQL database URL to the Sequelize instance, using process.env or config, to prevent connection errors.

Establish an Elasticsearch connection by exporting a shared client, logging the connection process, and implementing a health check that monitors cluster status (yellow or green) to ensure readiness.

Configure a security middleware on port 4002 to validate a bearer JWT from the authorization header and set req.currentUser, with cors, helmet, and API gateway integration.

Configure a secure microservice gateway with security and standard middleware, json and urlencoded handling, compression, queues, and a gateway error handler logging to elasticsearch and kibana.

This lecture demonstrates creating the app entry file, building root and app roots middleware, wiring auth service routes, and wiring a database connection in the startup sequence.

Implement signup validation schemas using joy to validate user data on front end and back end, covering username, password, email, country, and profile picture.

This lecture shows how to build a login schema using joy to validate the sign-in object, with a username field that accepts a username or an email and conditional validation.

Define a Sequelize model for a MySQL auth table, detailing fields such as id, username, email, password, country, profile picture and verification tokens, then sync, timestamps, and indexes.

Create a Sequelize auth model in models/auth.schema.ts for MySQL/SQLite, defining username, password, email, profile picture public id, country, email verification token, email verified boolean, and optional password reset fields.

Add unique indexes on email and username to speed searches. Before create, hash passwords with bcrypt using a salt rounds setting, and expose a compare password method for login.

Fix auth model prototype error by defining an extended interface for the Sequelize model, adding prototype methods compare password and hash password, and using a union type to apply them.

Start the ott service server with docker, configure a mysql url in the dot env file, and connect to mysql via tableplus to verify the database is ready.

Create and reuse a RabbitMQ connection in the art service, establish a channel, and implement a direct exchange producer to publish messages to notifications and users.

Implement the OTT service's create user workflow: create and store a user in MongoDB, publish a buyer event to the buyer exchange, and return the user data without the password.

Implement auth service get methods to retrieve user data by id, by user name or email, and by verification or password reset tokens, excluding passwords.

Implement update methods to modify email verified status and email verification token. Create a sign token function to generate a signed JWT with ID, email, and username.

Explore the auth service endpoints and how the front end communicates with the API gateway to the auth service, detailing sign up, sign in, verify email, and password reset routes.

Create a signup controller that validates input with Joi, checks for existing users, uploads a base64 profile picture to Cloudinary using a uuid-based public ID, and publishes a notification.

Generate a 20-byte hex email verification token with crypto, create the user with uploaded profile picture URL, publish the verify email message, and return a signed JWT with status 201.

Create the auth signup route and hook it to the API gateway by defining a post /sign up route under /api/v1/auth, using gateway verification and bearer token handling.

define auth gateway methods using axios auth and axios service, covering current user, refresh token, signup, sign in, forgot password, reset password, email resend, gig queries, and seed endpoint.

Create a gateway sign-up route by adding an auth controller, wiring the sign-up service, and storing JWT in the cookie session for API gateway to forward to auth service.

Obtain cloud name, api key, and api secret from cloudinary and add them to your environment file. Configure cloudinary in config.ts, initialize in app.ts, and test with a rest client.

Update the shared library gateway method to api/gateway/v1, configure the gateway jwt token, convert the header to lowercase, and add optional chaining with nest on success and a version bump.

Fix gateway integration by correcting the API path to api/gateway/v1 and adding the gateway JWT secret to the shared library, while outlining versioning and cross-service updates.

Develop a sign-in controller that validates login input, supports email or username, authenticates the user, issues a token, and returns secure user data.

Add a verify email controller that reads the token from the request body, validates it, sets email_verified to true, clears the token, and returns the updated user.

Implement a password controller to handle forgot password, reset password, and change password, validating email, generating a token, creating a reset link, and notifying the user via email.

Validate and match the new passwords, and retrieve the user by reset token. Hash the new password, update the user record, and send a reset password success email.

Add a change password method to the authentication controller, using current and new password from the authenticated user, and expose forgot, reset, and change password routes in the api gateway.

Add a protected current user read method to fetch authenticated user data and a resend email method to generate a verification link and send the email.

Add and secure the current user controller in the API gateway by wiring the read and resend email routes with check authentication and verify user middleware to ensure authenticated access.

Create a health and a refresh token controller that fetches a user by username, signs a token, and returns it to the API gateway with routing and unit tests.

Add a focused unit test for the current user controller by building a custom express http request and response mock with typed interfaces and a payload, using jest.

tests the current user read method by mocking the auth service to return an existing user or null, and verifying the response status and the user payload.

Debug and fix a jest environment teardown error by conditionally running model.sync only outside the test environment, using process.env.NODE_ENV. Alternatively, mock a database connection and manage syncing with before/after hooks.

Learn to test the resend email method across scenarios such as invalid email, bad request handling, verify email updates, and get user by email or ID in a microservices environment.

Initialize a git repository, create a GitHub project, and push the service and API gateway code. Update docker compose to include the auth service and set container names and ports.

Explore Elasticsearch and Kibana dashboards by using the JavaScript Elasticsearch client to create and manage indices, index documents, and perform get, update, delete, and search, with Kibana dev tools.

Create the gigs index in Elasticsearch using the JavaScript client, check for existence, create if missing, refresh, and log info.

Add test data to the gigs index in elastic search using the gigs service via rest api. Create documents with custom IDs or auto IDs and verify results with gigs/_search.

Explore how to implement a get document by id method in Elasticsearch to retrieve a specific gig from an index, returning the source data via a search service.

Upload gigs data by importing geeks.json into Elasticsearch with elastic dump after deleting and recreating the gigs index, creating 30 documents with structured fields.

Explore the Elasticsearch search API, including query basics, pagination limits of 10,000 hits, and how to target fields like user name, title, description, and price for refined results.

Explore building a gigs search query in the search service, implementing Gibbs search with delivery time and min/max price filters, and return types for gigs and seller gigs.

Build Elasticsearch queries to filter gigs by expected delivery and price, combine query conditions, and implement forward and backward pagination with search after and custom sort ID.

Develop a search controller using the elastic search client to return total hits and item sources, with pagination, delivery time and price filters, and backward sorting by id.

Learn to add search routes in the auth service, creating /search/gig and /search/gig/:gigId endpoints with from, size, and type parameters and connect to the gigs controller.

Add a search controller and routes in the API gateway to query gigs from the auth service, constructing a query string with delivery_time, minimum_price, and maximum_price.

Demonstrates testing the search endpoint by formatting request queries as a string for the auth service, logging before and after values, and validating results through the api gateway routes.

Set up a seed controller in the auth service to generate MySQL users, using Faker and a unique username generator, with optional MongoDB buyer creation and testing notes.

Enhance the auth seed controller by randomly setting email verification for seed users with Lodash. Expose a seed endpoint through the API gateway to create users in the MySQL database.

Create and test the auth seed endpoint using a put seed/{count} to populate seed users and publish updates through RabbitMQ exchanges and queues.

Pushes updates to GitHub after adding authentication, search, and seeding method and endpoints, commits with a clear message, and restarts VS Code to finalize OTT and API gateway changes.

Set up a dedicated users service by copying configurations from the OTT and auth service, initializing npm, installing dependencies, adjusting docker port to 4003, and updating tsconfig/jsconfig for users.

Add the src folder and update the .env to use MongoDB via Mongoose in a dockerized container. Configure Redis settings and Cloudinary, and establish the database connection.

Set up an Elasticsearch connection, reuse and adapt existing modules, implement a health check loop, and wire buyer and seller routes through the API gateway.

Set up the user service server.ts by adapting the existing server, implement security middleware and elasticsearch startup, convert to arrow functions, and run npm run dev to verify database connection.

Create a buyer schema with mongoose to store auth service data in MongoDB, including username, email, profile picture, country, is seller default false, and purchased gigs.

Learn to build a seller schema in part 1 by adding fields such as full name, user name, email, profile picture, description, country, languages, skills, ratings, and experience history.

Extend the seller schema with education, social links, and certificates, and add job metrics, earnings, and created ads, while evaluating embedding vs separate collections and removing the version key.

Learn to validate seller data through a complete validation scheme, extending the seller schema with ratings, skills, languages, and required fields to ensure consistent api gateway and user service data.

Implement the buyer service with methods to get a buyer by email and by username, seed random buyers using MongoDB's aggregate and the sample operator, and create a new buyer.

Implement two buyer service methods: set seller to true when a buyer becomes a seller, and update the purchased gigs array by adding or removing gig IDs.

Learn to implement seller service get methods in MongoDB, including by id, by user name, and by email, plus get random sellers, and to create and update sellers.

Learn to implement seller update methods using find one and update, increment, and set to modify seller documents and return the updated record; update gigs, jobs, delivery, and ratings.

Add buyer service to the api gateway and configure an axios instance with base URL and bearer token. Implement get current buyer by user name and get buyer by email.

Integrate the seller service into the api gateway with an axios-based client, exposing get by id, get by username, random, create, update, and seed endpoints, with bearer token authentication.

Set up a RabbitMQ queue connection in the user service with a producer and multiple consumers, publishing to the gig service via a direct exchange with routing key.

Create a RabbitMQ buyer message consumer that binds the exchange jobber buyer update to the user buyer queue and processes auth service user events to create or update buyers.

Set up a RabbitMQ seller message consumer that listens for direct messages from the order service to create, approve, update, or cancel orders and refresh ongoing and completed jobs.

Implement a RabbitMQ review message consumer using a fanout exchange to broadcast buyer reviews from the review service, updating seller reviews and publishing to the gig service.

Explore building buyer controller routes in a microservices architecture, implementing get by email, get by current user, and get by username, with routes, a health route, and API gateway integration.

Create robust seller controllers with validation, existence checks, and responsive error handling, then implement updates that restrict username and email changes while wiring routes for create and update.

Define distinct routes for seller by id, by username, and random to prevent the router from misrouting. Understand how route order and dynamic parameters affect matching.

Implement a seller get controller with endpoints to fetch a seller by id, by user name, and to retrieve random sellers, with proper route definitions and status messages.

Build a seed controller to seed sellers data by generating random experiences and random education with faker and lodash, including dates and descriptions.

Seed seller data with a NodeJS controller, generating sellers from random bias, validating uniqueness, and wiring the seed route for the API gateway in a microservices stack.

Add the seller controller and routes to the api gateway service, implementing create, update, get by id, get by user name, and random size-based cells retrieval, plus seeds.

Seed http test demonstrates seeding users and sellers through the api gateway, verifying published messages to the user service and MongoDB stores (11 buyers, 10 sellers) while all services run.

Implement a focused unit test for the buyer get controller, using mocks and jest to verify that a valid buyer document returns a 200 status and profile data in json.

Push the user service code to a private GitHub repository, initialize git, commit, and push, then configure a docker-compose local infrastructure with MongoDB, Redis, Elasticsearch, gateway, RabbitMQ, and MySQL containers.