NestJS Microservices: Build a Distributed Job Engine

Define a Prisma schema to connect a NestJS auth service to Postgres, generate types with the Prisma Client, use migrations, and model a user with id, email (unique), and password.

Learn to extend an NX workspace with Prisma targets, generate types from a Prisma schema, run migrations, and wire Prisma client into a NestJS app to manage users in Postgres.

Integrate GraphQL with the NestJS auth microservice using Apollo, enabling auto schema generation. Create an NX library for shared GraphQL models, including an abstract model with an id.

Create a GraphQL user model and resolver in a NestJS app using NX, expose email, keep password not queryable, and query all users via the users resolver and GraphQL playground.

Implement a create user mutation in a NestJS GraphQL API with an input type, class-validator validation, and bcrypt password hashing, persisting via Prisma; then expose a get users query.

Set up commit hooks in the nx workspace of a NestJS microservices project using husky and lint-staged to lint and format only changed files with eslint, prettier, and nx targets.

Fixes failing NestJS unit tests by updating the NX workspaces Jest preset to allow no tests, generating Prisma types before tests, and mocking Prisma services in the testing module.

Use the NestJS config module to read environment variables via the config service, inject it into providers, and validate the auth port defined in the env file for startup.

Set the NX native command runner to false in the dot env to prevent cross-platform issues, and set run build target dependencies to true for GraphQL auto schema generation.

Set up NestJS JWT module in the auth microservice, read secret and expiration from config, and expose a login mutation that issues a signed JWT on valid credentials.

Create a per-service dot env for auth service, move its variables there, remove the prefix, and restart to verify NestJS config loads env with database URL, port, and jwt settings.

Implement user verification in a NestJS GraphQL app by authenticating credentials, issuing an HTTP-only JWT cookie, and returning the user.

Generate a JWT with a user id payload and set it as an http-only, secure cookie that expires with the token, using NestJS config and JWT services.

Implement a JWT strategy in NestJS using passport to parse cookies, verify tokens with a secret, and guard GraphQL resolvers with a gql auth guard.

Create a custom current user decorator to extract the authenticated user from the GraphQL request via the execution context, using the JWT strategy token payload.

Configure default GraphQL playground settings in a Next.js app by defining a Playground Options object and setting request.credentials to include, ensuring HTTP only cookies are applied on startup.

Create a jobs microservice with nx nest generator as the entry point to list and trigger jobs via postman or UI, and configure main.ts with validation and port 3001.

Build a NestJS job system with a custom job decorator and metadata, a base abstract class, and a service to retrieve decorated jobs for HTTP-triggered execution.

Discover and expose decorated job providers at runtime using go level up nestjs discovery. Initialize discovery module, fetch providers by metadata, and access the underlying job instance to execute it.

Explore building a NestJS jobs resolver for a distributed job engine: define a job model, create GraphQL queries and mutations to fetch and execute jobs, and wire the GraphQL module.

Configure default graphql playground options by creating a playground object and a settings object, enabling request credentials to include http cookies for the back-end graphql server.

Set up a gRPC based authentication where the jobs microservice sends a JWT to the auth microservice for validation, then proceeds on success.

Generate TypeScript types from protobuf files using a global NX script and a new TS proto target; automate with caching, project root inputs, and GitHub workflow integration for NestJS microservices.

Configure a hybrid NestJS auth microservice to serve http and gRPC, wiring the gRPC transport to the auth proto. Copy the proto assets to dist for client and server use.

Configure a gRPC message controller in the auth microservice to receive authentication requests and expose the authenticate method via TS proto, enabling a JWT-based inter-service authentication.

Implement a GraphQL specific auth guard in NestJS that uses a gRPC client to authenticate JWT cookies via an auth microservice, wiring canActivate, onModuleInit, and GraphQL context for request validation.

Implement end-to-end authentication across NestJS microservices by adapting the GraphQL auth guard to gRPC transport, extracting JWT from token or cookies, and setting the current user on the request.

Create a singleton pulsar client and a reusable producer to publish messages to a pulsar topic, enabling horizontal scaling with additional consumers.

Create a pulsar producer in the abstract job to publish GraphQL input as a buffered message to a job topic, enabling downstream consumption.

Develop a job executor service that consumes Pulsar messages via a new abstract Pulsar consumer, enabling shared (round-robin) subscriptions under the jobber name and centralized client management.

Create a new NestJS executor microservice with a shared bootstrap pattern using an init function, and implement a Fibonacci Pulsar consumer for exactly-once message processing.

Extend GraphQL job execution mutation to accept dynamic input by adding a JSON scalar from the GraphQL type JSON library, and pass this data through pulsar messaging to job executor.

Implement robust message handling in NestJS pulsar consumer by adding a private listener that deserializes incoming messages, logs details, handles errors with try/catch, and always acknowledges to avoid reprocessing.

Validate incoming GraphQL job data at runtime using class-validator and class-transformer in a NestJS microservices setup, turning the Fibonacci data into a validated class and returning 400 on errors.

Refactor the Pulsar batch producer to handle an object or array of messages in a GraphQL request, enabling batch publishing, backlog, and scalable asynchronous processing of a distributed job engine.

NestJS microservices guide details building a distributed job engine, generating a large Fibonacci backlog via a script, logging in, and measuring throughput with Pulsar, Docker, and Kubernetes.

Dockerize a Node.js NestJS microservices app and scale with Kubernetes by running multiple dockerized instances as Pulsar consumers, using NX with npm workspaces for per-service package.jsons.

Restructure an nx workspace by creating GraphQL, NestJS, gRPC, and Prisma libraries, moving common bootstrap code, and centralizing dependencies in the root package.json to avoid duplication.

Move proto definitions and types into gRPC library, remove unused modules, and streamline npm workspaces and nx targets to generate TS proto, copy proto files to dist, and optimize builds.

Define webpack configs for each library, update tsconfig paths, and build with nx to emit dist/libs outputs. Run parallel builds with run-many, fix path aliases, and compile all libraries successfully.

Refactor the auth app to use NX dependency management for gRPC and Prisma types, then build all apps and libraries with NX for consistent dist output.

Install and configure module alias at the project root, map custom paths in package.json, and update webpack and proto paths for dist libs and NestJS microservices.

Build a lean dockerized jobs service using a multi-stage node 18 slim image, copying workspace and apps and libraries files, running NX build, and installing only production dependencies with npm.

Finish dockerfiles for the auth and executor services by adapting the jobs template, switching to prisma, building dist, and starting the apps with node in production mode.

Adopt a common webpack configuration with webpack merge to inherit root settings for libraries and apps, standardize output, and support npm start and docker builds.

Implement a NestJS Pinot HTTP logger to log every request with a correlation ID, using development and production transports, via a reusable logger module.

Implement a custom GraphQL logger plugin for nestjs microservices to log graphql requests and responses, including headers, query, variables, duration, and status, across jobs and auth services.

Create a NestJS gRPC logging interceptor to log each request and response using a uuid v4 correlation id and RxJS tap, then apply it to the gRPC controller.