Python Microservices: Breaking a Monolith to Microservices

Set up the Django ambassador project using docker, clone the front-end variants, update the URLs, install dependencies, and run migrations to populate the database and access the Django admin.

Log into Stripe test mode, replace the publishable key on the front end and the secret key on the back end, then complete a payment flow that redirects to Stripe.

Explore the codebase for breaking a monolith into microservices, review the common core, authentication endpoints, shared models, and Redis-backed product handling across admin and ambassador services.

Compare monoliths and microservices by illustrating a feature-rich monolith and its split into dedicated services, scaled via internal API calls or an event bus like Kafka, each in different languages.

Build the first microservice: an email service connected via Apache Kafka. Learn how to split a monolith and enable inter-service communication with a Kafka event bus.

Set up the email microservice for a Python monolith split by configuring docker compose and docker files, refining requirements, and preparing Kafka settings for email routing.

Set up Kafka on Google Cloud and Confluent Cloud for microservices deployment. Create a project, configure a Europe cluster, define topics, and produce and consume messages with credentials.

Learn to connect to Confluent Cloud from the terminal using the Confluent Cloud CLI, set the environment and cluster, create topics, and run producer and consumer to send messages.

Learn to set up a Confluent Kafka Python consumer, configure bootstrap servers and security, subscribe to a topic, pull and print messages, and test consumption with a produced message.

Create a Python producer using confluent kafka to publish orders to a topic, convert orders to dictionaries and JSON, and flush after sending.

Import and call the two send mail functions to deliver orders from the email microservice, including admin and ambassador revenue in the payload.

Create the users microservice, update the diagram, and ensure all other microservices access it via internal HDP requests to maintain user consistency instead of relying on Kafka-driven events.

Set up the users project, configure django with docker files and docker compose, connect the users database, run migrations, and import data from jungle ambassador for bootstrapping.

Import existing user data from the old Ambassador database into the new microservice by configuring a port, adding an old database connection, and running a populate_users script to migrate users.

Move views to the microservice architecture by updating api urls, copying authentication serializers, removing the ambassador column, and delegating user revenue calculation to the Django ambassador.

Test user registration across microservices by running the server, installing requests, posting to the register API, and adjusting allowed hosts to successfully create admin users.

learn how to implement login by generating a jwt, setting it in cookies, and handling the login flow across microservices with scope and request data.

Learn how to fetch the authenticated user in a microservices setup by sending the JWT cookie in request headers, validating authentication, and using the API/user endpoint.

Create a database-backed user token model to replace cookie-based logout, including user id, token, expiry, and migrations, and validate tokens during authentication while deleting tokens on logout.

Refactor the user API into a dedicated user service with a centralized endpoint, static get/post methods, and dynamic header handling to support login, logout, registration, and profile updates.

Create a general user API view in the Python microservices course, returning all users or a single user by primary key, filter for ambassadors, and prepare for authentication via middleware.

Learn how to implement a custom middleware that fetches the authenticated user from the user service and attaches it to the request for easy access across views.

Remove the user from the ambassador app, add endpoint to fetch users by primary key, remove foreign keys, and migrate user data to the user microservice, updating models and serializers.

Configure external docker networks and assign container names to connect microservices, such as the user microservice on port 8000. Demonstrate exposing or omitting ports for direct container access.

Enforce scopes and permissions across microservices by embedding scope in the URL and validating the JWT payload to prevent admin and ambassador cross-access.

Completed the user microservice and will split the admin microservice into a new project with its own database and Kafka communication.

Set up the admin microservice, adjust docker compose and requirements, create core project migrations, and configure models and admin database for a cross-service Python microservices architecture.

Migrate data from the monolith to microservices by populating the user, orders, and products databases in the respective microservices, mapping order and order item data from the old database.

Move the admin views and related services, middleware, and serializers into the core admin setup. Test endpoints; plan to clear the ambassador cache via Kafka.

Implement a Kafka consumer in the admin microservice, set up admin and email topics in Confluent Cloud, and process link creation and order creation events.

Explain how to replace inline commands with a separate Docker Compose service to run a Python Kafka consumer alongside the server, enabling admin queue testing via events.

Learn to modularize event listeners, create dedicated functions like link created, process messages with data serialization, and publish events to topics via a Django ambassador and confluent cloud.

Split the monolith into ambassador and checkout microservices, with each front-end having its own backend, and all communicating with the users service.

Create and configure the ambassador microservice, set up docker compose with the ambassador network, tailor the models to track each ambassador's total earnings, run migrations, and populate the database.

Import data by using Roskill queries to fetch orders and their items. Compute the total as the sum of order item ambassador revenues, and mark orders as complete.

Import views and related components to set up ambassador revenue calculations in Django, including serializers, caching, middleware, urls, and a management command to update rankings.

Integrate Kafka by wiring a product producer from Jungle Ambassador to publish product created, updated, and deleted events to the ambassador topic, with listeners updating the ambassador and cache.

Learn how syncing data across microservices works by creating, updating, and deleting products via endpoints, cache invalidation, and Kafka-driven events, with Docker and queue listeners.

Capture failed Kafka events in a Kafka error model, store the key, value, and error in the database, and use a cron job to replay them for consistency.

Set up the checkout microservice by scaffolding a Django project, configuring Docker Compose and the Dockerfile, wiring the checkout database, creating models, and running migrations.

Import views and serializers, migrate tables, populate orders and products, wire URLs and the user service as part of breaking a monolith into microservices, with Kafka coming next.

Set up Kafka integration in a microservice architecture, creating checkout, admin, and ambassador topics, wiring producers and listeners, and routing product and link events to multiple topics via dockerized services.

Test the front-end across admin, ambassador, and checkout microservices, verifying login, product creation, and the checkout flow, and fix serializer issues by switching to product related fields to generate correct links.

Learn how Google Cloud's free credits enable testing your app and how Kubernetes orchestrates Docker containers to deploy microservices; install Kubernetes via Docker Desktop and prepare a Kubernetes folder.

Convert a development docker compose to production by removing volumes and properly tagging and naming the image for a kubernetes pod deployment, then apply with kubectl and observe container recreation.

Configure Docker compose production to supply kafka credentials, email credentials (mailtrap), and bootstrap servers as environment variables, retrieve values with getenv, and update the Kubernetes deployment to image version 0.0.2.

Learn to convert Docker Compose files to Kubernetes with Kompose, handle environment variables, and adjust deployment files to run microservices as Kubernetes deployments and pods.

Learn to manage Kubernetes secrets for microservices by creating generic secrets for Kafka bootstrap servers and email credentials, including username and password, reusing values across services for secure deployments.

Deploy Kubernetes files to Google Cloud by creating a project, enabling Google Kubernetes Engine, and deploying Docker-based services on Autopilot clusters in Europe for microservices.

Learn to deploy a container on Google Cloud Shell by configuring gcloud credentials, creating the email.yaml, and addressing image pull issues before pushing the production Docker image to Google Cloud.

Install the Google Cloud SDK, add the bin path, and log in with gcloud to authenticate and set the current project for deploying Docker images to the container registry.

Push the built image to Google Container Registry, configure secrets for the email microservice, and adjust the YAML with resource settings to ensure deployment readiness.

Deploy the user microservice with Docker Compose for production. Configure a Google Cloud SQL database, set environment variables, and authorize your IP for access via port 3306.

Connect to the cloud and the Squirrel database, populate data, run migrations, and build Kubernetes files for the user microservice, exposing port 1000 for deployment to Google Cloud.

Push the users image to the Google Cloud Registry, deploy a users deployment and service in Kubernetes, and verify workloads and services before testing with ingress in the next tutorial.

Use a Kubernetes ingress as a central load balancer to route requests to multiple microservices under a single URL, mapping the /api/users path to the users service on port 8001.

enable a private IP connection to the database from the Django app in Kubernetes, and explain why this avoids the Docker container's external IP mismatch.

Demonstrates fixing subfolder routing for a Python microservice by configuring a script name, adding a success view, and deploying a new 0.4 image on Google Cloud with Kubernetes.

Create and deploy the admin microservice by adding a docker file, using the admin image version 0.0.1, and configuring Kafka with producer, consumer, and the admin topic; prepare db credentials.

Convert Docker Compose to Kubernetes files, run migrations and data seeding for orders and products, configure secrets for Kafka and bootstrap, and deploy the admin service to Google Cloud.

deploy the admin microservice on google cloud, update yaml for latest images and private database ip, apply configurations, and validate user registration and product retrieval.

Deploys the ambassador by updating the docker compose for ambassador, configuring environment values, databases, kafka topic ambassador, and redis host, then prepares Kubernetes files for deployment.

Configure docker compose and kubernetes files, run migrations and seed data, build and push images to Google Cloud Registry, and deploy ambassador with Kubernetes.

Deploy the ambassador microservice by creating YAML configs, configuring images and Redis endpoints, and validate user registration, Kafka-driven product updates, and checkout across front-end and admin services.

Set up the checkout microservice with Docker Compose, clone the database, and configure environment values for topics, credentials, hosts, and api endpoints, preparing for database creation and Kubernetes files.

Populate the database with docker compose and run python manage.py migrate, then build and push the checkout image to Google Cloud Registry, and convert to Kubernetes files for deployment.

Deploy microservices on Google Cloud by adding files, updating the user service URL, and wiring the database for checkout deployment.

Test the frontend by updating endpoints, validating the admin, ambassador, and checkout flows, and resolving login and cookie issues to demonstrate successful monolith to microservices migration.