How to Build 1 Million Requests per Minute with Golang
What you'll learn
- In the end of my course you will learn how to build a application with Go. You get knowliges in building high load application in which you organized the application logic into multiple packages to easily maintain the application.
Requirements
- You should have basic level of knowledge in http and go language
Description
The Problem
While working on a piece of our anonymous telemetry and analytics system, our goal was to be able to handle a large amount of POST requests from millions of endpoints. The web handler would receive a JSON document that may contain a collection of many payloads that needed to be written to Amazon S3, in order for our map-reduce systems to later operate on this data.
Traditionally we would look into creating a worker-tier architecture, utilizing things such as:
Sidekiq
Resque
DelayedJob
Elasticbeanstalk Worker Tier
RabbitMQ
and so on…
And setup 2 different clusters, one for the web front-end and another for the workers, so we can scale up the amount of background work we can handle.
But since the beginning, our team knew that we should do this in Go because during the discussion phases we saw this could be potentially a very large traffic system. I have been using Go for about 2 years or so, and we had developed a few systems here at work but none that would get this amount of load.
Conclusion
Simplicity always wins in my book. We could have designed a complex system with many queues, background workers, complex deployments, but instead we decided to leverage the power of Elasticbeanstalk auto-scaling and the efficiency and simple approach to concurrency that Golang provides us out of the box.
It’s not everyday that you have a cluster of only 4 machines, that are probably much less powerful than my current MacBook Pro, handling POST requests writing to an Amazon S3 bucket 1 million times every minute.
Who this course is for:
- It can be helpful from high school level
Instructor
I’m a solutions architect focused on building highly scalable native applications with a special in APIs, Microservices, containerized applications, and distributed systems. Currently specializes in NodeJS, JS, PHP & Golang. I’m passionate about building scalable back-end systems and Microservices in Go And NodeJS. I’m pragmatic minimalist who focuses on real-world practices for architecting solutions.
Worked with Erlang, Prolog ,Java, NodeJS, PHP, JS, Golang.
Web Servers: Apache HTTP Server, NGINX
Databases: PostgreSQL, Redis, MySQL, MongoDB
Back-End Programming Languages: PHP, Golang, Erlang, NodeJS
Back-End Development Skills: Google Web Toolkit (GWT), Docker, Git, RESTful, Kubectl, Kubernetes
Back-End Development Deliverables: Web Services, API, Database Architecture: RESTful API, PHP Script, Database
Other skills: Backend Rest APIArchitecture Design