Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
How to Build 1 Million Requests per Minute with Golang
Rating: 2.0 out of 5(5 ratings)
47 students

How to Build 1 Million Requests per Minute with Golang

Go: the Great Stack for high load applications
Last updated 12/2020
English

What you'll learn

  • In the end of my course you will learn how to build a application with Go. You get knowliges in building high load application in which you organized the application logic into multiple packages to easily maintain the application.

Course content

1 section8 lectures30m total length
  • Introduction1:17
  • More introduction12:00

    Explore building a Golang system to handle one million requests per minute, addressing memory leaks, telemetry, and scalable queues with channels, workers, and an elastic load balancer.

  • Handle function6:19
  • Memory leak2:54

    analyze memory leak issues in high-traffic api paths, tracing across mobile clients, gateways, and services, and implement strategies to reduce memory usage, improve response times, and prevent outages.

  • Http client1:07
  • Close resp body1:33
  • Time channel2:58
  • Time context2:36

Requirements

  • You should have basic level of knowledge in http and go language

Description

The Problem

While working on a piece of our anonymous telemetry and analytics system, our goal was to be able to handle a large amount of POST requests from millions of endpoints. The web handler would receive a JSON document that may contain a collection of many payloads that needed to be written to Amazon S3, in order for our map-reduce systems to later operate on this data.

Traditionally we would look into creating a worker-tier architecture, utilizing things such as:

  • Sidekiq

  • Resque

  • DelayedJob

  • Elasticbeanstalk Worker Tier

  • RabbitMQ

  • and so on…

And setup 2 different clusters, one for the web front-end and another for the workers, so we can scale up the amount of background work we can handle.

But since the beginning, our team knew that we should do this in Go because during the discussion phases we saw this could be potentially a very large traffic system. I have been using Go for about 2 years or so, and we had developed a few systems here at work but none that would get this amount of load.

Conclusion

Simplicity always wins in my book. We could have designed a complex system with many queues, background workers, complex deployments, but instead we decided to leverage the power of Elasticbeanstalk auto-scaling and the efficiency and simple approach to concurrency that Golang provides us out of the box.


It’s not everyday that you have a cluster of only 4 machines, that are probably much less powerful than my current MacBook Pro, handling POST requests writing to an Amazon S3 bucket 1 million times every minute.

Who this course is for:

  • It can be helpful from high school level