Udemy
    •  
    •  
    •  
    •  
    •  
    •  
    •  
    •  
Subscribe
Turn what you know into an opportunity and reach millions around the world.
Learn More
Your cart is empty.
Keep shopping
Concurrent and Parallel Programming in Python
Rating: 4.3 out of 5(793 ratings)
6,507 students

Concurrent and Parallel Programming in Python

Speed up your programs with concurrency
Created byMax A
Last updated 9/2021
English

What you'll learn

  • How to use concurrency and parallelism in Python
  • How to write multi-threaded programs
  • How to write multi-process programs
  • How to write asynchronous programs

Course content

3 sections • 28 lectures • 6h 7m total length

  • Threading, Multiprocessing, Async Intro13:52

    Explore concurrent programming in Python, covering threading, multiprocessing, and async, with insights into CPU utilization, IO-bound vs compute-bound tasks, thread safety, race conditions, and practical tradeoffs.

  • Threading in Python18:52
  • Creating a Threading Class14:58
  • Creating a Wikipedia Reader14:05

    Explore concurrent programming by building a wiki worker that fetches S&P 500 symbols from Wikipedia, retrieves Yahoo Finance prices, and saves results to a database using requests and BeautifulSoup.

  • Creating a Yahoo Finance Reader16:23
  • Queues and Master Scheduler15:06

    Systemize symbol processing with a multiprocessing queue and a master scheduler that feeds Yahoo Finance price workers, enabling scalable, thread-safe producers and consumers for price extraction.

  • Creating a Postgres Worker21:54

    build a postgres insertion worker that inserts price data into a postgres prices table (id serial, symbol, price, extracted time) driven by a postgres master scheduler with an input queue.

  • Integrating the Postgres Worker20:11
  • Yaml File Intro18:30
  • Creating a Yaml Reader30:54
  • Improving Our Wiki Worker28:21
  • Improving All Workers and Adding Monitoring29:01

    Refactor the yaml pipeline executor to run as a main worker thread, monitor progress, and only send done signals after all workers finish across downstream queues.

  • Final Program Cleanup7:13

    Clean up the program by configuring an environment via a local .env file, exporting the pipeline location and database variables, and validating with a test run for network bound threading.

  • Locking12:06

Requirements

  • Basic familiarity with Python

Description

In this course you'll learn how to create multi-threaded, asynchronous, and multi-process programs in Python, so that you can make your programs run even faster.


In applications communicating with other resources, a lot of time is spent just waiting for information to be passed from one place to another. You'll learn how to use multi-threading as well as asynchronous programming to speed up programs that are heavily bottlenecked by IO operations.


We'll go through an introduction first of where potential speed bottlenecks come from as well as how we could solve these issues, and then we'll dive directly into the technical content and build out a multi-threaded program together that grabs data from the internet, parses, and saves it into a local database.


Other programs may be more heavily affected by CPU limitations. We'll also learn how to implement multiprocessing in Python, the library that lets us use multiple CPUs in our Python code. With this we'll be able to spread our workload over all the cores available on the machine we're using.


Finally, we'll also look to combine both elements, taking a look at how we can use multiprocessing together with asynchronous programming to get the most benefit for yourself, maximizing your use of CPU resources and minimizing time spent siting idle waiting for IO response.


You can find the lecture code in the GitHub repository linked in the first lesson.

Who this course is for:

  • Python developers that want to make their programs faster by adding concurrency

Report abuse