Python Concurrency Simplified

This video will give you an overview about the course.                       

In this video, we will take a brief look at what a thread is, as  well as, how we can use multiple threads in order to speed up the  execution of some of our programs.                         

• Look at types of threads

• Introduction to multithreading

• Advantages and disadvantage of multithreading

Processes are very similar in nature to threads, they allow us to  do pretty much everything a thread can do but the one key advantage is  that they are not bound to a singular CPU core. Event-driven programming  is a huge part of our lives. In this video, we will study both the  concepts with examples and diagram.                         

• Understand the properties, advantages and disadvantages of processes

• Learn event driven programming with an image example

One excellent example of the benefits of multithreading is,  without a doubt, the use of multiple threads to download multiple images  or files. In this video, we will learn to download concurrent image  using two methods.                         

• Execute a program for sequential download

• Working example of concurrent download

In this video, we will look at the different mechanisms to improve the number crunching performance with Multiprocessing.                         

• Execute the first method that is using sequential prime factorization

• Execute the second method that is using concurrent prime factorization

In this video, we will learn about concurrency and the major bottlenecks that impact our applications.                         

• Understand the introduction to concurrency

• Look at the properties of concurrent systems

• Create a working example that requests a page and calculates the time taken

Parallelism is the art of executing two or more actions  simultaneously as opposed to concurrency in which you make progress on  two or more things at the same time. In this video, we will be  understanding parallelism and how does it differ from concurrency.                         

• Learn the concept of parallelism with an example

• Understand the concept of CPU-bound bottlenecks

In this video, we will look at two different architectural styles  employed by system designers, those are Uniform Memory Access pattern  (UMA) or a Non-uniform memory access (NUMA) pattern.                         

• Understand the basic introduction of computer memory architecture and its type

• Learn the features, advantages, and disadvantages of uniform memory access

• Learn the features, advantages, and disadvantages of non- uniform memory access

In this video, we will take a look at Thread state explained with a  flow chart and a working example. Then, we will discover the different  types of threads.                         

• Look at all different type of thread states

• Represent a diagram to explain five different states

• Explore about different types of thread

In this video, we will take a look at best practices when it comes to starting a new threads.                         

• Define a program to start a thread and inherit from the thread class

• Learn the technique of forking with an example

• Daemonizing a thread

In this video, we will learn effective ways to create and manage multiple threads in Python programs.                         

• Run a program to start loads of threads

• Slow down programs using threads

• Get the total number of active threads

In this video, we will take a look at the performance impact of  spinning up multiple threads and compare this to the spinning up of  multiple processes.                         

• Create processes versus threads

In this video, we will take a look at concept of deadlock with the  example of dinning philosophers. Then we will learn about race  conditions also explaining process execution sequence.                         

• Study Dining Philosophers problem with the help of an example

• Implement our own version of the Dining Philosophers problem in Python using RLocks

• Learn process execution sequence

In this video, we will give you brief overview of all of the  synchronization primitives available to you as well as few simple  examples of how you can use these within your programs.                         

• Utilize the join method

• Acquire a lock while it's in an "unlocked" state

• Define two functions that both modify a and b

In this video, first we will understand the detailed concept of Condition and semaphore explained along with working example.                         

• Look at the definition of our condition object

• Define a semaphore in class definition

• Use a bounded semaphore as opposed to a normal semaphore

In this video, we will take a look at the concepts of  synchronization primitive such as Events and Barriers with respective  examples.                         

• Modify and utilize Event public functions

• Utilize barriers in order to block the execution of our threads

In this video, we will take a look at few examples of locked set function, decorator and class decorators.                         

• Extend the class

• Extend other primitives

• Implement a class decorator function

In this video, we will take a look at the basic concept of Queue and its objects along with the respective examples.                         

• Utilize the queue.Queue() object in order to implement our own FIFO-based queue

• Create a series of subscribers that retrieve all items from this queue until it is empty

• Create PriorityQueue which we will populate with two sets of data.

In this video, we will look at full or empty queue. Then we will see a working example of deque objects.                         

• Create a series of publishers that attempt to publish to our queue object until it is full

• Create a number of subscribers that subscribe to our queue object

• Define our deque object by calling collections.deque()

In this video, we will learn to interact with the objects in the  queue by appending, removing and inserting elements with the help of  working examples.                         

• Run a code example that append to both the start and the end of our deque object

• Pop items from both the start and the end of our queues using pop() and popleft()

• Insert elements into anarray at specific points

In this video, we are going to build a very simple multithreaded web Crawler.                         

• List the requirements

• Declare our Crawler’s constructor function which will set base_url

• Utilize the atomicity of the queue object

In this video, we will learn to Test concurrent software systems. Then we will look at few examples of unit testing and PY unit.                         

• Define SimpleFunctionTest class which inherits from unittest.TestCase

• Follow a blend of different strategies while unit testing concurrent code

In this video, we will learn to catch an exceptions in child threads.                         

• Utilize the sys module to extract the information

In this video, we will look at the benchmarking techniques along  with working example of ‘time it’ module, utilizing decorators and  timing context manager.                         

• Utilize the timeit module to measure the time taken to execute two distinct functions

• Retrieve the differences between these two calls and display this on the console

• Define a Timer class which features a constructor, an entry point, and an exit point

In this video, we are going to learn profiling techniques which  includes cProfile and Memory profiling with the help of few tools such  as line profiler tool and Kernprof.                         

• Utilize the cProfile module in order to attain these attributes

• Add @profile decorator in order to see what has gone wrong

In this video, we will learn to create Thread pool executor with context manager and shutting down executor object.                         

• Utilize the threading.current_thread() function in order to determine

• Utilize our ThreadPoolExecutor as a context manager

• Define a function which will essentially "work" for n number of seconds

In this video, we will learn about future objects and unit testing  with future objects along with associated methods such as set_result  method and set_exception method. Then we will look at few other methods  like cancelling callable, getting a result and as completed.                         

• Use cancel() function on that specific task

• Call executor.map to map every value in the array

• Utilize the urllib.request module in order to request these URLs

In this video, we will create two programs, the first one will be  to utilize the callback functionality and in another program is to pass  the exception from child thread to another thread.                         

• Specify a callback for the said function using the add_done_callback function

• Add multiple callbacks to a single future object

• Define an isEven function, which takes in a value

In this video, we will look at few working examples on Process  Pool Executor, context manager and for improving speed computationally  bound problems.                         

• Use os module to find the current PID

• Employ a context manager to make our code nicer and more readable

• Utilize the timeit module

In this video, we will get to know few examples to improve our  crawlers. Then, we are going to look at refactoring our code and store  the results in a CSV file.                         

• Manage the startup and shutdown of all our threads

• define an appendToCSV function

In this video, first we will learn to Utilizing sub-processes.  Then we will understand Daemon Processes with the help of working  example.                         

• Spin up child processes using the multiprocessing module

• Define and start your own daemon processes

In this video, we will learn to identify the process using PID and terminating a process using terminate function                         

• Spin up a single child process, and pass in childTask function

• Simulate Executing something for 20 seconds

• Retrieve current process identifier for the process

In this video, we will get an overview of the multiprocessing  Pools. Then We will learn to interact with these pools in a number of  distinct ways.                         

• Utilized ThreadPoolExecutors

• Use a for loop to submit four tasks to our processing pool

• Pass the maxtasksperchild parameter

In this video, we will look at number of option helps to communicate with different processes.                         

• Learn different options to communicate between threads

• Look at an example of Pipes and its type

• Create a child process, which, , subclasses the multiprocessing.Process class

In this video, we will look at few multiprocessing managers which  includes Namespaces and queue. Then we will explore an example code in  greater detail to gain a basic understanding of communication using both  a listener and a client.                         

• Utilize namespaces in order to share some data across both a main process and a child process

• Define three separate processes that take in the myTask function

• Use Listener as a context manager, and wait to accept a connection

In this video, we will learn interaction between multiple concurrent models.                         

• Use pip to install pycsp

• Define two distinct functions, Process1 and Process2

Before we dive deep into the working, let’s get introduced to the  event-driven programming concepts and how it comes at rescue for complex  tasks in our program.                         

• Understand what event-driven programming is in a general sense

Asyncio was introduced to the Python programming language in  version 3.4 and added some excellent functionality and it has been an  overall hit with the Python community. Let’s get started with it.                         

• Learn about events, tasks, and futures

• Explore about coroutines and protocols

• Perform synchronization between coroutines

Thankfully, when it comes to debugging asyncio-based applications,  we have a couple of options to consider. It’s time to explore those  options in detail and how to work along with those.                         

• Create a simple event loop

• Submit some simple tasks to the event loop

Will you like to use a powerful event-driven networking engine  that can be used for a huge range of different projects such as web  servers, mail clients, subsystems, and more? Yes! Let’s do it right now!                         

• Set up a simple web server that serves local content from a tmp directory

Let’s get familiar with another networking library which is based  purely on top of coroutines. This video will walk you through the use of  the gevent library.                         

• Explore about Greenlets

• Use gevent within your Python applications