This course will teach you parallel programming techniques using examples in Python and help you explore the many ways in which you can write code that allows more than one process to happen at once.
Starting with introducing you to the world of parallel computing, we move on to cover the fundamentals in Python. This is followed by exploring the thread-based parallelism model using the Python threading module by synchronizing threads and using locks, mutex, semaphores queues, GIL, and the thread pool. Next you will be taught about process-based parallelism, where you will synchronize processes using message passing and will learn about the performance of MPI Python Modules.
Moving on, you’ll get to grips with the asynchronous parallel programming model using the Python asyncio module, and will see how to handle exceptions. You will discover distributed computing with Python, and learn how to install a broker, use Celery Python Module, and create a worker. You will understand anche Pycsp, the Scoop framework, and disk modules in Python. Further on, you will get hands-on in GPU programming with Python using the PyCUDA module and will evaluate performance limitations.
About the Author
Giancarlo Zaccone, a physicist, has been involved in scientific computing projects among firms and research institutions. He currently works in an IT company that designs software systems with high technological content.
In this video, we will take a look at Flynn's taxonomy.
Another aspect that we need to consider to evaluate a parallel architecture is memory organization. In this video you will understand this concept.
This video is the continuation of the previous video where we will take a closer look at distributed memory systems.
In this video, you will get an overview of parallel programming models.
The design of algorithms that exploit parallelism is based on a series of operations. This video shows us how to design such parallel programs.
The development of parallel programming created the need of performance metrics. This video will help us evaluate the performance of a parallel program.
Python is a powerful, dynamic, and interpreted programming language that is used in a wide variety of applications. In this video, we will get introduced to Python and its features.
In this video, we simply demonstrate how to start a single new program from inside a Python program.
This video simply shows you how to create a single thread inside a Python program.
The simplest way to use a thread is to instantiate it with a target function. This video shows us how to do that.
This video helps us in determining the thread which we created earlier.
This video will help us to implement a new thread using the threading module.
In this video, we describe the Python threading synchronization mechanism called lock().
If we want only the thread that acquires a lock to release it, we must use an RLock() object. This video will get you introduced to RLock.
A semaphore is an abstract data type managed by the operating system. In this video we will carry out the thread synchronization with semaphores.
A condition identifies a change of state in the application. In this video, we will carry out the thread synchronization with a condition.
Events are objects that are used for communication between threads. In this video, we will carry out thread synchronization with an event.
The "with" statement is useful when you have two related operations that must be executed as a pair with a block of code in-between. This video will show us how to use the "with" statement.
Queues are much easier to deal with and make threaded programming considerably safe. In this video, we will take a look at thread communication using queue.
In this video, we will verify the impact of the GIL, evaluating the performance of a multithread application.
Spawn means the creation of a process by a parent process. This video will show us how to spawn a process.
The procedure to name a process is similar to that described for the threading library. Let's check this out in this video.
Running a process in the background is a typical mode of execution of laborious processes. This video will show you how to do that.
It's possible to kill a process immediately using the terminate() method. Let's see how to do that in this video.
In this video, we will see how to implement a custom subclass and process.
The development of parallel applications has the need for the exchange of data between processes. We will see how to do that in this video.
Synchronization primitives are quite similar to those encountered for the library and threading. In this video, we will see how to synchronize the process.
Python multiprocessing provides a manager to coordinate shared information between all its users. In this video, we will see how to manage a state between processes.
The multiprocessing library provides the Pool class for simple parallel processing tasks. In this video, we will see how to use it.
The Python programming language provides a number of MPI modules to write parallel programs. In this video, we will see how to use the mpi4py library.
One of the most important features among those provided by MPI is the point-to-point communication. We will check that out in this video.
A common problem we face is that of the deadlock in processes. This video will help us to avoid such problems.
In a collective communication broadcast process, a single process sends the same data to any other process.
The scatter functionality sends the chunks of data in an array to different processes. This video will show us how to use scatter for collective communication.
With the gather function, all processes send data to a root process that collects the data received. Let's look at using gather for collective communication.
The Alltoall collective communication combines the scatter and gathers functionalities. In this video, we will see how to use Alltoall for collective communication.
Reduction takes an array of input elements in each process and returns an array of output elements to the root process. We will take a look at this operation in this video.
MPI allows us to assign a virtual topology to a communicator. In this video, we will see how to optimize the communication using such mechanism.
With the release of Python 3.2, the concurrent.future module was introduced. In this video, we will see how to use this module.
In this video, the focus is on handling events with the help of Asyncio.
In this video, we will see how to use the co-routine mechanism of Asyncio to simulate a finite state machine of five states.
The Asyncio module provides us with the asyncio. Task(coroutine) method to handle computations with tasks. In this video, we will see how to manipulate a task with Asyncio.
Another key component of the Asyncio module is the Future class. This video will teach you how to deal with Asyncio and futures.
Celery is a Python framework used to manage a distributed tasks, following the object-oriented middleware approach. In this video, we will see how to use Celery to distribute tasks.
In this video, we'll learn to create and call a task using the Celery module.
SCOOP is a Python module to distribute concurrent tasks (called Futures) on heterogeneous computational nodes. In this video we will take a look at scientific computing with Scoop.
The SCOOP Python modules define more than one map function; and they allow asynchronous computation that could be propagated to its workers. In this video, we will see how to handle Map functions with SCOOP.
Python Remote Objects (Pyro4) is a library that resembles Java's Remote Method Invocation (RMI), which allows you to invoke a method of a remote object. In this video we will see how to do that.
Implement a chain of objects with Pyro4 using Python scripts.
In this video, we will see how to build a simple client-server application with Pyro4.
PyCSP is a Python module based on communicating sequential processes, which is a programming paradigm developed to build concurrent programs via message passing. We will take a look at that in this video.
Remote Python Call (RPyC) is a Python module that is used for remote procedure calls as well as for distributed computing. In this video, we will see how to carry out a remote procedure call with RPyC.
PyCUDA is a Python wrap for Compute Unified Device Architecture (CUDA), the softwarelibrary developed by NVIDIA for GPU programming. In this video, we will see how to use PyCUDA.
The PyCUDA programming model is designed for the common execution of a program on a CPU and GPU. This video will show us how to build a PyCUDA application.
In the CUDA-capable GPU card, there are four types of memories. In this video, we will take a look at those with the help of matrix manipulation.
In this video, we will see the common use case of GPU computations in order to invoke a kernel function.
The PyCuda.elementwise.ElementwiseKernel function allows us to execute the kernel on complex expressions. We will see how to do that in this video.
PyCUDA provides a functionality to perform reduction operations on the GPU. We will take a look at that in this video.
NumbaPro is a Python compiler that provides a CUDA-based API to write CUDA programs. In this video we will demonstrate GPU programming with NumbaPro.
NumbaPro provides a Python wrap for CUDA libraries for numerical computing. We will understand it with the help of this video.
In this video, we'll examine the Python implementation of OpenCL called PyOpenCL.
As for programming with PyCUDA, the first step to build a program for PyOpenCL is the encoding of the host application. This video will show us how to build the application.
PyOpenCL provides the functionality in the pyopencl.elementwise class that allows us to evaluate the complicated expressions in a single computational pass. We will see how to do that in this video.
In this video, we will test the GPU application by implementing the regular definition schema of an application for PyOpenCL.
Packt has been committed to developer learning since 2004. A lot has changed in software since then - but Packt has remained responsive to these changes, continuing to look forward at the trends and tools defining the way we work and live. And how to put them to work.
With an extensive library of content - more than 4000 books and video courses -Packt's mission is to help developers stay relevant in a rapidly changing world. From new web frameworks and programming languages, to cutting edge data analytics, and DevOps, Packt takes software professionals in every field to what's important to them now.
From skills that will help you to develop and future proof your career to immediate solutions to every day tech challenges, Packt is a go-to resource to make you a better, smarter developer.
Packt Udemy courses continue this tradition, bringing you comprehensive yet concise video courses straight from the experts.