Master Parallel & Concurrent Programming Using Python:2 in 1

In this video, we will take a look at Flynn's taxonomy.

• Learn about SISD, MISD, SIMD, and MIMD

Another aspect that we need to consider to evaluate a parallel architecture is memory organization. In this video you will understand this concept.

• Understand shared memory systems

This video is the continuation of the previous video where we will take a closer look at distributed memory systems.

• Understand distributed memory systems

In this video, you will get an overview of parallel programming models.

• Understand the shared memory model and the multithread model

• Understand the distributed memory or message passing model

• Understand the data parallel model

The design of algorithms that exploit parallelism is based on a series of operations. This video shows us how to design such parallel programs.

• Learn task decomposition and assignment

• Understand agglomeration and mapping

The development of parallel programming created the need of performance metrics. This video will help us evaluate the performance of a parallel program.

• Analyze the performance with the help of performance indexes

Python is a powerful, dynamic, and interpreted programming language that is used in a wide variety of applications. In this video, we will get introduced to Python and its features.

• Download Python

• Explore Python features

• Understand Python's functionalities in parallel world

In this video, we simply demonstrate how to start a single new program from inside a Python program.

• Start a process

• Manage a parallel execution of more processes

This video simply shows you how to create a single thread inside a Python program.

• Create a thread with the help of a program

The simplest way to use a thread is to instantiate it with a target function. This video shows us how to do that.

• Define a thread with the threading module

• Instantiate a thread

• Pass an argument to the function

This video helps us in determining the thread which we created earlier.

• Instantiate a thread with a target function

This video will help us to implement a new thread using the threading module.

• Define a new subclass of the Thread class

• Override methods

• Create a new subclass

In this video, we describe the Python threading synchronization mechanism called lock().

• Manage a thread through the mechanism of lock()

If we want only the thread that acquires a lock to release it, we must use an RLock() object. This video will get you introduced to RLock.

• Introduce a class with the add and remove methods

• Carry out the RLock() call within the class

A semaphore is an abstract data type managed by the operating system. In this video we will carry out the thread synchronization with semaphores.

• Use the consumer-producer model

• Initialize a semaphore to 0

A condition identifies a change of state in the application. In this video, we will carry out the thread synchronization with a condition.

• Illustrate the condition mechanism

Events are objects that are used for communication between threads. In this video, we will carry out thread synchronization with an event.

• Understand the thread synchronization through the event object

The "with" statement is useful when you have two related operations that must be executed as a pair with a block of code in-between. This video will show us how to use the "with" statement.

• Define a list of thread communication directives

• Test the list

Queues are much easier to deal with and make threaded programming considerably safe. In this video, we will take a look at thread communication using queue.

• Use the threading module with the queue module

In this video, we will verify the impact of the GIL, evaluating the performance of a multithread application.

• Perform four tests using different functions

Spawn means the creation of a process by a parent process. This video will show us how to spawn a process

• Create a series (five) of processes

• Run the process

The procedure to name a process is similar to that described for the threading library. Let's check this out in this video

• We create a process with and without a name

• Run the process

Running a process in the background is a typical mode of execution of laborious processes. This video will show you how to do that.

• Set the daemon parameter

• Execute the process in the background

It's possible to kill a process immediately using the terminate() method. Let's see how to do that in this video.

• Create a process with a target function

• Kill the process with the terminate function

In this video, we will see how to implement a custom subclass and process.

• Define a new subclass of the Process class

• Override a method to add additional arguments

• Override a method to implement what Process should when it is started

The development of parallel applications has the need for the exchange of data between processes. We will see how to do that in this video.

• Use queue to exchange objects

• Use pipes to exchange objects

Synchronization primitives are quite similar to those encountered for the library and threading. In this video, we will see how to synchronize the process.

• Use barrier method to synchronize two processes

Python multiprocessing provides a manager to coordinate shared information between all its users. In this video, we will see how to manage a state between processes.

• Declare the manager

• Create a data structure

• Launch the multiprocess

The multiprocessing library provides the Pool class for simple parallel processing tasks. In this video, we will see how to use it.

• Implement a process pool to perform a parallel application

• Create a pool of four processes

• Use the pool's map method toperform a simple calculation

The Python programming language provides a number of MPI modules to write parallel programs. In this video, we will see how to use the mpi4py library

• Examine the classic Hello world program using mpi4py

One of the most important features among those provided by MPI is the point-to-point communication. We will check that out in this video.

• Utilize the directives to exchange messages between different processes

A common problem we face is that of the deadlock in processes. This video will help us to avoid such problems.

• Introduce a typical deadlock problem

• Swap the sending and receiving functions

In a collective communication broadcast process, a single process sends the same data to any other process.

• Use the broadcast function

The scatter functionality sends the chunks of data in an array to different processes. This video will show us how to use scatter for collective communication.

• Distribute data to different processes using scatter

With the gather function, all processes send data to a root process that collects the data received. Let's look at using gather for collective communication.

• Build the process data and send it to root processes

• Collect the data in an array

The Alltoall collective communication combines the scatter and gathers functionalities. In this video, we will see how to use Alltoall for collective communication.

• Implement the Alltoall function

• Run the code with a communicator group of file processes

Reduction takes an array of input elements in each process and returns an array of output elements to the root process. We will take a look at this operation in this video.

• Implement a sum of an array of elements using the reduction operation

MPI allows us to assign a virtual topology to a communicator. In this video, we will see how to optimize the communication using such mechanism

• Implement a Cartesian topology

• Define a set of coordinates

With the release of Python 3.2, the concurrent.future module was introduced. In this video, we will see how to use this module.

• Build a list of numbers for each element in the list

• Execute the task in a sequential mode

• Use the concurrent.futures module for a thread pool

In this video, the focus is on handling events with the help of Asyncio.

• Define asynchronous tasks

• Run the code

In this video, we will see how to use the co-routine mechanism of Asyncio to simulate a finite state machine of five states.

• Define each state of the automata

• Determine to which state the finite state machine will be passed

The Asyncio module provides us with the asyncio. Task(coroutine) method to handle computations with tasks. In this video, we will see how to manipulate a task with Asyncio.

• Execute three mathematical functions

Another key component of the Asyncio module is the Future class. This video will teach you how to deal with Asyncio and futures.

• Use the Futures class for the management of two coroutines

Celery is a Python framework used to manage a distributed tasks, following the object-oriented middleware approach. In this video, we will see how to use Celery to distribute tasks.

• Install celery, message broker, RabbitMQ, and Flower

• Use Celery to distribute tasks

In this video, we'll learn to create and call a task using the Celery module.

• Create a Celery application instance

• Introduce the task

SCOOP is a Python module to distribute concurrent tasks (called Futures) on heterogeneous computational nodes. In this video we will take a look at scientific computing with Scoop.

• Install Scoop

• Use the Monte Carlo method

The SCOOP Python modules define more than one map function; and they allow asynchronous computation that could be propagated to its workers. In this video, we will see how to handle Map functions with SCOOP.

• Compare the MapReduce version of SCOOP with its serial implementation

Python Remote Objects (Pyro4) is a library that resembles Java's Remote Method Invocation (RMI), which allows you to invoke a method of a remote object. In this video we will see how to do that.

• Build and use a simple client-server communication using the Pyro4

Implement a chain of objects with Pyro4 using Python scripts.

• Run the Pyro4 name server

In this video, we will see how to build a simple client-server application with Pyro4.

• Create three Python scripts

• Enable the Pyro4 name sever and execute the code

• Start the customer simulation

PyCSP is a Python module based on communicating sequential processes, which is a programming paradigm developed to build concurrent programs via message passing. We will take a look at that in this video.

• Define processes

• Define the communication between these processes

Remote Python Call (RPyC) is a Python module that is used for remote procedure calls as well as for distributed computing. In this video, we will see how to carry out a remote procedure call with RPyC.

• Run a client that connects to the server

• Execute the print statement on the server

PyCUDA is a Python wrap for Compute Unified Device Architecture (CUDA), the softwarelibrary developed by NVIDIA for GPU programming. In this video, we will see how to use PyCUDA.

• Understand how a hybrid programming model works

• Understand the kernel and thread hierarchy

• Build a PyCUDA library for Python

The PyCUDA programming model is designed for the common execution of a program on a CPU and GPU. This video will show us how to build a PyCUDA application.

• Create an array on CPU and transfer it to GPU

• Building a PyCUDA application

• Transfer the array from the GPU to the CPU

• Print the results

In the CUDA-capable GPU card, there are four types of memories. In this video, we will take a look at those with the help of matrix manipulation.

• Prepare the input matrix and the output matrix

• Transfer matrixes in the GPU device with the PyCUDA function

• Verify whether the computation is correct

In this video, we will see the common use case of GPU computations in order to invoke a kernel function.

• Import all the modules required

• Perform the computation in the GPU

The PyCuda.elementwise.ElementwiseKernel function allows us to execute the kernel on complex expressions. We will see how to do that in this video.

• Evaluate the linear combination of the vectors to use ElementwiseKernel call

PyCUDA provides a functionality to perform reduction operations on the GPU. We will take a look at that in this video.

• Define the inputs

• Define the MapReduce operation and invoke the kernel function

• Print the input vectors and the resulting dot product

NumbaPro is a Python compiler that provides a CUDA-based API to write CUDA programs. In this video we will demonstrate GPU programming with NumbaPro.

• Give demonstration of the NumbaPro compiler using the annotation @guvectorize

NumbaPro provides a Python wrap for CUDA libraries for numerical computing. We will understand it with the help of this video.

• Import libraries and Define the matrix dimension along with input matrices

• Compare the calculation done with NumPy and cuBLAS

In this video, we'll examine the Python implementation of OpenCL called PyOpenCL.

• Enumerate all major hardware features using the OpenCL library

As for programming with PyCUDA, the first step to build a program for PyOpenCL is the encoding of the host application. This video will show us how to build the application.

• Execute the parallel sum of two vectors

PyOpenCL provides the functionality in the pyopencl.elementwise class that allows us to evaluate the complicated expressions in a single computational pass. We will see how to do that in this video.

• Add two integer vectors of 100 elements using ElementwiseKernel class

In this video, we will test the GPU application by implementing the regular definition schema of an application for PyOpenCL.

• Evaluate the kernel call and GPU time

• Call the testing function and print_device_info method