High-Performance Computing with Python 3.x

Thread-based parallelism is one of the basic ways of achieving parallelism in our code. We need to explore the in-built module available in Python and how we can use it to implement threading in our programs.

   •  Explore the concept of threads

   •  Understand the threading module

   •  Implement a basic program involving multiple threads

We discuss one of the most common problem with multi-threaded systems, i.e. Locks on shared data.

   •  Understand what are Locks

   •  Implement Locks in Python

   •  Implement Re-entrant Locks

We explore the concept of Global Interpreter Lock and how it ties in to threading in Python.

   •  Explore the concept of Global Interpreter Lock

   •  Understand how it affects multi-threaded system

   •  Visually explore how GIL works in a multi-threaded system through an example

Once we have explored multithreading, we move on to exploring Multiprocessing and how we can implement it in Python.

   •  Understand the concept of Multiprocessing

   •  Explore the Multiprocessing module in Python

   •  Implement a basic example involving multiple processes

We explore the Pool class available in the Multiprocessing module in Python and how we can use to write more efficient multiprocessing programs.

   •  Explore the Pool class

   •  Explore the various functions available using Pool class

   •  Implement an example using Pool Class

NumPy is the standard module for performing numerical operations on arrays in Python. We need to understand the basics of NumPy arrays and how they are faster than traditional lists.

   •  Understand the idea behind NumPy module

   •  Explore the concept of NumPy arrays

   •  Explore how NumPy arrays are faster than traditional lists

Once we have understood the basic idea behind NumPy, we dig deeper into NumPy’s N-dimensional arrays.

   •  Explore the different ways of creating NumPy arrays

   •  Implement multi-dimensional arrays

   •  Explore various functions and attributes of NumPy arrays

Before performing complex operations using NumPy arrays, we take a look at what are the different ways of doing indexing on NumPy arrays.

   •  Explore the different ways of indexing on NumPy arrays

   •  Use slicing and integer indexing

   •  Use Boolean masks for indexing NumPy arrays

Once we are done with the basics, we move on to performing complex mathematical operations on NumPy arrays. Also, we take a look at the concept of broadcasting and how it can be used to perform operations.

   •  Implement multiple mathematical operations on NumPy arrays

   •  Understand the concept of broadcasting

   •  Implement broadcasting of NumPy arrays

After exploring the various operations using NumPy, we do a time analysis of the speedup that can be achieved using NumPy.

   •  Implement a simple mathematical operation with large arrays in Python

   •  Implement the same program using NumPy

   •  Do a time analysis of both implementations

SciPy is a powerful scientific library built on top of NumPy. We explore the various functions available in SciPy and how it can be combined with NumPy arrays.

   •  Understand the idea behind SciPy

   •  Explore the SciPy module

   •  Combine SciPy functions with NumPy arrays

Before diving into the nuts and bolts of using C with Python, we must first understand the idea behind Cython and why we should use it.

   •  Understand the idea behind using Cython Module

   •  Understand what is the workflow of a typical Cython program

   •  Explore why Cython gives a considerable speedup to your existing Python Code

Once we have understood the basic idea behind Cython, we move towards implementing a few basic programs using Cython.

   •  Setup Cython on your local machine

   •  Implement a simple hello world program using Cython

   •  Implement a factorial program using Cython

Once we have implemented our first Cython program, we dig deeper by doing time analysis of our code.

   •  Use Cython in your Jupyter notebook

   •  Perform time analysis of Cython code with traditional Python code

   •  Implement the statically typed nature of Cython for more speedup by using the cdef keyword

We explore the various data types available at our disposal while using Cython.

   •  Understand the various data types available using Cython

   •  Hands-on on using the data types in Cython

   •  Understand how Cython data types work in a Jupyter notebook

After exploring the various data types available to us in Cython, we move towards understanding how Cython functions work, and how we can create them.

   •  Explore the various types of functions in Cython

   •  Understand the idea behind cdef and cpdef functions

   •  Understand the difference between the various types of functions

Once we have mastered the basics of Cython, we move towards a more complicated topic, i.e. combining NumPy code with Cython for more speedup.

   •  Implement a computationally heavy program in NumPy and combine it with Cython

   •  Understand the concept behind Memory Views in

   •  Explore more ways of reducing time for array indexing and accessing using Cython

Before diving into the nuts and bolts of using Numba, we need to understand the basics of Numba, how it works and what its limitations are.

   •  Understand the idea behind using Numba Module

   •  Explore the different ways in which Numba optimizes the Python code

   •  Explore the limitations of using Numba

Once we have understood the basic idea behind Numba, we take a look at how we can setup Numba both locally and using Google Colab Notbooks (in case you don’t have a GPU).

   •  Setup Numba on your local machine

   •  Setup Numba on Google Colab notebook

   •  Use the notebook on Google Colab, in case you don’t have a GPU

Once Numba has been setup correctly, we move towards implementing a basic program using Python and them optimizing it using Numba.

   •  Implement a factorial program using Python

   •  Explore the @jit decorator and use it to optimize code

   •  Do a time comparison of our optimized code

We explore the different compilations modes available in Numba and dig deeper into implementing universal functions using Numba.

   •  Understand the various compilations  modes available using Numba

   •  Implement NumPy’s universal functions in Numba

   •  Understand different modes of operations for @vectorize

We explore the idea of using threads with optimized Python code using Numba.

   •  Explore the ways in which threading can be activated in Numba

   •  Understand the different threading layers in Numba

   •  Implement a program using Threading with Numba

Finally, we do a time comparison of the different techniques we have learned to optimize Python code using Numba.

   •  Implement the basic version of do_some_op function and do time analysis

   •  Implement the NumPy version of do_some_op function and do time analysis

   •  Optimize the code using @jit and using eager evaluation

Synchronous and Asynchronous programming are two basic models of programming. Before moving on to advanced topics, we must understand the difference between the two and how they work.

   •  Understand the concept of Synchronous programming.

   •  Explore Synchronous programming with an example

   •  Understand the various drawbacks of Synchronous programming

Once we have understood the concept of Synchronous programming, we explore the concept of Asynchronous programming and how it is different from the former.

   •  Understand the concept of Asynchronous programming

   •  Explore the difference between Synchronous and Asynchronous programming

   •  Understand the various advantages of Asynchronous programming

Once we have explored the concept of Asynchronous programming, we must understand how we can apply it in Python using AsyncIO module.

   •  Explore the AsyncIO module

   •  Implement a simple Asynchronous example in Python

   •  Do a comparative analysis between a Synchronous and Asynchronous function

We explore the idea behind Distributed System architecture, it advantages and use cases.

   •  Understand what is a distributed system

   •  Explore how distributed systems work

   •  Discover the various advantages of distributed system

Dask is one of the most commonly used Python library for scalable and parallel Python code. To get the best performance for our code, we must learn how Dask works and how we can leverage it to parallelize our code.

   •  Understand the idea behind Dask module

   •  Understand why Dask is able to speed up our code

   •  Explore the various Use cases for Dask

Once we have a basic idea of Dask, we move towards setting up Dask on our systems.

   •  Setup Dask locally on our system

   •  Install all the different components of Dask in one go

   •  Setup Dask on Google Colab notebook

Before moving on to implementing our first program using Dask, we must understand the basic idea about how Dask arrays work and how we can use Blocked Algorithms

   •  Understand the concept Blocked Algorithms

   •  Explore Dask arrays and how they work

   •  Explore how Blocked algorithms work with Dask arrays

Once we have finished learning about the basics of Dask, we implement our first program using Dask arrays

   •  Implement a basic program using Dask arrays

   •  Understand how lazy evaluation works with Dask

   •  Explore the various functions of Dask arrays

Sometimes it is not possible to simply use Dask arrays to achieve parallelism. In such cases, we can use @delayed decorator to parallelize our custom code.

   •  Explore how @delayed works

   •  Implement a program with @delayed to speedup code

   •  Visualize task graph for Parallelized code

Finally we do a time and performance analysis for Dask code using the concepts we have learned in this section.

   •  Implement an operation for arrays and implement the NumPy code for the same

   •  Change the code using Dask arrays

   •  Do a time and performance analysis for all the techniques to record the speedup

Previously we learnt about Asynchronous programming and how we can implement it in Python. Reactive programming is also based on Asynchronous programming of data streams and, we need to have an overview about it before moving deeper into it.

   •  Understand the concept of Reactive programming

   •  Explore the different properties of Reactive programming

   •  Discover the advantages of Reactive programming

Once we have a basic idea of Reactive programming, we must learn about the basic building blocks of Reactive programming, i.e. Observables and Observers (subscribers).

   •  Understand the concept of Observables

   •  Understand the concept of Observers

   •  Explore how Observables and Observers (subscribers) work together

Data Operators allow us to perform operations on data streams in Reactive Programming. Before moving over to the implementation part, we must explore the various types of data operators available at out disposal.

   •  Understand the concept of data operators in Reactive Programming

   •  Explore different types of data operators and how they work

   •  Explore the working of data operators through an example

Once we have finished learning about the basics of Reactive Programming, we move towards implementing the same using RxPy module.

   •  Understand what exactly the RxPy module is

   •  Setup RxPy module

   •  Implement Observables and Observers in Python

After implementing a basic program using RxPy, we learn how to implement various data operators in Python and also learn how to chain different data operators together.

   •  Implement different types of data operators in Python

   •  Implement chaining of data operators

   •  Explore the various data operators available in RxPy module