Mastering Python 3 Programming

This video will give you an overview about the course.

Install the Python development environment.

• Pick the correct installer

• See operating system-specific considerations

Running Python programs and support utilities.

• Run Python programs

• Install utilities

• Run utilities

How to create a graphical/touch interface?

• Look at the basic Kivy app structure

• The Kv design language

• Use Kv canvas instructions

The user needs to be able to interact with our program.

• Create an event handler

• Receive touch/click events

• Decide whether to respond to the events

How does the observer pattern work?

• Create properties

• Observe properties

• Explore the output based on property changes

We can expand the observer pattern into a full programming paradigm.

• Create observable event source

• Create processing pipe for observed events

• Subscribe to the final output of the pipe

Putting the things, we’ve learned to use.

• Use Kivy to build the user interface

• Use ReactiveX to handle input from the server

• Run the program

How to reduce the overhead of an asynchronous program without sacrificing logical structure?

• See Asynchronous I/O

• Learn about Coroutines

• Go in depth about the cooperative scheduling

How to write a microservice that communicates using the HTTP protocol?

• Handle HTTP requests

• Use HTTP methods

• Representative State transfer

How to combine reactive programming with asynchronous programming?

• Look at the Asyncio-based reactive scheduler

• Feed the pipe

• Handle pipe output with tasks

Using what we’ve learned to write a game server.

• See pickles and the network

• Serve multiple clients

• Cooperate tasks

Using mypy can help us prevent bugs by eliminating type mismatches.

• Use a data type that doesn’t “fit” causes problems

• Get to know that the mypy tool checks whether a type mismatch is possible

• Understand how this helps us find bugs early and fix them correctly

Using pytest can help us identify existing bugs and prevent reversions.

• Tests help us find logic errors in our code

• See how the pytest tool provides a simple and Pythonic way to create and run our tests

• Execute the code and feel good about the test being successful

Using tests to guide our development process.

• Write the tests first to define correct behavior

• Write code to make the tests pass

• Repeat the cycle until the whole program is complete

Using what we’ve learned to improve our Oware server.

• Add and check type annotations

• Add and run tests

• Fix the bugs we find

Provides a high-level interface for multi-process computing.

• See what process structure does concurrent.futures employ?

• Use map()

• Use submit()

Provides a mid-level interface for multi-process computing.

• Determine the number of available CPU cores

• Launch worker processes

• Create communication channels between processes

Provides a low-level interface for running processes and communicating with them.

• Run workers

• Communicate with workers

• Shut down workers

Allow us to be more intelligent about transferring data between processes.

• When can we benefit?

• How to use __getstate__

• How to use __setstate__

What are decorators and what do they do?

• Learn about the basic decorators

• Apply wrappers with decorators

• Explore parameterized decorators

Using annotations for more than typing information

• Look at the annotations as metadata

• See current semantics

• Look at the future semantics

We need to control what happens when attributes are assigned, read, or deleted.

• Learn how to make properties

• See how to make descriptors

• Look at how to assign descriptors to attributes

We need to be able to control what happens at the beginning and end of a scope, ensure that acquired resources are properly released and related tasks.

• See how to create context managers as decorated functions

• Learn how to add context manager functionality to a class

• Look at how to create asynchronous context managers

We want to distribute software to team-mates or people in our organization.

• What is a ZipApp?

• Make a ZipApp

• Run a ZipApp

We want to distribute software to the Python developer community.

• What is a wheel?

• Make a wheel

• Upload a wheel

We want to distribute a program to end users.

• What is PyInstaller?

• Run PyInstaller

• Run the packaged program

We want to create a native code version of our Python software.

• What does Cython do?

• Compile Python to C

• Compile C to a native program

This video provides an overview of the entire course.

Before moving ahead, we need to understand the various data types available in Python and how we can use them efficiently.

• Explore how to create List, Tuple, Set and Dictionary

• Look at the various built-in functions for these datatypes

• Learn how these datatypes are different from each other in terms of their functionality

Lambda expressions allow us to create function on the go, making our code more efficient. In this section, we will explore how we can use lambda expressions in Python.

• Understand what are lambda expressions

• Implement anonymous functions using lambda expressions

• Combine lambda expressions with other functions for improved functionality

It provides a significant speed boost to our code. In this video, we explore how we can implement them in Python to speed up our programs.

• Explore the concept of List and Dictionary comprehensions

• Implement the List and Dictionary comprehensions in Python

• Perform time analysis of the code we have implemented

We explore Iterators and Generators in Python, and how they utilize lazy evaluation to work with large data.

• Understand what are Iterators and the Iterator Protocol

• Implement Iterators in Python

• Implement Generators in Python using the yield keyword

Decorators are a powerful tool available in Python. We explore the various ways through which we can implement decorators in Python in order to enhance our existing code.

• Understand the concept of Decorators

• Use @notation for decorators and passing arguments using Decorators

• Use Decorator for time analysis of our existing functions

Thread-based parallelism is one of the basic ways of achieving parallelism in our code. We need to explore the in-built module available in Python and how we can use it to implement threading in our programs.

• Explore the concept of threads

• Understand the threading module

• Implement a basic program involving multiple threads

We discuss one of the most common problem with multi-threaded systems, i.e. Locks on shared data.

• Understand what are Locks

• Implement Locks in Python

• Implement Re-entrant Locks

We explore the concept of Global Interpreter Lock and how it ties in to threading in Python.

• Explore the concept of Global Interpreter Lock

• Understand how it affects multi-threaded system

• Visually explore how GIL works in a multi-threaded system through an example

Once we have explored multithreading, we move on to exploring Multiprocessing and how we can implement it in Python.

• Understand the concept of Multiprocessing

• Explore the Multiprocessing module in Python

• Implement a basic example involving multiple processes

We explore the Pool class available in the Multiprocessing module in Python and how we can use to write more efficient multiprocessing programs.

• Explore the Pool class

• Explore the various functions available using Pool class

• Implement an example using Pool Class

NumPy is the standard module for performing numerical operations on arrays in Python. We need to understand the basics of NumPy arrays and how they are faster than traditional lists.

• Understand the idea behind NumPy module

• Explore the concept of NumPy arrays

• Explore how NumPy arrays are faster than traditional lists

Once we have understood the basic idea behind NumPy, we dig deeper into NumPy’s N-dimensional arrays.

• Explore the different ways of creating NumPy arrays

• Implement multi-dimensional arrays

• Explore various functions and attributes of NumPy arrays

Before performing complex operations using NumPy arrays, we take a look at what are the different ways of doing indexing on NumPy arrays.

• Explore the different ways of indexing on NumPy arrays

• Use slicing and integer indexing

• Use Boolean masks for indexing NumPy arrays

Once we are done with the basics, we move on to performing complex mathematical operations on NumPy arrays. Also, we take a look at the concept of broadcasting and how it can be used to perform operations.

• Implement multiple mathematical operations on NumPy arrays

• Understand the concept of broadcasting

• Implement broadcasting of NumPy arrays

After exploring the various operations using NumPy, we do a time analysis of the speedup that can be achieved using NumPy.

• Implement a simple mathematical operation with large arrays in Python

• Implement the same program using NumPy

• Do a time analysis of both implementations

SciPy is a powerful scientific library built on top of NumPy. We explore the various functions available in SciPy and how it can be combined with NumPy arrays.

• Understand the idea behind SciPy

• Explore the SciPy module

• Combine SciPy functions with NumPy arrays

Before diving into the nuts and bolts of using C with Python, we must first understand the idea behind Cython and why we should use it.

• Understand the idea behind using Cython Module

• Understand what is the workflow of a typical Cython program

• Explore why Cython gives a considerable speedup to your existing Python Code

Once we have understood the basic idea behind Cython, we move towards implementing a few basic programs using Cython.

• Setup Cython on your local machine

• Implement a simple hello world program using Cython

• Implement a factorial program using Cython

Once we have implemented our first Cython program, we dig deeper by doing time analysis of our code.

• Use Cython in your Jupyter notebook

• Perform time analysis of Cython code with traditional Python code

• Implement the statically typed nature of Cython for more speedup by using the cdef keyword

We explore the various data types available at our disposal while using Cython.

• Understand the various data types available using Cython

• Hands-on on using the data types in Cython

• Understand how Cython data types work in a Jupyter notebook

After exploring the various data types available to us in Cython, we move towards understanding how Cython functions work, and how we can create them.

• Explore the various types of functions in Cython

• Understand the idea behind cdef and cpdef functions

• Understand the difference between the various types of functions

Once we have mastered the basics of Cython, we move towards a more complicated topic, i.e. combining NumPy code with Cython for more speedup.

• Implement a computationally heavy program in NumPy and combine it with Cython

• Understand the concept behind Memory Views in

• Explore more ways of reducing time for array indexing and accessing using Cython

Before diving into the nuts and bolts of using Numba, we need to understand the basics of Numba, how it works and what its limitations are.

• Understand the idea behind using Numba Module

• Explore the different ways in which Numba optimizes the Python code

• Explore the limitations of using Numba

Once we have understood the basic idea behind Numba, we take a look at how we can setup Numba both locally and using Google Colab Notbooks (in case you don’t have a GPU).

• Setup Numba on your local machine

• Setup Numba on Google Colab notebook

• Use the notebook on Google Colab, in case you don’t have a GPU

Once Numba has been setup correctly, we move towards implementing a basic program using Python and them optimizing it using Numba.

• Implement a factorial program using Python

• Explore the @jit decorator and use it to optimize code

• Do a time comparison of our optimized code

We explore the different compilations modes available in Numba and dig deeper into implementing universal functions using Numba.

• Understand the various compilations modes available using Numba

• Implement NumPy’s universal functions in Numba

• Understand different modes of operations for @vectorize

We explore the idea of using threads with optimized Python code using Numba.

• Explore the ways in which threading can be activated in Numba

• Understand the different threading layers in Numba

• Implement a program using Threading with Numba

Finally, we do a time comparison of the different techniques we have learned to optimize Python code using Numba.

• Implement the basic version of do_some_op function and do time analysis

• Implement the NumPy version of do_some_op function and do time analysis

• Optimize the code using @jit and using eager evaluation

Synchronous and Asynchronous programming are two basic models of programming. Before moving on to advanced topics, we must understand the difference between the two and how they work.

• Understand the concept of Synchronous programming.

• Explore Synchronous programming with an example

• Understand the various drawbacks of Synchronous programming

Once we have understood the concept of Synchronous programming, we explore the concept of Asynchronous programming and how it is different from the former.

• Understand the concept of Asynchronous programming

• Explore the difference between Synchronous and Asynchronous programming

• Understand the various advantages of Asynchronous programming

Once we have explored the concept of Asynchronous programming, we must understand how we can apply it in Python using AsyncIO module.

• Explore the AsyncIO module

• Implement a simple Asynchronous example in Python

• Do a comparative analysis between a Synchronous and Asynchronous function

We explore the idea behind Distributed System architecture, it advantages and use cases.

• Understand what is a distributed system

• Explore how distributed systems work

• Discover the various advantages of distributed system

Dask is one of the most commonly used Python library for scalable and parallel Python code. To get the best performance for our code, we must learn how Dask works and how we can leverage it to parallelize our code.

• Understand the idea behind Dask module

• Understand why Dask is able to speed up our code

• Explore the various Use cases for Dask

Once we have a basic idea of Dask, we move towards setting up Dask on our systems.

• Setup Dask locally on our system

• Install all the different components of Dask in one go

• Setup Dask on Google Colab notebook

Before moving on to implementing our first program using Dask, we must understand the basic idea about how Dask arrays work and how we can use Blocked Algorithms

• Understand the concept Blocked Algorithms

• Explore Dask arrays and how they work

• Explore how Blocked algorithms work with Dask arrays

Once we have finished learning about the basics of Dask, we implement our first program using Dask arrays

• Implement a basic program using Dask arrays

• Understand how lazy evaluation works with Dask

• Explore the various functions of Dask arrays

Sometimes it is not possible to simply use Dask arrays to achieve parallelism. In such cases, we can use @delayed decorator to parallelize our custom code.

• Explore how @delayed works

• Implement a program with @delayed to speedup code

• Visualize task graph for Parallelized code

Finally we do a time and performance analysis for Dask code using the concepts we have learned in this section.

• Implement an operation for arrays and implement the NumPy code for the same

• Change the code using Dask arrays

• Do a time and performance analysis for all the techniques to record the speedup

Previously we learnt about Asynchronous programming and how we can implement it in Python. Reactive programming is also based on Asynchronous programming of data streams and, we need to have an overview about it before moving deeper into it.

• Understand the concept of Reactive programming

• Explore the different properties of Reactive programming

• Discover the advantages of Reactive programming

Once we have a basic idea of Reactive programming, we must learn about the basic building blocks of Reactive programming, i.e. Observables and Observers (subscribers).

• Understand the concept of Observables

• Understand the concept of Observers

• Explore how Observables and Observers (subscribers) work together

Data Operators allow us to perform operations on data streams in Reactive Programming. Before moving over to the implementation part, we must explore the various types of data operators available at out disposal.

• Understand the concept of data operators in Reactive Programming

• Explore different types of data operators and how they work

• Explore the working of data operators through an example

Once we have finished learning about the basics of Reactive Programming, we move towards implementing the same using RxPy module.

• Understand what exactly the RxPy module is

• Setup RxPy module

• Implement Observables and Observers in Python

After implementing a basic program using RxPy, we learn how to implement various data operators in Python and also learn how to chain different data operators together.

• Implement different types of data operators in Python

• Implement chaining of data operators

• Explore the various data operators available in RxPy module