Modern Python Application Development in Practice!

This video gives an overview of this section.

In this video, we look at a demo of the final product after the three sections, including a templated Django response and browsing a Django REST Framework API.

• Look at HTML response

• Look at REST API server

In this video, we will install Python, Django and the Django REST Framework onto our development computer

• Install Python 3 based on the Operating System of our computer. Create a Virual Environment

• Use Pip to install Django and the Django REST framework

• Use pip requirements.txt to identify installed python components in a development environment

In this video, we will install PostgreSQL, create a new database and configure Python support

• Install PostgreSQL into our development environment

• Use Psql to check our installation and create a new database

• Use pip to install PostgreSQL support into Python

In this video, we will create a new Django project and then create an application within that project.

• Create a project directory and use django-admin startproject

• Use manage.py startapp to create a Django application

• Review Django external application publishing guidelines

In this video, we will use the development environment we have setup in previous videos to show the Django initial success web page.

• Introduce settings.py and configuring our database

• Highlight the DEBUG flag and placing sensitive information in settings.py

• Use Django’s development webserver via the manage.py runserver command and seeing the Django initial default page

This video gives an overview of MVC and MVT Framework.

• Look at the concept of MVC and MVT Framework.

• Look at Django models

• Look at Django template

In this video, we will discuss Django models, create our first model, create methods on our models and test them.

• Introduce Django model and how relationships between Models can be used.

• Create a Django model

• Create methods on Django models and test them

In this video, we will apply migrations to our Django models, which keep our database structure up to date. We will perform queries on our database.

• Consider what Django migrations are and have Django create schema migrations

• Consider what data migrations are and create one, and then apply all migrations created so far in this video

• Perform some simple queries against our database

In this video, we will create a simple “Hello World” view, consider Django class based views, perform URL configuration and add a parameter to our view.

• Discuss Django class based views and use view to create a simple view that returns a hard coded response

• Perform URL configuration or routing at both the Project and App level to make our new view accessible. Access the view in a browser

• Make use of URL querystring and kwarg parameters. Discuss the pros and cons of each approach

In this video, we will discuss how Django handles requests, Django’s URL routing, matching URLs with path and re_path and HTTP methods.

• Discuss how Django handles requests from the request in to middleware, then URL config, then view and then the reverse for the response

• Review URL configuration from previous video and discuss use of path() and re_path()

• Cover methods (verbs) of Http Requests including Get and Post requests. How Http methods then translate to class methods

In this video, we will cover Django Templates and the TemplateView. We will cover Django’s template inheritance system, how to pass data to templates, where to place template files and template tags and filters.

• Explain how hardcoded responses are undesirable. Introduce Django’s TemplateView

• Learn about using base templates and view specific templates that extend base templates. Learn how to make data available within templates.
  Review template settings in settings.py and see our first set of templates in action

• Explain how templates should not perform core logic, however we can use template tags and filters to perform presentation logic

In this video we will consider why we should test, the core tools Django provides for testing, how to configure PostgreSQL for testing and some integration and unit tests.

• Learn how we need to use automated testing tools and how we can write test cases that place our code under test

• Make use of pqsl to configure PostgreSQL for testing

• Develop Integration tests that make use of the Django test client. 
  Develop unit tests that test specific view functions using a RequestFactory as well as class attributes

This video gives an overview of the entire section and then considers what is an API, what is REST, REST resources, methods and CRUD operations and HTTP Status codes.

• Explain the difference between traditional web responses and API responses. Learn how REST provides an API on HTTP

• Learn what Resources, HTTP Methods and CRUD operations are

• Explain how a result of an API request is communicated via a status code

In this video, we will create an API end point by configuring a URL endpoint for a view that generates JSON output. We will also accept input and perform simple validation.

• Make use of a URL pattern configuration to give access to our API view

• Define a View using the base View class that uses JsonResponse to return an API response

• Include in our View definition validation of a name parameter and return an error if this is equal to fred. Show results in a browser

In this video, we will discuss why we use the DRF, what key parts of the DRF are, who uses it and the web browsable API feature of the DRF.

• Learn how the DRF helps us in providing code that we would otherwise have to provide ourselves many times over

• Learn what the key parts of the DRF are, including serializers, DRF views, routers and authentication and permissioning. Learn how these relate to CRUD operations

• Explore who uses the DRF and see the web browsable feature of the DRF

In this video, we will learn about DRF serializers. We will first look at manually defining a serializer and then use ModelSerializer to base a definition on an existing model.

• Review our existing models with focus on the Bookmark Model, then we will define a serializer with create and update methods

• Using the Python console, we will create some initial data in the database and then demonstrate what our serializers do with this data along with simulated received data

• We will define a serializer based on Bookmark using a ModelSerializer. 
  We will then demonstrate how this serializer can be used with a list of Bookmarks in the Python console

In this video, we will combine our learnings from the previous two videos with DRF views by building a Bookmarks list and a Bookmarks detail view.

• Learn about basic DRF view URL configuration, including format suffixes. 
  Define a Bookmark List view that can build a list and create new Bookmarks

• Learn about basic DRF detail view URL configuration. Define a Bookmark detail view that can get detail, update a Bookmark and delete a Bookmark

• Watch a demonstration of all of the functionality previously developed in this video, including listing existing Bookmarks, creating new Bookmarks, updating existing Bookmarks and deleting them.

In this video, we will replace our views from the previous video by exploiting pre-built functionality of the DRF for common CRUD operations.

• Make use of ListCreate-APIView for Bookmark listing and new Bookmark creation

• Make use of RetriveUpdate-APIView for Bookmark detail queries, updating a Bookmark and deleting a Bookmark

• Take advantage of the ModelViewset that combines all the functionality seen earlier in this video in one convenient package.

In this video we will use DRF routers to clean up our URL configuration and create a specialised API action.

• Learn about Hyperlinked-Model-Serializers and how they can create related links in our API. Significantly simplify our URL configuration with a DRF router

• Create a specialised viewset action that can add a like to a Bookmark. Learn how to define this in the viewset and ensure the DRF router is aware of it. See how it is accessed from the browser

• Consider each of the view strategies covered in this and previous videos and scenarios where each would be the best fit

In this video we will consider how we can use DRF automated testing tools to perform integration and unit tests of our API.

• Learn how to write an integration test of a DRF based viewset and place different actions under test

• Learn how to write a unit test of specific methods of a DRF based viewset, how to isolate dependencies via patching and using mocks to test each operation of the code

• See how the Django test runner is used in combination with DRF test tools

This video gives an overview of the entire course.

Find the most common causes of performance issues in Python applications.

• Distinguish CPU bound tasks and I/O bound tasks

• Use the time Unix command to diagnose performance issues

• Use the cProfile Python module for advanced profiling

Take an existing I/O intensive Python application and speed it up.

• Understand concurrent I/O task execution in Python

• Use a ThreadPollExecutor to parallelize I/O tasks

• Use the time command to check the speed increase

Take an existing CPU intensive Python application and speed it up.

• Understand concurrent CPU task execution in Python

• Use a multiprocessing Pool to parallelize CPU tasks

• Use the time command to check the speed increase

Take an existing CPU intensive Python application and speed it up using a C/C++ library.

• Create a Python extension that calls C/C++ code

• Parse Python arguments in C/C++ and return C/C++ objects to Python

• Use setuptools to build the extension

Speed up a Python application by using an alternative Python implementation.

• Choose an appropriate Python implementation

• Replace C/C++ extensions in your code with plain Python implementations

• Use PyPy to run the refactored application

Prepare your Python module to be distributed to the Python Package index.

• Write the necessary files for setuptools distribution

• Configure module using the setup.py file

• Choose a license for your code

Distribute your Python module to the Python Package Index in 2 popular formats.

• Create and inspect a source distribution from your Python module

• Create and inspect a wheel distribution from your Python module

• Use twine to upload your distributions to the Python Package Index

Distribute a user-facing Python application to end-users in a friendly way.

• Use PyInstaller to generate executables for a command-line application

• Use PyInstaller to generate executables for a GUI application

• Inspect resulting executables

Distribute a web application using Docker images and containers.

• Docker definitions

• Write a Dockerfile for a simple Flask application

• Build a Docker image and run a container

Ensure your code works as expected by writing unit tests.

• Identify the testable units in your code

• Write unit tests using PyTest

• Calculate the coverage to understand what is missing from your tests

In this video, you’ll deal with the complex part of the testing and also include the coverage part.

• Create complex part of the unit testing

• Measure the amount of code which is tested

Ensure that the code you are writing is working correctly right from the start.

• Translate requirements into unit test cases

• Implement the code after writing the tests

• Change the code until the tests are passed

Ensure that your code integrates properly with external units and that any external changes will be caught early.

• Write tests for functionality not units

• Run the tests in production-like environments

• Check test results regularly

Ensure functionality requirements are met throughout the development lifecycle.

• Define features using the Gherkin syntax

• Implement the scenarios using pytest-bdd

• Build the code until the requirements are met

Install application dependencies separately from each other and from the system environment.

• Identify the components of a typical Python installation

• Create separate Python environments with virtualenv

• Use the virtualenvs to isolate application dependencies

Recreate an entire Python environment to use on a different machine.

• Install dependencies in a virtualenv

• Freeze dependencies in a requirements.txt file

• Use pip to recreate the environment based on requirements.txt

Debug Python applications with built-in tools.

• Run your application using pdb

• Use pdb commands to control the execution

• Use the dis module to inspect Python bytecode

Quickly prototype ideas in Python and share interactive prototypes with other developers.

• Use the Python shell to research a new library

• Use Jupyter to create an interactive prototype

• Use Jupyter to visualize data plots

Find stylistic issues in your Python code.

• Navigate PEP 8 to understand the common guidelines

• Use pylint to check your code for style issues

• Customize pylint messages to fit your needs

Check your code for potential bugs before running it and import all reports in one place.

• Activate SonarQube Python rules

• Use sonar-scanner to check for style issues and potential bugs

• Add unit test coverage and results in SonarQube

Check Python types before running the code. Remove run-time type checking.

• Identify the need for type checking in Python

• Add type hints to your code

• Use mypy to check the types before running

This video gives glimpse of the entire course.

Time is of essence and timeit is your tool. In this video we explore timeit.

• Understand what is timeit

• Set up our timing experiment

• Run our timing experiment

In this video, we will look at profile module and understand where time is spent.

• Understand what is profiling

• Understand how Profiling our Fibonacci sequence generator is done

• Profile the sequence generator using the least recently used cache

In this video we will explore and understand tips and tricks which will allow us to track time more precisely using cProfile.

• Understand the difference between cProfile vs profile

• Explore and create artificially slowed functions

• Understand profiling using cProfile

In this video we will understand the memory_profiler.

• Explore how to go about profiling a simple function

• Explore and understand how profiling manual garbage collection is done

This video is all about reducing the execution time and memory consumption.

• Understand what are slots

• Create classes that use slots

• Inspect the performance of slots vs class dicts

In this video we will use tuples instead of lists when your data does not change.

• Understand what are tuples and how to declare them

• We will see by example that tuples are fasters than lists

• We will understand why are they faster than lists

Save on memory consumption with generators instead of lists.

• What are generators?

• Why are they memory efficient?

• Implementing generators

Generators are designed for a specific purpose, here is when not to use them.

• when you need features from a list

• when you need to iterate through a list multiple times

Chain together a series of processes so one output is another's output.

• Composing two generators

• Using parenthesis to define generator comprehensions

Illustrate why we need to move from Python lists to NumPy arrays

• Problem 1: Memory consumption of lists for large amounts of data

• Problem 2: Dynamic Typing

Show how we can use NumPy and the different ways we can create NumPy arrays

• The difference between NumPy arrays and lists

• Creating NumPy arrays from Python lists

• Creating NumPy arrays from NumPy helpers

Show that for large n, NumPy arrays are much faster than list comprehensions

• Setting up our problem with list comprehensions

• Setting up its cousin in NumPy

• Comparing their performance

Show how to use broadcasting - applying binary functions on arrays of varying length

• What is NumPy's broadcasting?

• Broadcasting: an example

• How broadcasting works - three examples

Show the simple interface of numexpr and where we can use it to optimize

• What is numexpr?

• How to use numexpr?

• Use cases of numexpr

If you have a list of URLs, serially requesting the HTML body of each page is super slow

• Using requests to scrape the HTML of a web URL

• Timing 10 web scraping requests

What are coroutines and how they help you request all webpages at once

• What is gevent?

• What are semaphores?

• Creating a gevent scraper

What is event driven concurrency, and how that can speed up web scraping

• What is tornado?

• What is an IO Loop?

• Creating a tornado based scraper

Futures and asyncio driven concurrent web scraping

• What is asyncio?

• await and async

• Creating an asyncio based scraper

Introduce the audience to parallel programming concepts.

• What is parallel processing?

• Embarrassingly parallel and less embarrassingly parallel

• The global interpreter lock of Python

Show how the decorate-sort-undecorate pattern can be beaten

• The decorate-sort-undecorate pattern

• Beating the DSU pattern with a sort key

Show how we can use Pool to achieve multiprocessing

• What is Pool?

• Our first process Pool

• Asynchronous process Pools

In this video we will understand how we can stop processes from interfering with each other using locks.

• We will be using multiprocessing.Value to share variables

• Why does it not add up?

• Using a lock to remove overwrites

This video covers the effects of many processes.

• Using the logging module

• Creating the right logger format for multiprocessing

• Logging multi-processes

This video is focused on understanding how to copy and clone objects the right way.

• Identify the wrong object instance

• Use the correct object clone

This video will show us how to mini-classes of Python: namedtuples

• Learn about mini-classes of Python

In this video we will understand how to create smart values with static methods and properties.

• Create smart values with static methods and properties

In this video we will be understanding how to compare two different objects.

• Understanding how to compare two different objects

In this video we will explore how to do real OOP by implementing abstract base classes in Python.

• Do real OOP by implementing abstract base classes in Python