
Navigate to coronavirus-project.dunderdata.com to view the live dashboard!
Download the course material, unzip the file, and move the folder to an organized data courses location instead of keeping it in downloads or the desktop.
Explore the course structure by inspecting the notebooks, project folder, and a single PDF; note that the PDF is a convenience, while 13 chapters map to 13 notebooks.
Update!
Step 7 has changed since the video was created. Make sure to use the step below in the text!
Create the virtual environment quickly. Use the next video for a more detailed explanation.
cd into the project directory
python -m venv dashboard_venv
source dashboard_venv/bin/activate
pip install -U pip
pip install wheel
pip install -r requirements.txt
pip install matplotlib==3.4.3 nbconvert==5.6.1 notebook==6.1.6
pip install jupyter-dash==0.4
Note for windows users - Activate your virtual environment by running the command dashboard_venv\Scripts\activate.bat. Do not use source as shown in the video.
Update!
Step 7 has changed since the video was created. Make sure to use the step below in the text!
cd into the project directory
python -m venv dashboard_venv
source dashboard_venv/bin/activate
pip install -U pip
pip install wheel
pip install -r requirements.txt
pip install matplotlib==3.4.3 nbconvert==5.6.1 notebook==6.1.6
pip install jupyter-dash==0.4
Note for windows users - Activate your virtual environment by running the command dashboard_venv\Scripts\activate.bat. Do not use source as shown in the video.
Activate and deactivate the virtual environment to control Python mappings and libraries, using the deactivate command or the bin activate script, and optionally reactivating the conda environment for consistent dashboard.
Launch the Dash dashboard by activating the virtual env and running python dashboard.py, then explore world and U.S. data, interactive tables, charts, maps, and hover details, with stop via Ctrl-C.
Open the Jupyter notebooks by activating the virtual environment in the project directory and launching Jupyter Notebook from the course’s notebooks folder to run the dashboard code.
Navigate a hands-on data analytics course by reading and completing notebooks, watching videos, and solving exercises with production code and provided solutions.
Learn to download data by building a Python function that fetches csv data from Johns Hopkins, mapping world or usa groups to cases or deaths.
Download all data feeds from John Hopkins repository, storing each group-kind pair in a dictionary mapping to pandas data frames, including world cases, world deaths, USA cases, and USA deaths.
Save data locally by writing a dictionary of data frames to a local directory, using each key as the file name, with optional keyword arguments passed to the write operation.
Learn to clean and transform data by selecting the correct columns in pandas. Use the columns attribute, is in filters, and two forward slash date filters with a local indexer.
Improve data preparation by updating the run function to apply successive cleaning and transformation steps to a data frame, starting with selecting columns and progressing toward modeling readiness.
Standardize area names across four data frames using a replacements dictionary. Aggregate cruise ships into 'cruise ship' and apply changes with pandas in a stepwise run.
Aggregate repeating areas by grouping the first column and summing all other columns to produce per-state totals for USA and per-country totals for the world across 379 days.
Transpose the data to move dates into rows and countries into columns, creating a time series with a date time index using pandas.
Identify and fix bad data in cumulative totals using a pandas mask and linear interpolation in Python, demonstrated with Spain data.
Encapsulate all data cleaning and transformation steps into a single prepare data class, with an option to download new data or read local data, producing four prepared data frames.
Explore seven-day moving average smoothing of daily data in pandas, center windows, and relate it to cumulative trends while noting potential end bias.
Explore lowess smoothing, or locally weighted scatterplot smoothing, a technique that fits many short linear regressions around each point with distance-based weighting and frac control.
Learn to implement a single function that applies loess smoothing to cumulative data, deriving daily values, using a data-driven window fraction, and returning an aligned, non-negative smoothed cumulative series.
Explore exponential growth and decline using a × b^x, where a is the initial value, b the growth rate, and x the time unit; b>1 grows, b<1 declines toward zero.
Use scipy's least_squares to optimize parameters a and b for an exponential growth model of total cases, fitting Texas data from March to September 1 and minimizing squared error.
Train the model on data up to May 30 with least squares to predict unseen data, then compare to actual values and note divergence over time, motivating more stable models.
Automate model training by wrapping least squares optimization into a reusable function that fits a simple exponential model to any data series with given bounds and an initial guess.
Automate daily predictions by converting cumulative model outputs to daily values, using training days, future days to predict, and the model parameters from a simple exponential model.
Automate cumulative predictions by returning a daily cumulative series starting after the last model date, adding daily predictions to the last known value with pandas date alignment.
Plot the original and predicted series with a function that takes two series and a title, filters the original to the last prediction date, and shows a legend.
Automate smoothing, training, predicting, and plotting to build a data analytics model with a mega function chaining smooth, train_model, get_daily_prediction, get_cumulative_prediction, and plot_prediction, using a start date to model waves.
Explore continuous exponential growth models by moving from discrete compounding to e-based continuous growth, using code to fit and compare models for coronavirus cases.
Model exponential decline in coronavirus curves, introducing increasing exponential decline with a shift parameter to keep values positive; compare with growth using Italy data and note the upcoming logistic model.
Explore asymptotes in logistic growth models and how they differ from exponential curves, using the coronavirus cumulative curve and 1/x, a/(b+x^2) to illustrate limits.
Explore the logistic function and s-curves with asymptotes, and learn to rotate and shift the curve to model coronavirus case trajectories using three parameters: L, X_not, and K.
Implement a logistic function and tune its parameters (L, k, x0) to fit data, plot the curve against Italy's case data, and practice manual curve fitting.
Smooth Italy case data, fit a logistic model by least-squares to estimate L, x0, and K within defined bounds, and project future cases up to May 1 using the model.
Explore the generalized logistic function to capture asymmetry in growth curves, using a new parameter V to shift and shape the curve and improve data fitting.
This lecture demonstrates handling multiple waves in a generalized logistic model by limiting data to the most recent 50 games, separating waves, and testing model fit against Australian data.
Apply a vertical shift to the generalized logistic function with a new parameter s to better fit a second coronavirus wave, improving the asymptote behavior and trajectory prediction.
Explore how the upper bound of the L parameter shapes the predicted curve for Australia's second wave, using a June through August training period and varying max L.
Automate parameter bounds by using the last smoothed value and a 3.5% daily growth to bound L with five- and fifty-day projections, and derive the midpoint as the initial guess.
Discover why detecting wave starts is unnecessary; the model shifts vertically and horizontally to align with wave, from Italy, United Kingdom, and New York state data toward logistic asymptote.
Define the final model as a generalized logistic function with L, X0, K, V, and S parameters, and apply data smoothing with recent data for a dashboard-wide analytic model.
Encapsulate data workflows by organizing smoothing, training, predicting, and combining results into the case model class, handling data preparation, parameter estimation, and daily and cumulative forecasts.
Instantiate the cases model, load data via the prepared data class, train on 60 days, smooth with 15 points, forecast 30 days with bounds, and view world and USA results.
Predict deaths by estimating the case fatality ratio from lagged cases, using a 50 day lag and 30 day windows to compute CFR for US and world.
The deaths model class computes a CFR per area using lag and period, then predicts future deaths by multiplying cases from past dates by that CFR, followed by smoothing.
Instantiate and run the deaths model with training data to generate daily and cumulative death predictions for the first 10 states, and view a 30-day forecast with a 15-day lag.
Combine world and USA data frames to compute daily and cumulative cases and deaths, reshape with stack and concatenate, and export a single multi-index CSV file for the dashboard.
Create a current date summary table by left joining the data frame with the population table and computing deaths per million and cases per million for the dashboard.
Explore how the cases model and the deaths model live in the models module, and how the prepared data class and the combine all data function create summary tables.
Explore update.py, which runs all course data analysis to update two CSV files—one for all data and one for the summary—via the command line with an optional last-date argument.
Launch the update.py script to refresh data in the project and notebooks directories, then start the dashboard to view the latest data; activate the virtual environment if you used one.
Learn to create interactive visualizations with Plotly using a simple three-step approach—create a figure, add a trace, and update the layout—applied to Texas data.
Discover how to set trace properties for a scatter plot in a Python dashboard, including mode values and nested marker and line options via dictionaries.
Learn to update a polygraph layout in Python by adjusting the figure's height, width, title, axes, legend margins, and annotations with the update_layout method.
Plot a figure with multiple traces, showing actual and predicted values as two color-differentiated lines with a legend, using data up to the last date and the first prediction date.
Learn to automate plotting for a specific area by filtering a data frame, selecting a kind, and producing a bar plot of actual versus predicted values over time.
Create two subplots in a single figure with make_subplots to compare actual and predicted deaths and cases in Texas, arranging two rows by one column and tuning layout to consolidate legends.
learn to add customized subplot titles using annotations in a plotting workflow, employing update layout to label cases and deaths and customize legend placement.
Build interactive choropleth maps for global and US data by coloring by deaths or cases, using an add choropleth plot method and Robinson projection.
Learn to build interactive data visualizations with Plotly by creating figures with go.Figure or subplots, adding traces, and updating the layout, including geo maps with polygon data.
This course teaches you all of the skills to build interactive data analytics dashboards with Python. Specifically, you will be building a Coronavirus Forecasting Dashboard that shows historical and predicted values for deaths and cases for all countries in the world and US states from the ongoing coronavirus pandemic. The final product will be a live dashboard, automatically updated daily, hosted on a remote server for anyone, anywhere in the world to see!
You will learn and complete the entire process for building the dashboard including the following topics:
Getting, Cleaning, and Transforming the Data - You will learn how to collect the data, find and clean bad data, and transform it so that it can be used for building models capable of prediction.
Data Smoothing - You will learn several different techniques such as LOWESS to smooth the jagged raw data so that the model can better detect trends.
Exponential Growth and Decline Models - You will begin modeling coronavirus cases for each area of the world with these simple models that can capture a single exponential growth or decline phase, but not both.
Logistic Growth Models - You will learn about a separate class of "S-Curve" models capable of capturing both exponential growth and decline in the same model.
Modeling New Waves - Coronavirus cases appear in waves over different time periods. You will learn how to change your model so that it can detect any number of new waves in the future.
Encapsulation into Classes - After selecting your model, you will encapsulate all of the code together into Python classes, eventually to be used in your final production code.
Visualizations with Plotly - You will learn how to use the Plotly Python library to create interactive data visualizations targeted for the web
HTML and CSS - You are building a web application and will learn the fundamentals of HTML and CSS to help add customization with the help of Dash.
Building the Dashboard with Dash - You will learn how to build all of the components and interactivity of the dashboard with the Dash Python library.
Deployment - One of the most exciting parts of a project is deploying it on your own server for the world to see. You will learn two different deployment options - one simple and the other more complex, but with more flexibility.
What's Included?
This course comes with a massive amount of material including:
13 Jupyter Notebooks
26 Exercises with detailed solutions
200 page PDF of the entire course content
All production code for the dashboard
Technologies used
All code for developing the dashboard will be done using Python
Pandas will be used extensively for analyzing and transforming data
Statsmodels will be used for smoothing
Scipy will be used for building the models for coronavirus cases and parameter optimization
Matplotlib will be used in the notebooks for static visualizations
Plotly will be used for interactive data visualizations that appear in the dashboard
Dash will be used for building the dashboard itself
HTML/CSS will be used together with Dash to customize components of the dashboard
You'll learn how to setup your own Linux Ubuntu server to run your dashboard