Applied Statistics for Data Science: A Hands-On Approach!

In this lecture you will learn about the field of Data science, and where Statistics comes into play:

• What is data science's main objective.

• Why is Statistics so important?

• What is the basic Data Science pipeline?

• The responsibility of Data Scientists.

In this Lecture we will introduce the concept of the Histogram:

• How to build a histogram.

• What the histogram tells us about our data.

In this lecture we build on our knowledge of the histogram to introduce the Probability Density Function (PDF).

We will also briefly cover the main properties of the Probability Density Function of the Normal Distribution.

In this lecture we will learn:

• Different Types of Data

• The details of Tabular Data

• How to implement Tabular Data in Python

• How to convert Pandas Dataframes to Python Arrays

In his lecture we will learn:

• Why we want to generate our own data

• How we can generate artificial data using Python

• How to visualise data in a histogram using Python

In this lecture we will learn:

• The central idea in Statistics

• The difference between Population and Sample

• What is Representative Data?

• How to simulate the Sample-Population dynamic using Python

In this lecture we will learn:

• How to compute the Mean, Median, and Variance of our Sample and Population

• How to collect these Basic Statistics into a Pandas Dataframe

• How to showcase that these number change from sample to sample.

In this lecture we will learn:

• How to Visualise Population and Sample histograms

• How to Visualise the Basic Statistics onto our histograms

• Visually compare the Basic Statistics between Sample and Population

• Visually observe the Sample Statistics change from sample to sample

In this lecture we will learn:

• How to build an elaborate simulation using Python

• How to visualise the fluctuations of Sample Statistics for sample with different sizes.

• How to draw conclusions from the resulting Visualisation

In this lecture we will learn:

• The three different levels of data distributions

• What the Central Limit Theorem actually means

• How to simulate the Central Limit Theorem with our own data, using Python

• Use our simulation to play around with the Central Limit Theorem

In this lecture we will learn:

• How to extend the Central Limit Theorem to non-normal distributions

• Where the Central Limit Theorem breaks down!

In this lecture we will:

• Revisit the idea behind Population and Sample, but now include theoretical values.

• Learn how to implement these concepts using Python code

• Build a visual understanding of the mean.

In this lecture we will learn:

• What are Data Percentiles

• How to compute Data Percentiles using Python

• What are Data Intervals

• How to compute Data Intervals using Python

• How to Visualise Percentiles and Data Intervals using Python

In this lecture we will learn:

• What the Standard Deviation really represents.

• How to build a bridge between Data Intervals and the Standard Deviation

• How to visualise this connection in a beautiful graph, using Python

• Discuss the different sigma-levels of a normal distribution

In this lecture we will learn:

• What the Cumulative Distribution Function (CDF) is and what it represents

• How to generate a CDF and how to compare it with the Histogram

• Why the CDF is useful

• How to implement a CDF using Python

• How to visualise a CDF using Python

In this lecture we will learn:

• The details of the normal distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

In this lecture we will learn:

• The details of the Uniform Distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

In this lecture we will learn:

• The details of the Exponential Distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

In this lecture we will learn:

• The details of the Poisson Distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

In this lecture we will learn:

• The details of the Bernoulli Distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

In this lecture we will learn:

• The details of the Rayleigh Distribution

• What parameters it requires

• In which scenarios it occurs

• What the histogram and CDF tell us about this distribution

I would love to hear your feedback. It would really help me improve the quality of these courses (:

In this lecture we will introduce statistical testing by going through an example step-by-step. Doing so will show the necessity and reasoning behind statistical testing and the p-value.

In this lecture we will learn:

• What the p-value really represents building on our reasoning of the previous lecture

• Introduce the null-hypothesis

• How Statistical Significance is connected to the p-value

• Examples of difference significance levels

• Important consequences of choosing a significance level!

In this lecture we will learn:

• How to implement the reasoning behind Statistical Testing in Python Code

• How to manually compute the P-value

• How to visualise the P-value on a histogram

• How to visualise the Statistical Significance on a histogram

• How to use built-in functions to compute the P-value

In this lesson we will play around with the simulation we built:

• Change the underlying data statistics

• Change the underlying data sizes

• Observe where the p-value gives wrong conclusions

• Learn from these changes to build an intuitive understanding.

In this lecture we will learn how to test for normalcy:

• Define the problem context

• Implement the Shapiro-Wilk test using Python

• Visualise the Statistical Test using Python

• Implement the Q-Q plot

• Compute the R-squared value for the Q-Q plot

• Use the CDF comparison method

In this lecture we will learn how to test for Equal Variances:

• Define the problem context

• Implement the Levenes test using Python

• Visualise the Statistical Test using Python

In this lecture we will learn how to test for Equal Means between two distributions:

• Define the problem context

• Learn about Independent VS paired data sets

• Implement the Independent Student-T test using Python

• Implement the Paired Student-T test using Python

• Implement the Welsh's-T test using Python

• Visualise the Statistical Test using Python

• In this lecture we will learn how to test for Equal Means between Multiple Populations:

  • Define the problem context

  • Learn bout the difference between "Between-variance" and "Within-variance"

  • Implement the ANOVA test using Python

  • Visualise the Statistical Test using Python

In this lecture we will learn how to test for Equal Distributions:

• Define the problem context

• Implement the Kolmogorov-Smirnov test using Python

• Visualise the Statistical Test using Python

• Use the Kolmogorov-Smirnov test to test for specific distribution (hack!)

In this lecture we will learn about non-parametric tests:

• Define the problem context: What Are Non-parametric Tests?

• Discuss the different non-parametric tests (Brown-Forsythe, Mann-Withneyu, Willcoxon, Kruskal-Wallis)

• Implement these tests in Python.

In this lecture you will learn how to detect a biased coin with 95% certainty:

• How to take a problem statement and transform it into a statistical test.

• How to map the concept of the p-value onto a realistic problem.

• What are the statistics of coin flipping.

In this lecture we will:

• Implement our problem of detecting a biased coin using Python.

• Create a beautiful and informative figure containing all the necessary information, using Python.

In this lecture we will interpret our results:

• Play around with the simulation parameters

• Increase the bias of the coin

• Increase the number of coin flips

• Observe the limitations of the p-value

In this lecture we will go through a very realistic scenario of A/B testing from start to finish:

• Learn about the problem statement

• How to gather the suitable dataset

• How to translate the problem into a statistical test

• How to implement the statistical test

• How to interpret the results

• How to formulate a conclusion for upper management

In this lecture you will learn:

• What correlation between two variable actually represents

• What questions correlation seeks to answer

• That there are two types of correlation: between continuous and categorical variables

• An important note on correlation vs causation

In this lecture you will learn:

• What linear correlation actually means, both in words and visually

• What the strategy is to generate two data sets that are linearly correlated

• How to be able to fine-tune this correlation between the two generated data sets

• Interpret the visuals of linear correlation

In this lecture you will learn:

• How to write Python code that simulates two linearly correlated data sets

• How you can fine-tune this correlation

• How to visualise the correlation between these two data sets

• How to play around with the simulation parameters and interpret the results

In this lecture you will learn:

• How to quantify linear correlation

• What the Pearson Correlation Coefficient is

• How to compute this Coefficient and the corresponding P-value, using Python

• How to visualise the Pearson Correlation Coefficient

• What the two alternative correlation coefficients are: Spearman coefficient and Kendall-tau

In this lecture you will learn:

• What categorical variables are (with examples)

• What correlation means for categorical variables

• How to represent and visualise this correlation

• What the contingency table is

• How to generate categorical data sets that are correlated using Python code

• How to fine-tune this correlation in Python

• How to build the corresponding contingency table

In this lecture you will learn:

• How to quantify the correlation between categorical variables

• What the expected contingency table is

• What the null-hypothesis is

• What the Chi-Squared test measures

• How to compute the Chi-Squared test using Python Code