Statistics & Probability for Data Science & Machine Learning

Explore the measure of central tendency by computing mean, median, and mode using real datasets. Learn how distribution and outliers affect these values and their interpretation.

Learn to quantify data spread with range, interquartile range, percentiles, variance, standard deviation, and mean absolute deviation, plus outlier effects on mean and median.

Learn to quantify data spread with variance, standard deviation, range, mean absolute deviation, and outliers, and compare population versus sample formulas (n vs n-1) along with iqr-based boxplot insights.

Explore linear regression by plotting a line from two numerical variables, and interpret slope, intercept, and positive or negative relationships.

Explore covariance and the covariance matrix, using X (height) and Y (weight) to show positive and negative relationships, discuss sample vs population formulas, and relate to Pearson and Spearman correlations.

Explore how the Pearson correlation coefficient measures the strength and direction of the linear relation between variables, using positive, negative, and no-correlation examples, scatter plots, and the r formula.

Spearman rank correlation coefficient offers a monotonic, non-linear friendly alternative to covariance and Pearson. The lecture covers ranking data, handling duplicates, and interpreting values from -1 to 1.

Derive the regression equation from data by computing the correlation coefficient, slope, and intercept; then analyze residuals to assess model fit and linearity using predicted Y and actual values.

Explore the coefficient of determination, r-squared, to measure regression accuracy by comparing residuals to the total variance, and use rmse as the standard deviation of residuals.

Understand the normal distribution and the empirical rule, using mean, standard deviation, and z-scores on the standard normal curve. Apply these tools to estimate percentiles, medians, and comparisons of performance.

Apply the normal distribution and the empirical rule to compute areas under the curve for percentages below or above a z-score using the z-table, with mean and standard deviation.

Examine the normal distribution (Gaussian) with z-scores, percentile calculations, and standardization to set screening thresholds based on mean and standard deviation.

Explore symmetric distribution, skewness, kurtosis, and kernel density estimation; learn how skewness indicates asymmetry, how excess kurtosis compares to normal, and how bandwidth controls smoothing in kernel density estimation.

Explore the basics of probability with coin tosses, dice, and marbles to compute simple outcomes and events. Contrast theoretical and experimental probability and relate them to data science applications.

Explore statistical significance through real-world experiments on advertising effects on children's eating, and master probability concepts, including sample space, independent events, and the addition rule.

Examine dependent probability through draws from a bag, biased coins, and dice, and apply conditional probability using P(A) times P(B|A) and P(B) times P(A|B).

Explore how false positives and false negatives affect the probability of actual drug use given a positive test, with a 5% base rate, and examine weather delays' dependence on snow.

Master permutation and combination concepts for data science and machine learning through practical examples, building sample spaces and applying factorial-based formulas to compute arrangements and probabilities.

Explore combinatorics in probability, using permutations and combinations to count outcomes—like exactly three heads in eight coin flips, basketball free throws, and officer selections—guided by factorial-based formulas.

Explore discrete and continuous random variables, probability distributions, and key measures like mean (expected value), variance, and standard deviation, with practical examples and transformations.

This lecture derives variance rules for independent random variables, showing Var(X+Y)=Var(X)+Var(Y) and Var(X−Y)=Var(X)+Var(Y), discusses dependent cases, and applies to real problems like commute fuel consumption and height comparisons.