Mathematical Statistics for Data Science

Explore random variables, sample spaces, and probability distributions, including discrete pmf and continuous pdf. Learn about Bernoulli distribution, uniform distribution, and normal distribution, and how parameters describe their centers.

Model a two-outcome experiment with zero or one, where one occurs with probability p and zero with probability 1-p, expressed via the pmf.

Explore the uniform distribution, a continuous model with equal likelihood on [0, theta], its density function 1 over theta, and computing interval probabilities via area under the curve.

Study the normal distribution, a bell-shaped continuous distribution centered at mu with variability sigma, and apply the empirical rule that 95% of values lie within two sigma of mu.

Define a random variable and its sample space, describe Bernoulli distributions with parameter p, and explain continuous distributions like uniform and normal through probability density functions and shaded areas.

Determine the expected value of a Bernoulli random variable by summing x times its PMF over {0,1}, yielding p, the probability of X being one.

Compute the expected value of a uniform random variable on [0, theta] by integrating x times the pdf 1/theta from 0 to theta, showing the mean is theta over two.

Mean, or expected value, of a normal distribution equals mu due to symmetry, and the mean also equals the median; the lecture discusses existence and contrasts with the Cauchy distribution.

Explore the expected value of random variables, from sample means to population means, and how Bernoulli, uniform, and normal distributions determine their means for the method of moments.

Apply the method of moments to estimate parameters by setting the sample mean equal to the first moment, solving for theta to obtain estimates.

Compute the method of moments estimator for p in a Bernoulli distribution by equating E[X] with p and using x-bar as the approximate mean, yielding p-hat = x-bar.

Estimate theta for a uniform(0, theta) distribution using the method of moments by equating theta/2 to the sample mean to obtain theta_hat = 2 x-bar.

Apply the method of moments to a normal distribution by equating the sample mean to the distribution's mean, yielding mu_hat = x_bar.

Learn the method of moments, linking the expected value to the sample mean to form estimators for Bernoulli, uniform, and normal distributions, and prepare to study unbiased estimators.

Explore the sampling distribution of estimators and how to compare them. Learn how bias, defined as expected value minus true value, identifies unbiased estimators.

Apply core properties of expected value to estimators by pulling out constants, summing expectations, and using the law of the unconscious statistician to analyze bias.

Learn that the method of moments estimator for a normal distribution is the sample mean, and prove that its expected value equals the population mean, making it an unbiased estimator.

Define bias as the difference between an estimator's expected value and true value, and prove unbiasedness for Bernoulli, uniform, and normal estimators using linearity of expectation and method of moments.

Identify how variance measures spread, compare normal and uniform distributions, and derive Var(X) = E[X^2] − (E[X])^2 to compute variance from E[X] and E[X^2].

Compute the variance of a Bernoulli random variable by using Var(X)=E[X^2]-E[X]^2, showing that for 0-1 outcomes the variance equals p(1-p).

Compute the variance of a uniform distribution on zero to theta by using E[X]=theta/2 and E[X^2]=theta^2/3, yielding var(X)=theta^2/12.

Explain why the variance of estimators matters by comparing unbiased estimators with different variances using a political poll example, and review variance properties for constants and independent sums.

Explore the variance of the method of moments estimate for p in a Bernoulli distribution; p-hat equals the sample proportion, with variance equal to p(1-p)/n, and larger n reduce variability.

Learn how maximum likelihood estimation uses the likelihood function, not the pdf, to infer the mean of a normal distribution from data, starting with a single observation and known variance.

Explore how to form joint pdfs and joint likelihoods from a random sample, using Bernoulli, uniform, and normal distributions to perform maximum likelihood estimation.

maximize the log-likelihood by differentiating with respect to theta, setting the derivative to zero, and solving for theta, using the score function and second derivative to confirm a maximum.

Review how logarithms convert multiplication to addition, simplify the log-likelihood by turning products of pdfs into sums, and apply exponent and denominator rules for easier differentiation.

Learn how to compute the maximum likelihood estimator for a uniform distribution, showing that theta hat equals the sample maximum, and compare it to the method of moments estimator.

Examine mean squared error as bias squared plus variance, comparing maximum likelihood estimator and method-of-moments for a uniform distribution, highlighting when slight bias with low variance outperforms unbiased estimates.

Derive the maximum likelihood estimator for the mean of a normal distribution, showing that mu hat equals the sample mean x-bar, the best estimator under normality.

Learn how maximum likelihood estimation uses likelihood and log-likelihood, with the score and second derivative, to find estimates and compare to method of moments for Bernoulli, normal, and uniform distributions.

explain why the fisher information and the crlb cannot be computed for the uniform distribution, due to parameter-dependent support and the distribution’s exclusion from the exponential family.

Derive the Cramer–Rao lower bound for mu in a normal distribution, showing single-observation information is 1/σ^2; with n observations, the bound is σ^2/n, and X-bar attains it as best estimator.

Define efficiency as the ratio of the Kromer rule lower bound to an unbiased estimator's variance and compare estimators by variance, illustrating asymptotic relative efficiency with mean and median examples.

Explain the Fisher information as the negative expected second derivative of the log-likelihood, and show that for Bernoulli and normal distributions the estimator variance meets the lower bound.

Explore the asymptotic distribution of estimators and apply the central limit theorem to assess probability statements for large samples.

Know that for a normal distribution, mu-hat equals the sample mean and is exactly normal; the nine-sample IQ example shows mean 100 and sd 5, with 95% within 90-110.

Demonstrate how consistency ensures that as sample size grows, the estimator converges to the true value, with decreasing variance and a distribution concentrating near the truth.

Learn to construct confidence intervals by pivoting from theta hat to theta, using the central limit theorem to create 95% intervals that contain the true parameter with repeated sampling.

Construct a 95% confidence interval for theta in a uniform distribution using the method of moments estimator theta_hat. Pivot the interval around theta_hat to get [10,14] for n=48.

Compute a 95% confidence interval for mu in a normal distribution using x bar plus or minus two sigma over sqrt(n); with four samples, interval is 63 to 71 inches.

Connects point estimators, such as method of moments and maximum likelihood estimates, to interval estimates via the central limit theorem. Demonstrates forming 95% confidence intervals as a long-run recipe.