Bayesian Statistics

Build on central probability concepts to introduce one of the most central topics in statistics, starting simple with several examples to illustrate the ideas.

Construct a contingency table to apply Bayes theorem and compute the probability of high temperature given sunny from joint and marginal probabilities in a weather example.

Apply Bayes theorem to determine the probability of having the flu given a positive test. Use a contingency table with prior 5% infection, 90% sensitivity, and 50% false positive rate.

Apply Bayes' formula to compute the probability of a high temperature given sunny weather, avoiding the full contingency table. Demonstrates multiplying and dividing probabilities to yield the result.

Bayes' theorem to update the probability you have the flu given a positive test, starting from a 5% prior with 90% true positives and 50% false positives.

Analyze the Monty Hall problem with Bayes' theorem to compute door probabilities after a host reveals a sheep and decide to switch or stay.

Advance through Bayesian statistics by working through examples and exercises to introduce base theorem; write the general formula on paper to prepare for upcoming sections.

Compute joint and conditional probabilities for weather and temperature using contingency table and base theorem. Summarize that sunny weather probabilities add up to 39% and verify against the contingency table.

Apply law of total probability to compute the chance of sunny weather by conditioning on low or high temperatures, showing how twenty-one percent and eighteen percent sum to thirty-nine percent.

Explore absolute independence and conditional independence in probability, illustrated with run and weather examples, and clarify how conditioning on a third variable affects independence.

Investigate whether conditional independence implies absolute independence by calculating the joint probability for John and Kate's runs using a contingency table across low and high temperatures.

For the conditional to absolute independence exercise, compute the second value for the contingency table: the joint probability of John not running and Kate not running.

Analyze a contingency table to compare the probability of John running given Kate runs with the marginal probability of John running, illustrating that conditional independence does not imply absolute independence.

Examine whether absolute independence implies conditional independence using a running partners example, analyzing joint probabilities and meeting events, and introducing notation for conditional independence given they meet.

Compute the probability P(John runs | Kate runs and they don't meet) using the absolute independence of John and Kate, yielding about 42.86 percent.

Apply Bayes theorem to compute P(John runs | they don't meet) using Kate and John's absolute independence, by formulating the joint probabilities and the necessary denominator for conditional probability.

Explore how absolute independence does not imply conditional independence in a John and Kate running scenario, as conditioning on Kate's run and their meeting status shifts probabilities.

Explore the two key learnings: absolute independence does not imply conditional independence, and conditional independence does not imply absolute independence.

Build intuition about absolute and conditional independence through a graphical model, using a separation procedure to decide which variables are independent given a third variable.

Explore how a bayesian network captures six variables: customer type, region, number of products, usage, price increase, and cancellation, and reduces conditional probability information compared with a full joint table.

Explore how Bayesian networks encode interdependencies among variables and determine whether customer type is conditionally independent of reach given the number of products.

Assess whether cancellation is independent of customer type in a second exercise, using intuition to judge the relationship between the second variable and customer type.

Describe d-separation steps in a Bayesian network: list variables, trace parents, disorient the graph, remove observed nodes, and assess if type and region stay connected given price.

Explore d-separation in simple Bayesian networks by examining Jon and Kate's independence given the temperature. See how conditioning on the temperature disconnects Jon and Kate, establishing conditional independence.

This lecture demonstrates d-separation in a simple Bayesian network with Jon and Kate; they are absolutely independent, but not conditionally independent given that they share a joint child.

Evaluate whether reachin is conditionally independent of price increases, given usage, within the d-separation framework from the second exercise.

Determine whether the region is independent of customer cancellation given a price increase, illustrating conditional independence concepts in Bayesian statistics.

Learn to use Bayesian networks to determine absolute and conditional independence and apply Bayes' theorem to compute precise probabilities of outcomes given others.

Explore calculating posterior probabilities in a simple Bayesian network, using temperature to predict whether John and Kate run and meet, with two approaches to exact computation.

Apply enumeration to compute the probability that John goes running and they do not meet, using conditional independence, a Bayesian network, and summing over temperature and Kate's running.

Calculate the denominator for the probability that John goes running given that they don't meet using the 42.25% joint and 38% non-meeting terms, then apply Bayes to obtain 52.6%.

Apply the enumeration approach to calculate the probability that Jon and Kate meet, given a higher temperature.

Master two methods for exact posterior probabilities in Bayesian statistics, network enumeration and elimination, and prefer elimination to reduce calculations while tackling the overarching problem.

Review probability, independence concepts, and Bayesian networks. Introduce d-separation and demonstrate calculating posterior probabilities using enumeration and elimination.

Investigate a Bayesian network from past cancellations in healthcare education software, testing independence and conditional probabilities among customer type, region, usage, price history, and contract cancellation.

This miniproject exercise examines conditional independence in a Bayesian network: given usage, customer type becomes irrelevant to predicting cancellation, illustrated on the Asian network.

Apply bayesian networks and bayes theorem to compute the probability of a customer from the north given observed factors, using elimination and conditional tables to reach about 22.23%.

Use Bayes' theorem to compute P(cancel|hospital) from joint probabilities, yielding about 25.14% for hospital patients.

Complete the mini project by applying all learned concepts in Bayesian statistics, and learn practical methods to estimate posterior probabilities in complex Bayesian networks.

Apply sampling methods to approximate conditional probabilities in a healthcare Bayesian network of customer cancellations, avoiding the computational burden of the elimination approach.

Explore three sampling methods—rejection sampling, Gibbs sampling, and a version of the Metropolis-Hastings algorithm—using conditional probability tables to estimate the probability a customer cancels given origin in the north.

Explore rejection sampling by iterating rounds, drawing from conditional probabilities, and accepting or rejecting samples to estimate a conditional probability. Learn convergence criteria and note potential high rejection rates.

Discover Gibbs sampling in a bayesian network, using the Mackoff blanket to simplify conditional probabilities and iteratively update variables to approximate the target probability.

Gibbs sampling iteratively updates one variable by sampling from its conditional distribution given others, comparing its efficiency to rejection sampling in Bayesian statistics and Bayesian networks.

This lecture formalizes the Metropolis-Hastings algorithm, using a uniform proposal over other variables, calculating an acceptance ratio from current and proposed probabilities, and iterating to infer conditional probabilities.

Estimate posterior probabilities in inference networks using rejection sampling, Gibbs sampling, and a Petropoulos Hastings algorithm; contrast Bayesian hypothesis testing with frequentist views in continuous statistics and machine learning.

Explore the definition of probability, compare the frequentist and Bayesian perspectives, and show how hypothesis testing and confidence relate to Bayesian statistics for analyzing programs.

Contrast the Bayesian view of probability, which assigns degrees of belief, with the frequentist view, which relies on long-term frequencies for repeatable events.

Explore frequentist hypothesis testing with 20 flips, testing H0: p=0.5 against p>0.5 by the binomial distribution to compute P(X≥14) ≈ 5.77%, yielding a fail-to-reject decision at 5%.

Bayesian hypothesis testing updates prior beliefs about the true proportion of heads using Bayes' theorem, combining priors with observed data to yield posterior probabilities.

Combine priors with the data to form posterior probabilities for the true hit probability, given 14/20. Highlight p=0.7 as the posterior mode and leading estimate.

Compare frequentist hypothesis testing with Bayesian inference using a coin toss example: 14/20 heads, null probability 0.5, prior distribution, posterior probabilities, and predicting the next flip.

Apply a bayesian approach to hypothesis testing by updating your prior beliefs—80% for equal colors and 20% for population preference—to compute posterior probabilities for blue being 1/3 or 0.5.

Apply Bayes' theorem to compute the probability of flu after two positive tests. Use a 1% infection rate, 90% positive test when infected, and 20% positive when not infected.

Use Bayes' rule to compute the posterior probability of the flu after two positive tests: prior 1%, likelihood 0.9 times 0.9, denominator combines flu and not-flu cases, yielding about 17%.

Apply Bayesian statistics to hypothesis testing with a loss function that weighs misclassification costs, using the 17 percent posterior to minimize expected loss.

Explore the rift between the frequencies perspective and Bayesian probability, linking long-term frequency with coin tossing and expressing belief in repeatable events, hypotheses, and credibility.

Explore continuous probability distributions, including the uniform on [0,1], the beta distribution with alpha and beta parameters, and the normal distribution with mean and sigma, highlighting shape shifts.

Compare discrete priors for the true heads probability with a continuous uniform prior on [0,1], and outline obtaining the posterior for p from 14/20 trials using Bayes' rule.

Using a beta(1,1) prior and binomial(20, p) data with 14 successes, the posterior for p is beta(15,7).

The lecture explains that with binomial data, the conjugate prior is a beta distribution, updating from a uniform prior to a posterior concentrated between 0.5 and 0.9 after 14 heads.

Explore conjugate priors across binomial, geometric, multinomial, and continuous likelihoods, showing beta, gamma, and normal priors yield the same distribution family in the posterior.

Explore continuous Bayesian hypothesis testing by updating a beta posterior with binomial data to compute probability that the true hits proportion lies to the right of 0.5, 0.6, and 0.7.

Compare rejection sampling with the Metropolis-Hastings algorithm, illustrating proposing states from a uniform or normal distribution, computing the acceptance ratio, and deciding to accept or reject via a uniform draw.

This lecture compares rejection sampling and Metropolis-Hastings to the exact posterior, showing MH converges quickly after about 1k iterations, while rejection sampling remains slower due to a rare event.

Explore how to sample from a normal distribution conditioned on being greater than two by implementing rejection sampling and the Metropolis Hastings algorithm, and discuss which algorithm to use.

Extend Bayesian ideas from simple discrete distributions like weather outcomes to continuous distributions. Use rejection sampling and the metropolis-hastings algorithm to estimate posteriors for complex models.

Explore how patient statistics relate to machine learning, showing their role in supervised and unsupervised learning within the latest section of the course.

Explore how Bayesian statistics underpins machine learning with probability for non-repeatable events, and compare supervised, unsupervised, and reinforcement learning; supervised learning predicts churn from inputs, using Bayesian networks.

Develop conditional probability tables for region given customer cancellation outcomes, and apply Laplacian smoothing to handle unseen regions.

Explore how tree augmented naive Bayes extends naive Bayes by allowing limited interdependencies among inputs conditioned on the output, improving performance while adding model complexity.

Explore how linear discriminant analysis uses normal distributions with shared standard deviation and prior probabilities to predict cancellations based on weekly usage.

Finish the Bayesian statistics course and recognize its relevance in practical settings. Share ideas and feedback on the discussion page to extend your learning beyond the course.