Correlations, Association & Hypothesis Testing (with Python)

Compute and interpret covariance as a measure of how numerical variables move together, using deviations from means and sample covariance formula. Note its scale sensitivity, which motivates using correlation.

Explore the correlation coefficient as a standardized measure of the linear association between two variables, defined as covariance divided by the product of standard deviations, reflecting scale invariance.

Illustrate how the correlation coefficient assesses variable associations and varies with data structure. See it approach one or minus one as data align to a line.

Explore two provided datasets, the data simulated system and the insurance charges dataset, and follow along by watching the video or using Spyder to perform the task.

Import pandas and numpy, load the data simulated CSP dataset, compute the correlation matrix for all five columns, and visualize it with an annotated seaborn heatmap.

Assess linear associations between variables with hypothesis testing, using the sample correlation coefficient, t distribution, and p-values to decide on rejecting H0 at a chosen alpha.

Run a Pearson correlation test in Python to obtain the correlation coefficient and p-value, then compare the p-value to the 5% significance level to conclude linear correlation with medical charges.

Download the DTI Insurance dataset and begin Lab 2 tasks, detailing the data file, to practice correlations, associations, and hypothesis testing in Python.

Import the dataset with pandas, then compute correlation coefficients and p-values for age, EMI, and medical charges, interpreting significance at common levels.

Demonstrates using Python's Pandas cross tab to compare income class and gender, and assesses the strength and statistical significance of their association.

Explore the chi-square test for two categorical variables, testing independence with a cross-frequency table, computing expected counts under H0, and using the chi-square statistic to decide.

Test for association between two categorical variables using a chi-square statistic, observed and expected cross frequencies, and a decision rule based on p-values and a significance level.

Begin lab 3 by reviewing the task specification and downloading the data set, then start the lap's tasks using Python to explore correlations and association through hypothesis testing.

Load the deeper income class dataset with pandas, compute a cross frequency table of income class and education, and perform a chi-square test to show a significant association.

Test whether the mean medical charges for males and females differ, using null and alternative hypotheses, a t-based test, and p-values at a chosen alpha level.

Determine when a t test is valid by checking normality and homogeneity of variances. Use SHAPIRO function and valid tests, and consider log transformations if needed.

Perform a t test in python using stats library to compare medical charges by gender, and interpret the t statistic and p value to conclude no evidence of gender impact.

We extend two-category tests to multiple categories using ANOVA and the F-test, testing H0 that all means are equal versus H1 that some differ, based on the p-value.

Download the data and charges extended dataset, review the task specifications for lab 4, and begin the lab using the dataset.

Examine the limits of correlation and association, including that correlation does not imply causation and nonlinear patterns, and see how regression methods improve predictions.