Business and Data Analytics in Python

Explore data governance, data quality, dashboards, and predictive analytics to transform data into actionable insights, guiding data-informed decisions and revealing hidden trends.

Explore the art and science of business analytics, balancing intuition with data through descriptive and predictive analytics, using Python and real-world cases to inform decisions.

Explore how business analytics in python addresses big data, defined by volume, velocity, variety, veracity, and value, with thresholds, storage, processing, and data quality considerations.

Execute data science projects using CRISP-DM, starting with business understanding and problem framing, then data collection, ETL, preparation, quality checks, and align with SMEs for measurable outcomes.

Explore the CRISP-DM modeling phase by selecting algorithms and splitting data for training, testing, and validation. Understand supervised, unsupervised, and reinforcement learning, apply feature engineering, and tune hyperparameters toward KPIs.

Compare Crisp-DM and the Microsoft TDSP approach, outlining data acquisition and understanding, environment setup, data wrangling, acceptance testing, and formal customer handoff.

Explore essential Python data science tools and libraries—local notebooks and IDEs, cloud platforms like Google Colab and AWS SageMaker, with pandas, numpy, scikit-learn, and TensorFlow for visualization and modeling.

Discover how daily choices reveal statistical thinking, from descriptive statistics and the mean to distribution, correlation, contingency tables, and learn to use statistics for decision making under uncertainty.

Explore statistics as a branch of mathematics that analyzes and interprets data, and learn descriptive statistics and exploratory data analysis to question assumptions and interpret real-world phenomena.

Explore typical data sets in tabular form, define observations and variables, and learn to describe data with statistics and visualizations like bar charts and histograms.

Identify and classify data types by contrasting quantitative and qualitative variables, then distinguish discrete from continuous, and nominal from ordinal and interval from ratio with practical salary and temperature examples.

Explore difference between probabilities and statistics: probabilities predict future events with rules. Statistics analyzes past data to infer rules, as with an unfair coin and two consecutive flips at 52%.

Evaluate investment claims about bitcoin using descriptive analytics to spot data deception, assess volatility, and demand solid data and analysis before deciding.

Discover how descriptive statistics summarize data and how random variables represent real-world processes, using commuting time as a case study to illustrate sampling, distribution, and probabilities.

The mean, or average, summarizes data by dividing the sum by the count, serving as a measure of central tendency and a baseline for future prediction.

Explore how the mean acts as a balance point and central-tendency measure, with x-bar or mu. Compare Lisa and Avril’s commutes and introduce standard deviation as the spread measure.

Compute the spread around the mean by using deviations, variance, and standard deviation, and apply n-1 (Bessel's correction) to obtain unbiased estimates.

Explore descriptive statistics hands-on in Google Colab, using numpy to define random variables and compute deviations, variance, and standard deviation, with pandas for subsequent frequency distribution.

Load a real data set with pandas and set the id column as the index. Compute descriptive statistics, quantiles, and frequencies, then convert counts to percentages with a lambda function.

Calculate quantiles and percentiles with pandas to analyze median income, top earners, and inequality across family types; compare plots and seaborn visuals, notches, interquartile range, addressing outliers and data limits.

Model Washington DC 311 service calls as a random variable, derive distributions and probabilities, and visualize with geolocation and descriptive statistics including skewness and kurtosis.

Explore calculating probabilities of a day’s call counts using empirical data, normal distribution assumptions, and kernel density estimation, illustrating how data limitations shape results.

Explore geolocation data in Python, using geopandas and folium to create heat maps of Washington, DC service calls, filter by services, and visualize hotspots for recycling.

Explore visualizing salary data with histograms, density functions, and box plots, using pandas converters to handle currency values, compare starting vs mid-career salaries, and assess skewness and kurtosis.

Learn to identify and quantify linear relationships with scatter plots, covariance, and correlation in data such as blood pressure and cholesterol, exploring starting salaries by college type via cross tabulation.

Explore sampling to infer population insights when data are incomplete, and learn how training and testing splits, bias and variance, and bootstrapping underpin model generalization.

Identify the population and frame, extract a meaningful sample using random, stratified, cluster, or convenience sampling, and make inferences while minimizing bias.

Generate and sample data using pandas and numpy, build a synthetic data set, and apply simple random sampling to illustrate reproducibility and sample selection.

Explore stratified random sampling in SQL, comparing equal sample size versus proportional samples across regional strata, and learn how to compute counts and proportions for accurate population representation.

Apply cluster sampling by dividing the population into clusters, randomly selecting clusters, and sampling five elements from each chosen cluster.

Learn how to estimate population parameters from samples by using the sample mean, point estimates, and sampling distribution, with emphasis on simple random sampling and estimation error.

Learn how the sampling distribution of the mean informs parameter estimation, including unbiased estimates and standard error, using a real loan dataset to connect sample means to population values.

Explore bootstrapping to estimate the distribution of the sample mean, assess estimation error, and observe how increasing sample size narrows variance and approaches normality.

Load and organize data in Google Colab, set up runtimes and folders, bin loan amounts into 50,000 intervals, compare population and sample distributions, and visualize with Seaborn histograms.

Explore bootstrapping to generate the sample distribution of the mean and observe how larger samples tighten variance and approach normality, tested with QQ plots and Shapiro-Wilk tests.

Explore the central limit theorem and its role in point estimation: sampling distribution of the mean becomes normal with large samples, enabling accurate inferences for means, standard deviation, and proportions.

Explore how the central limit theorem makes the distribution of sample means approach normal across exponential, uniform, and Poisson populations, as sample sizes grow.

Explore how summing random variables demonstrates the central limit theorem, showing that the distribution of sums becomes normal as sample size grows, just like the means.

Apply the central limit theorem to a binomial problem, using cumulative distribution function for probability. Check np>5 and n(1-p)>5, apply continuity correction, then compute the probability with z-scores using norm.cdf.

Learn how point estimates infer population parameters from samples using the sampling distribution and central limit theorem, and measure estimation error with bias, variance, and efficiency.

Learn to replace single point estimates with confidence intervals that quantify uncertainty, using the central limit theorem, z scores, and appropriate distributions for mean, variance, and proportion.

Practice hands-on calculation and interpretation of confidence intervals using loan data, central limit theorem, z and t distributions, and adjusted world method for proportions.

Identify internal and external data sources, master data formats, and apply cleaning techniques to prepare high quality data for machine learning.

Access sample data sets from scikit-learn, seaborn, and statsmodels for quick practice. Learn three data types: load, fetch, and make synthetic data via loaders that return dictionaries.

Explore Seaborn datasets by loading them into pandas data frames with the load data set method, noting limited attribute descriptions and provenance, including Old Faithful geyser and car crashes.

Explore sample datasets from Python libraries like stat models, seaborn, and scikit learn, loaded into pandas data frames for analysis. Review dataset descriptions, notes, and variable details to guide experiments.

Explore how data formats shape analytics by organizing data into tabular observations and features, covering delimited formats, JSON, SQL, and columnar storage like Apache Arrow and Parquet with pandas.

Explore delimited formats, from comma delimited csv to headless files and arff in Weka, learn string handling with quotes and escaping, and examine json data exchange in python with pandas.

Learn to read comma-delimited CSV files with Pandas read_csv, inspect a dataframe's shape, info, and dtypes, and recognize how missing values and delimiters affect data quality.

Use pandas describe to generate summary statistics for each column, preview with head, project by column name, and load only selected columns from delimited files.

Learn to read other delimited files in Python using pandas, specify the separator as tab, select and drop columns, and use inplace to modify dataframes.

Load a headless data set in pandas by setting header=None, then define semantic column names and, if needed, use the id column as the index.

Master using pandas index for fast retrieval with the log and iloc functions, selecting by label or position, slicing ranges, projecting columns, and understanding index types and the iloc deprecation.

Handle arff format data by fetching a zip from a url, extracting it, reading arff data with a specialized library, and loading the result into a pandas dataframe for analysis.

Practice reading json into pandas data frames, handle line-delimited json, fix id types, access location data, and fetch currency exchange rates via an API.

Learn how to read data from a relational database using SQLite in Python and load the results into a pandas dataframe. Explore reading configuration from property files and SQL queries.

Document data with clear project context, data sources, formats, and licensing details. Describe data collection protocols, access credentials, environment, consent, and governance to ensure reproducibility.

Explore data preprocessing tasks in data engineering, including handling missing values, outliers, normalization, standardization, feature extraction and selection, for numerical and categorical data within the CRISP-DM framework.

Identify missing, noisy, and inconsistent data, and detect outliers. Harmonize features across scales to reduce garbage in, garbage out and improve model reliability.

Learn automatic data cleaning techniques—filling missing values, smoothing noise, handling outliers, normalizing and aggregating data—while shuffling to reduce order bias, except for time series.

Explore feature engineering by applying feature extraction and feature selection to improve data quality, normalize data, handle missing values, encode categoricals, and derive date parts to reduce overfitting.

Learn how to encode categorical variables for machine learning, using ordinal encoding for ordered categories like Likert scales, and one-hot encoding for non-ordinal values, while ensuring consistent labels.

Learn practical techniques for handling missing values in data, including mean or mode imputation, constant values, new category for categorical variables, record removal, and column pruning.

Identify and fix missing values using a Crisp-dm pre-processing approach, converting zeros to NaN to preserve data quality in the Pima Indians diabetes data set.

Imputing missing values with the mean in pandas, this lecture demonstrates using a Na means data frame and compares effects on diastolic blood pressure and skinfold.

Replace missing values with random samples drawn from the distribution for each column in a copied data frame, using a lambda function to preserve distributional properties.

Identify outliers through visual methods such as histograms, scatter plots, and box plots, and understand their impact on models like linear regression, as they may signal emerging trends or anomalies.

Visualize outliers using histograms with binning and scatter plots. Use mean imputation for missing values and assess multivariate and univariate anomalies in the Pima Indian dataset, with domain expert input.

Display a two-panel visualization on a canvas: a kernel density estimate of skinfold values and a box plot showing Q1, median, Q3, and whiskers to flag outliers using the IQR.

Learn to transform numeric data with normalization and standardization to scale variables for machine learning and outlier detection, including min-max normalization, z-score standardization, and decimal scaling.

Explore data transformations to normalize variables with different scales using min max and z-score methods, visualize transformed distributions with overlapping histograms, and understand mean-based imputation and formatting in pandas.

Apply z-score normalization by subtracting mean and dividing by the standard deviation to center data at zero and express values in standard deviation units; note decimal scaling as another transformation.

Explore decimal scaling normalization that expresses values as decimals by dividing by ten to number of digits, and compare original, min-max, z-score, and decimal scaling transformations in a pandas dataframe.

Identify univariate and multivariate outliers using box plots and scatter plots, then apply z-score and IQR methods to set robust bounds and guide data cleaning.

Apply the z-score method to detect outliers beyond three standard deviations in the pima data. Filter, reverse to original values, and sort to prepare for interquartile range analysis.

Apply the interquartile range method to a data frame to identify columnwise outliers, using Q1, Q3, and 1.5 IQR thresholds, then report and review candidates.

Learn to change skewed data toward normal distribution using transformations such as log, square root, inverse square root, and Box-Cox, and assess skewness and normality with QQ plots.

Apply log transformation to highly skewed variables to achieve near normal distribution, assess skewness, and compare transformed and original variables using the Pima Indians diabetes data.

Transform the data with a log transformation and assess normality via a QQ plot comparing the transformed variable to a normal distribution with the same mean and standard deviation.

Transform categorical variables into numerical features using flag variables and careful encoding, avoiding unintended orderings; apply binning and other encodings to improve model performance.

Learn to transform categorical variables into numerical form using one-hot encoding with pandas get_dummies, illustrated on a card data set and including handling missing values and prefixes.

Map categorical to numerical values with a custom mapping using numpy.where and a dictionary, turning sedan into 1 and other styles into 0, and encode num doors likewise.

Learn to convert numerical data into categorical bins using equal width intervals with Pandas cut, creating four categories: low, average, high, and very high, for highway mileage.

Apply equal frequency binning using quantile cut to create intervals with approximately equal frequency. Label bins with interval syntax and control precision to convert numeric data to categorical.

Explore turning a two-value categorical into a single binary variable using dummy variables. Learn to apply a drop-first trick to avoid multicollinearity in analytics.

Explore data imbalance in Python analytics, learn to detect nonuniform class distributions with visualization and a chi-square test, and apply undersampling or oversampling to balance classes for reliable downstream analysis.

Balance unbalanced data by applying undersampling or oversampling with the imbalanced learn library, demonstrated on the car data set using random under sampler and random over sampler.

Learn how oversampling addresses data imbalance by generating duplicate records with a non-majority strategy using the oversample function and a random over sampler, a key pre-processing step.

Regression analysis quantifies how advertising spend relates to sales and other factors, enabling predictions of future trends. It guides marketing, risk management, HR, and financial forecasting through data-driven decision making.

Explore simple linear regression, estimating a target from a single feature using a best-fit line and squared errors, with coefficients b1 and b0.

Apply a simple linear regression to predict cereal ratings from nutritional data, clean data by replacing negatives with zeros, and explore the sugar rating correlation using scikit-learn.

Explore simple linear regression in Python with scikit-learn and statsmodels, training on sugar and rating data, and visualize coefficients, r-squared, and regression line with matplotlib.

Explore how a simple ordinary least squares linear regression predicts cereal nutritional ratings from sugar content using the regression line, regression coefficients, and residual degrees of freedom.

Explain how SSR, SSE, and SST relate to the regression line and data variability; introduce r-squared as the coefficient of determination; discuss typical value ranges across fields.

Explore how a simple linear regression estimates sugar content and ratings, using standard errors, t tests, and 95% confidence intervals to judge coefficient reliability and model quality.

Calculate the sac, the ssr, the sst, and the r square, and analyze standard errors, t statistics, and p-values for regression coefficients using simulated samples.

Calculate the slope and intercept for simple linear regression from sugar and rating means, using standard formulas and the correlation coefficient to reproduce the Statsmodels OLS results.

Explore residuals in linear regression, verify homoscedasticity, independence, and normality, and apply tests such as the white test, breusch-pagan test, durbin-watson, ljung-box, shapiro-wilk, anderson-darling, and qq plots.

Extend from simple to multiple regression by adding predictors like sugar, sodium, and fibers, and compare models using adjusted R square, F statistics, and log likelihood.

learn to build a multiple linear regression model to estimate ratings from sugar and sodium, guided by a correlation matrix, and visualize the three-dimensional hyperplane with a meshgrid plot.

Explore how adjusted R-squared flags meaningless features by countering noise that inflates R-squared while leaving adjusted values stable, and show how a meaningful fiber boosts adjusted R-squared.

Use the f-statistic to test the significance of all coefficients in simple and multiple regression. Compare to the intercept-only model and interpret the p-value to reject null hypothesis.

Assess model fit by comparing full and reduced models using log-likelihood ratio and chi-square tests. The simpler model performs as well, guiding selection via p-value, degrees of freedom, and AIC/BIC.

Compare multiple regression models using log likelihood, likelihood ratio tests, and AIC/BIC to assess fit and penalize complexity, with AICc for small samples.

Apply AIC and BIC for model selection, noting lower scores favor models; ICC highlights small data effects, and the simpler model wins per the log likelihood ratio test.

Compare AIC and BIC to rank models by log likelihood and complexity, and apply multiple regression checks like linearity, normality, independence, homoscedasticity, with adjusted R-squared and F statistics.

Test residual independence and homoskedasticity by plotting residuals against predictions, using the Durbin-Watson result of 2.03 and the Breusch-Pagan test, then apply a log transformation if needed.

Verify regression assumptions by checking normality, independence, and constant variance of residuals with QQ plots, histograms, and Shapiro-Wilk/Anderson-Darling tests, noting small-sample caveats.

Explore econometrics and causal inference to forecast trends and inform pricing, demand, and investment decisions, with time series analysis and Google's causal impact framework.

Explore time series data by decomposing seasonal, trend, and cyclic components for accurate forecasting. Compare additive and multiplicative models and apply techniques like sarima to real-world data.

Learn to forecast milk production by decomposing a time series into a linear trend and seasonal component, detrending the data, and combining them additively for accurate predictions.

Build a milk production forecast in Google Colab using simple linear regression on monthly data, detrend, and apply month-based seasonal means in an additive model for future predictions.

Evaluate time series forecast quality by comparing models with the same metrics (MAD, MRP, or MSE), and use smoothing and ARIMA to address cyclic patterns for short-term forecasts.

Explore hands-on time series forecasting by building a one-year forecast using linear trend and seasonal indexes, then evaluate quality with MAD, MAPE, and MSE to compare models.

Explore time series analysis for stock prices in Python, examining predictability debates and applying techniques to make informed decisions even when data appears random.

Explore autocorrelation in time series, showing how current values relate to past ones. Learn about momentum, mean reversion, stationarity, non-stationary walks, and the limits of linear forecasting in stock prices.

Explore stationarity in time series and learn tests like the Dick-Fuller and ADF, plus Phillips-Perron. Transform non-stationary data with differencing to obtain quasi stationary changes for forecasting stock prices.

Explore stationarity by creating a stationary time series, testing with the augmented Dickey-Fuller test, and examining autocorrelation plots, then apply differencing to a random walk and evaluate a naive forecast.

Explore how the simple moving average (SMA) forecasts prices and signals trades through crossovers, while understanding lag, performance metrics like ROI and Sharpe ratio, and concepts like support and resistance.

Compute 20-day simple moving average on Microsoft stock data, plot price with sma 20, estimate next-day price as 20-day mean, and evaluate mean absolute deviation and mean absolute percentage error.

Explore how the simple moving average acts as a lagging signal from price crossovers and buy-sell signals. See how rolling calculations in pandas reveal signal timing with a 53-period window.

Explore weighted moving averages and the dual simple moving average to improve stock price forecasts, balancing faster signals with whipsaws and crossovers.

Explain how the exponential moving average weights recent prices for quicker trend signals, then introduce the double exponential moving average to reduce lag and aid entry or exit decisions.

Discover the Hull moving average, a four-step indicator blending weighted moving averages and the EMA to balance lag and responsiveness while emphasizing recent prices.

Explore the directional moving index (plus DMI, minus DMI, and ADX) to gauge trend direction and strength, using MACD crossovers for buy and sell signals when ADX exceeds 25.

Explore moving beyond the simple moving average by comparing double simple, exponential, double exponential, and whole moving averages with adx, dmi, and macd to assess crossovers, trends, and signal strength.

Explore causal inference using the potential outcome framework, counterfactuals, and parallel trends, applying difference-in-differences and the Google Causal Impact method to compare treatment and control effects.

Apply causal inference in time series using Google Causal Impact and difference-in-differences to measure event effects on stock prices, building counterfactuals with treatment and control groups.

Identify the intervention date, set pre and post windows, select a control group via correlation with Tesla, and compare using difference-in-differences or the Google Causal Impact framework.

Learn to use the difference-in-difference method for causal inference, comparing treatment and control groups over time, estimating counterfactuals, and applying regression and hypothesis testing.

Explore causal impact analysis using difference-in-differences to estimate the effect of Tesla's Sept 22, 2020 announcement on its stock price, through counterfactuals, pre/post treatment comparisons, and significance testing.

Explore the Google causal impact analysis framework, building counterfactuals with a Bayesian structural time series on pre- and post-treatment data to assess treatment effects while accounting for confounding and uncertainty.

Apply the causal impact analysis framework to analyze stock price reactions to Tesla announcements using pre and post treatment periods, counterfactuals, and p-values. Interpret results with confidence intervals.

Explore customer segmentation by applying RFM analysis to understand behaviors, cluster customers with k-means, and refine groups with Gaussian mixture models for nuanced insights.

Explore segmentation techniques to group customers by behavior and value, and apply RFM analysis and Gaussian mixture models to tailor marketing, product, and retention strategies.

Explore the Pareto principle, an 80/20 heuristic, and learn to segment customers with recency, frequency, and monetary value to prioritize high-value opportunities.

Explore the RFM analysis process—recency, frequency, and monetary values—to segment customers and rank top segments, using additive, multiplicative, or weighted scoring.

Explore the three RFM formulations: additive, multiplicative, and weighted, and learn how recency, frequency, and monetary scores drive customer segmentation with practical scoring strategies and limitations.

Perform an RFM analysis on a UK online retailer dataset, loading Excel data and computing recency, frequency, and monetary to segment customers into champions, loyal, potential, and needs attention.

Explore partitioning k-means clustering to form homogeneous groups by minimizing within-cluster variability and maximizing between-cluster differences, using centroids, distances, and Lloyd's algorithm in an unsupervised setting.

Explore hands-on k-means clustering in Python using scikit-learn, covering data standardization, centroid updates, Voronoi visualization, PCA plotting, and practical notes on elbow and silhouette methods, init, and train-test considerations.

Explore how the k medoids variation uses real data points as cluster centers, replacing centroids to improve interpretability, and learn the criterion that minimizes within-cluster variability.

Compare k medoids to k-means by selecting medoids from the data set to improve explainability, and learn how to install scikit learn extra to run the medoids method.

Learn how to select distance functions for k-means, including Euclidean distance, Manhattan distance, cosine distance, and Hamming distance, and how to convert categorical data to numeric for effective clustering.

Explore how to select a good value of k in k-means using the elbow method and silhouette scores, balancing within-cluster variability and between-cluster variability.

Explore elbow and silhouette methods to determine the number of clusters in a k-means solution using a standardized auto mpg dataset; compare inertia and silhouette scores to justify choosing k=3.

Apply k-means clustering to RFM scores for online retail segmentation, use silhouette scores to select four clusters, and describe segments such as champions, loyal, potential, and need attention.

Examine how data shape affects k-means, why spherical assumptions matter, and when to transform data or switch to methods like Gaussian mixtures for anisotropic or high-dimensional data.

Discover how K-means assumptions of spherical data affect circular and semicircular shapes, and learn to improve clustering by polar transforming the data (and using density methods like DBSCAN).

Explore k-means on spherical data with unequal variance and varying densities to see how density influences centroid allocation and spillover, and how seeds affect stability.

Explore how anisotropic data shapes affect k-means clustering using linear algebra transformations, such as rotation and stretching, applied via matrices. Identify when the data geometry warrants a different clustering approach.

Learn how gaussian mixture models overcome k-means limitations by using multimodal data to segment customers with multiple normal distributions, via an expectation maximization approach and flexible covariance structures.

Explore Gaussian mixture models (GMM) to estimate densities, detect anomalies, and perform behavioral segmentation with soft clustering. Apply model selection via BIC and AIC, and estimate parameters with EM.

Apply a Gaussian mixture model for customer segmentation on a European bank dataset. Pre-process with standardization, one-hot encoding, and Box-Cox transforms to profile customers and guide service strategies.

Apply a gaussian mixture model to segment bank customers, select five components based on AIC/BIC values, profile segments with descriptive statistics, and craft business-focused recommendations for leadership.