Business and Data Analytics in Python

Explore data governance, data quality, dashboards, and predictive analytics to transform data into actionable insights, guiding data-informed decisions and revealing hidden trends.

Explore the art and science of business analytics, balancing intuition with data through descriptive and predictive analytics, using Python and real-world cases to inform decisions.

Explore how business analytics in python addresses big data, defined by volume, velocity, variety, veracity, and value, with thresholds, storage, processing, and data quality considerations.

Execute data science projects using CRISP-DM, starting with business understanding and problem framing, then data collection, ETL, preparation, quality checks, and align with SMEs for measurable outcomes.

Explore the CRISP-DM modeling phase by selecting algorithms and splitting data for training, testing, and validation. Understand supervised, unsupervised, and reinforcement learning, apply feature engineering, and tune hyperparameters toward KPIs.

Compare Crisp-DM and the Microsoft TDSP approach, outlining data acquisition and understanding, environment setup, data wrangling, acceptance testing, and formal customer handoff.

Explore essential Python data science tools and libraries—local notebooks and IDEs, cloud platforms like Google Colab and AWS SageMaker, with pandas, numpy, scikit-learn, and TensorFlow for visualization and modeling.

Discover how daily choices reveal statistical thinking, from descriptive statistics and the mean to distribution, correlation, contingency tables, and learn to use statistics for decision making under uncertainty.

Explore statistics as a branch of mathematics that analyzes and interprets data, and learn descriptive statistics and exploratory data analysis to question assumptions and interpret real-world phenomena.

Explore typical data sets in tabular form, define observations and variables, and learn to describe data with statistics and visualizations like bar charts and histograms.

Identify and classify data types by contrasting quantitative and qualitative variables, then distinguish discrete from continuous, and nominal from ordinal and interval from ratio with practical salary and temperature examples.

Explore difference between probabilities and statistics: probabilities predict future events with rules. Statistics analyzes past data to infer rules, as with an unfair coin and two consecutive flips at 52%.

The mean, or average, summarizes data by dividing the sum by the count, serving as a measure of central tendency and a baseline for future prediction.

Explore how the mean acts as a balance point and central-tendency measure, with x-bar or mu. Compare Lisa and Avril’s commutes and introduce standard deviation as the spread measure.

Explore descriptive statistics hands-on in Google Colab, using numpy to define random variables and compute deviations, variance, and standard deviation, with pandas for subsequent frequency distribution.

Load a real data set with pandas and set the id column as the index. Compute descriptive statistics, quantiles, and frequencies, then convert counts to percentages with a lambda function.

Calculate quantiles and percentiles with pandas to analyze median income, top earners, and inequality across family types; compare plots and seaborn visuals, notches, interquartile range, addressing outliers and data limits.

Model Washington DC 311 service calls as a random variable, derive distributions and probabilities, and visualize with geolocation and descriptive statistics including skewness and kurtosis.

Explore calculating probabilities of a day’s call counts using empirical data, normal distribution assumptions, and kernel density estimation, illustrating how data limitations shape results.

Explore geolocation data in Python, using geopandas and folium to create heat maps of Washington, DC service calls, filter by services, and visualize hotspots for recycling.

Identify the population and frame, extract a meaningful sample using random, stratified, cluster, or convenience sampling, and make inferences while minimizing bias.

Apply cluster sampling by dividing the population into clusters, randomly selecting clusters, and sampling five elements from each chosen cluster.

Learn how to estimate population parameters from samples by using the sample mean, point estimates, and sampling distribution, with emphasis on simple random sampling and estimation error.

Learn how the sampling distribution of the mean informs parameter estimation, including unbiased estimates and standard error, using a real loan dataset to connect sample means to population values.

Load and organize data in Google Colab, set up runtimes and folders, bin loan amounts into 50,000 intervals, compare population and sample distributions, and visualize with Seaborn histograms.

Explore the central limit theorem and its role in point estimation: sampling distribution of the mean becomes normal with large samples, enabling accurate inferences for means, standard deviation, and proportions.

Explore how summing random variables demonstrates the central limit theorem, showing that the distribution of sums becomes normal as sample size grows, just like the means.

Apply the central limit theorem to a binomial problem, using cumulative distribution function for probability. Check np>5 and n(1-p)>5, apply continuity correction, then compute the probability with z-scores using norm.cdf.

Learn how point estimates infer population parameters from samples using the sampling distribution and central limit theorem, and measure estimation error with bias, variance, and efficiency.

Learn to replace single point estimates with confidence intervals that quantify uncertainty, using the central limit theorem, z scores, and appropriate distributions for mean, variance, and proportion.

Practice hands-on calculation and interpretation of confidence intervals using loan data, central limit theorem, z and t distributions, and adjusted world method for proportions.

Access sample data sets from scikit-learn, seaborn, and statsmodels for quick practice. Learn three data types: load, fetch, and make synthetic data via loaders that return dictionaries.

Explore Seaborn datasets by loading them into pandas data frames with the load data set method, noting limited attribute descriptions and provenance, including Old Faithful geyser and car crashes.

Explore sample datasets from Python libraries like stat models, seaborn, and scikit learn, loaded into pandas data frames for analysis. Review dataset descriptions, notes, and variable details to guide experiments.

Learn to read other delimited files in Python using pandas, specify the separator as tab, select and drop columns, and use inplace to modify dataframes.

Load a headless data set in pandas by setting header=None, then define semantic column names and, if needed, use the id column as the index.

Master using pandas index for fast retrieval with the log and iloc functions, selecting by label or position, slicing ranges, projecting columns, and understanding index types and the iloc deprecation.

Handle arff format data by fetching a zip from a url, extracting it, reading arff data with a specialized library, and loading the result into a pandas dataframe for analysis.

Practice reading json into pandas data frames, handle line-delimited json, fix id types, access location data, and fetch currency exchange rates via an API.

Learn how to read data from a relational database using SQLite in Python and load the results into a pandas dataframe. Explore reading configuration from property files and SQL queries.

Document data with clear project context, data sources, formats, and licensing details. Describe data collection protocols, access credentials, environment, consent, and governance to ensure reproducibility.

Explore data preprocessing tasks in data engineering, including handling missing values, outliers, normalization, standardization, feature extraction and selection, for numerical and categorical data within the CRISP-DM framework.

Identify missing, noisy, and inconsistent data, and detect outliers. Harmonize features across scales to reduce garbage in, garbage out and improve model reliability.

Learn automatic data cleaning techniques—filling missing values, smoothing noise, handling outliers, normalizing and aggregating data—while shuffling to reduce order bias, except for time series.

Explore feature engineering by applying feature extraction and feature selection to improve data quality, normalize data, handle missing values, encode categoricals, and derive date parts to reduce overfitting.

Learn how to encode categorical variables for machine learning, using ordinal encoding for ordered categories like Likert scales, and one-hot encoding for non-ordinal values, while ensuring consistent labels.

Learn practical techniques for handling missing values in data, including mean or mode imputation, constant values, new category for categorical variables, record removal, and column pruning.

Identify and fix missing values using a Crisp-dm pre-processing approach, converting zeros to NaN to preserve data quality in the Pima Indians diabetes data set.

Identify outliers through visual methods such as histograms, scatter plots, and box plots, and understand their impact on models like linear regression, as they may signal emerging trends or anomalies.

Visualize outliers using histograms with binning and scatter plots. Use mean imputation for missing values and assess multivariate and univariate anomalies in the Pima Indian dataset, with domain expert input.

Learn to transform numeric data with normalization and standardization to scale variables for machine learning and outlier detection, including min-max normalization, z-score standardization, and decimal scaling.

Apply z-score normalization by subtracting mean and dividing by the standard deviation to center data at zero and express values in standard deviation units; note decimal scaling as another transformation.

Explore decimal scaling normalization that expresses values as decimals by dividing by ten to number of digits, and compare original, min-max, z-score, and decimal scaling transformations in a pandas dataframe.

Apply the z-score method to detect outliers beyond three standard deviations in the pima data. Filter, reverse to original values, and sort to prepare for interquartile range analysis.

Apply the interquartile range method to a data frame to identify columnwise outliers, using Q1, Q3, and 1.5 IQR thresholds, then report and review candidates.

Learn to change skewed data toward normal distribution using transformations such as log, square root, inverse square root, and Box-Cox, and assess skewness and normality with QQ plots.

Apply log transformation to highly skewed variables to achieve near normal distribution, assess skewness, and compare transformed and original variables using the Pima Indians diabetes data.

Transform categorical variables into numerical features using flag variables and careful encoding, avoiding unintended orderings; apply binning and other encodings to improve model performance.

Learn to convert numerical data into categorical bins using equal width intervals with Pandas cut, creating four categories: low, average, high, and very high, for highway mileage.

Apply equal frequency binning using quantile cut to create intervals with approximately equal frequency. Label bins with interval syntax and control precision to convert numeric data to categorical.

Explore turning a two-value categorical into a single binary variable using dummy variables. Learn to apply a drop-first trick to avoid multicollinearity in analytics.

Balance unbalanced data by applying undersampling or oversampling with the imbalanced learn library, demonstrated on the car data set using random under sampler and random over sampler.

Learn how oversampling addresses data imbalance by generating duplicate records with a non-majority strategy using the oversample function and a random over sampler, a key pre-processing step.

Apply a simple linear regression to predict cereal ratings from nutritional data, clean data by replacing negatives with zeros, and explore the sugar rating correlation using scikit-learn.

Explore simple linear regression in Python with scikit-learn and statsmodels, training on sugar and rating data, and visualize coefficients, r-squared, and regression line with matplotlib.

Explain how SSR, SSE, and SST relate to the regression line and data variability; introduce r-squared as the coefficient of determination; discuss typical value ranges across fields.

Explore how a simple linear regression estimates sugar content and ratings, using standard errors, t tests, and 95% confidence intervals to judge coefficient reliability and model quality.

Calculate the slope and intercept for simple linear regression from sugar and rating means, using standard formulas and the correlation coefficient to reproduce the Statsmodels OLS results.

Explore residuals in linear regression, verify homoscedasticity, independence, and normality, and apply tests such as the white test, breusch-pagan test, durbin-watson, ljung-box, shapiro-wilk, anderson-darling, and qq plots.

Extend from simple to multiple regression by adding predictors like sugar, sodium, and fibers, and compare models using adjusted R square, F statistics, and log likelihood.

Explore how adjusted R-squared flags meaningless features by countering noise that inflates R-squared while leaving adjusted values stable, and show how a meaningful fiber boosts adjusted R-squared.

Use the f-statistic to test the significance of all coefficients in simple and multiple regression. Compare to the intercept-only model and interpret the p-value to reject null hypothesis.

Assess model fit by comparing full and reduced models using log-likelihood ratio and chi-square tests. The simpler model performs as well, guiding selection via p-value, degrees of freedom, and AIC/BIC.

Compare AIC and BIC to rank models by log likelihood and complexity, and apply multiple regression checks like linearity, normality, independence, homoscedasticity, with adjusted R-squared and F statistics.

Test residual independence and homoskedasticity by plotting residuals against predictions, using the Durbin-Watson result of 2.03 and the Breusch-Pagan test, then apply a log transformation if needed.

Explore time series data by decomposing seasonal, trend, and cyclic components for accurate forecasting. Compare additive and multiplicative models and apply techniques like sarima to real-world data.

Build a milk production forecast in Google Colab using simple linear regression on monthly data, detrend, and apply month-based seasonal means in an additive model for future predictions.

Evaluate time series forecast quality by comparing models with the same metrics (MAD, MRP, or MSE), and use smoothing and ARIMA to address cyclic patterns for short-term forecasts.

Explore hands-on time series forecasting by building a one-year forecast using linear trend and seasonal indexes, then evaluate quality with MAD, MAPE, and MSE to compare models.

Explore time series analysis for stock prices in Python, examining predictability debates and applying techniques to make informed decisions even when data appears random.

Explore autocorrelation in time series, showing how current values relate to past ones. Learn about momentum, mean reversion, stationarity, non-stationary walks, and the limits of linear forecasting in stock prices.

Explore stationarity in time series and learn tests like the Dick-Fuller and ADF, plus Phillips-Perron. Transform non-stationary data with differencing to obtain quasi stationary changes for forecasting stock prices.

Explore stationarity by creating a stationary time series, testing with the augmented Dickey-Fuller test, and examining autocorrelation plots, then apply differencing to a random walk and evaluate a naive forecast.

Explore how the simple moving average (SMA) forecasts prices and signals trades through crossovers, while understanding lag, performance metrics like ROI and Sharpe ratio, and concepts like support and resistance.

Compute 20-day simple moving average on Microsoft stock data, plot price with sma 20, estimate next-day price as 20-day mean, and evaluate mean absolute deviation and mean absolute percentage error.

Explore how the simple moving average acts as a lagging signal from price crossovers and buy-sell signals. See how rolling calculations in pandas reveal signal timing with a 53-period window.

Discover the Hull moving average, a four-step indicator blending weighted moving averages and the EMA to balance lag and responsiveness while emphasizing recent prices.

Explore the directional moving index (plus DMI, minus DMI, and ADX) to gauge trend direction and strength, using MACD crossovers for buy and sell signals when ADX exceeds 25.

Explore causal inference using the potential outcome framework, counterfactuals, and parallel trends, applying difference-in-differences and the Google Causal Impact method to compare treatment and control effects.

Apply causal inference in time series using Google Causal Impact and difference-in-differences to measure event effects on stock prices, building counterfactuals with treatment and control groups.

Identify the intervention date, set pre and post windows, select a control group via correlation with Tesla, and compare using difference-in-differences or the Google Causal Impact framework.

Learn to use the difference-in-difference method for causal inference, comparing treatment and control groups over time, estimating counterfactuals, and applying regression and hypothesis testing.

Explore the Google causal impact analysis framework, building counterfactuals with a Bayesian structural time series on pre- and post-treatment data to assess treatment effects while accounting for confounding and uncertainty.

Apply the causal impact analysis framework to analyze stock price reactions to Tesla announcements using pre and post treatment periods, counterfactuals, and p-values. Interpret results with confidence intervals.

Explore customer segmentation by applying RFM analysis to understand behaviors, cluster customers with k-means, and refine groups with Gaussian mixture models for nuanced insights.

Explore segmentation techniques to group customers by behavior and value, and apply RFM analysis and Gaussian mixture models to tailor marketing, product, and retention strategies.

Explore the Pareto principle, an 80/20 heuristic, and learn to segment customers with recency, frequency, and monetary value to prioritize high-value opportunities.

Explore the RFM analysis process—recency, frequency, and monetary values—to segment customers and rank top segments, using additive, multiplicative, or weighted scoring.

Explore partitioning k-means clustering to form homogeneous groups by minimizing within-cluster variability and maximizing between-cluster differences, using centroids, distances, and Lloyd's algorithm in an unsupervised setting.

Explore how the k medoids variation uses real data points as cluster centers, replacing centroids to improve interpretability, and learn the criterion that minimizes within-cluster variability.

Compare k medoids to k-means by selecting medoids from the data set to improve explainability, and learn how to install scikit learn extra to run the medoids method.

Learn how to select distance functions for k-means, including Euclidean distance, Manhattan distance, cosine distance, and Hamming distance, and how to convert categorical data to numeric for effective clustering.

Explore how to select a good value of k in k-means using the elbow method and silhouette scores, balancing within-cluster variability and between-cluster variability.

Explore elbow and silhouette methods to determine the number of clusters in a k-means solution using a standardized auto mpg dataset; compare inertia and silhouette scores to justify choosing k=3.

Apply k-means clustering to RFM scores for online retail segmentation, use silhouette scores to select four clusters, and describe segments such as champions, loyal, potential, and need attention.

Examine how data shape affects k-means, why spherical assumptions matter, and when to transform data or switch to methods like Gaussian mixtures for anisotropic or high-dimensional data.

Discover how K-means assumptions of spherical data affect circular and semicircular shapes, and learn to improve clustering by polar transforming the data (and using density methods like DBSCAN).

Explore k-means on spherical data with unequal variance and varying densities to see how density influences centroid allocation and spillover, and how seeds affect stability.

Learn how gaussian mixture models overcome k-means limitations by using multimodal data to segment customers with multiple normal distributions, via an expectation maximization approach and flexible covariance structures.

Apply a gaussian mixture model to segment bank customers, select five components based on AIC/BIC values, profile segments with descriptive statistics, and craft business-focused recommendations for leadership.