Data Science Statistics A-Z : Python

Explore how the pandas package enables data analysis, data manipulation, and visualization, using data frames and cities to prepare data and support machine learning models.

Create pandas series from lists, explore homogeneous data types (integers, booleans, strings as objects), index and slice values, and generate date ranges with pandas date_range.

Learn to create pandas data frames from arrays, random data, and dictionaries using numpy; and load or save data from csv and Excel files.

Learn to set a data frame index from a column, sort by index or by column, and drop a column using in place or axis options to make changes permanent.

Explore how to use iloc for position-based indexing and loc for label-based selection in pandas, including row and column slicing, boolean masks, and combining conditions for data filtering.

Read csv data into a data frame in a Jupyter notebook, convert and analyze it, and save or read back from excel files using read_csv, to_excel, and read_excel.

Create two sample data frames with city and weather-related columns, then merge them on the city key using pandas' merge, exploring left and outer joins.

Explore pivot tables in pandas as an Excel-like way to create aggregate tables. Compute the mean temperature by event to show how temperatures vary across events.

Explore how linear algebra underpins machine learning by using vectors, distances, dot and cross products, and projections to analyze n-dimensional data in Python.

Explore linear independence and dependence among matrix columns, determine rank, and use determinants and inverses to assess and remove redundant features in data analysis for machine learning.

Learn how to represent lines in 2d and planes in 3d as linear equations, and extend to hyperplanes in n dimensions using weight vectors and the X vector.

Learn how inferential statistics use probability theory to draw population conclusions from sample data, illustrated by exit polls and quality checks.

Explore probability theory with coin and dice examples, defining sample space and events, and use random variables like X to quantify outcomes in a bank default scenario.

Construct a probability distribution for the random variable X, the number of red balls drawn, from 75 trials and plot a histogram to illustrate the outcomes 0–4.

Explore how to compute the expected value from a probability distribution using a red balls example, deriving the mean and its implications for game outcomes.

Compute the expected value by weighting outcomes (150 rupees gain, -10 rupees loss) with probabilities 0.133 and 0.867, yielding 11.2, indicating a loss on average; discuss adjusting prizes or penalties.

Discover how to use the binomial distribution to calculate probabilities without experiments, with step-by-step examples of drawing red and blue balls and applying product and combination rules.

The lecture explains using bins to estimate probabilities for continuous data, compute cumulative probability (CDF), and compare probability density functions with histograms and areas under the curve.

Explore hypothesis testing as a tool to infer population parameters from samples, differentiate it from general inferential statistics, and validate claims with real-world examples like food safety thresholds.

Explore hypothesis testing by formulating null and alternative hypotheses using equality, inequality symbols, and terms like at least and at most, illustrated with sales and air conditioners as examples.

Apply the critical value method to test hypotheses about the monthly average units sold, using known population standard deviation and the sample mean to decide on rejecting the null.

Explore hypothesis testing with a one-tailed test, defining null and alternative hypotheses, computing the critical value and z-score, and deciding whether to reject the null at 5 percent significance.

Learn to perform a one-sided hypothesis test for a lead content limit, choosing alpha, computing the critical value, and deciding whether to reject the null or take no action.

Demonstrate transforming raw data into graphs with Matplotlib and Seaborn. Visualize data using line plots, scatter plots, box plots, and histograms, with labels, axes, and subplots.

Explore seaborn to visualize distributions, box plots, pair plots, and heat maps, uncovering correlations, outliers, and multivariate patterns in Python data science workflows.

Learn to perform exploratory data analysis on user data, uncover patterns, and generate actionable business insights as you learn data science basics.

Explore steps to prepare data for analysis, focusing on data sourcing and cleaning, and address issues like missing values encoded as xx or 9 9 9 and city name changes.

Explore univariate analysis on a single column to identify patterns, then examine two or more columns together, and create derived metrics to count outcomes across groups.

Explore banking and telecommunication data use cases, focusing on credit card data and delinquency indicators, where data is sensitive; discuss churn analysis, plan optimization, and promotional decisions.

Explore public data sets from government portals and sector-specific sources, such as agriculture and finance, and learn how to locate, access, and reuse data for research and development.

Identify and fix missing data, standardize units and formats, treat invalid values, and remove duplicates to prepare clean, comparable data for analysis.

Explore univariate analysis of numerical data, reviewing mean, median, mode, and standard deviation, and learn to handle outliers using median, quartiles, and box plots for unit rate analysis.

Perform segmented analysis by grouping data and applying summary statistics such as mean and median, then compare groups using box plots and hypothesis testing.

Learn how machine learning algorithms learn from data to solve real-world problems across industries. See examples from voice assistants, search autocomplete, health care bed planning, and banking churn prediction.

Explore regression, classification, and clustering as core machine learning types, and understand supervised and unsupervised learning with outputs that are continuous or categorical.

Explore how to derive the best fit line by selecting beta0 and beta1 to minimize the residual sum of squares, using errors, predictions, and squared errors.

Explore a simple linear regression case study by comparing actual and predicted values, inspecting residuals, and evaluating performance with mean squared error and R-squared.

Explore a data science project on investment analysis, focusing on data cleaning, exploratory plots, sector and country analysis in english-speaking markets, with no machine learning involved.

Load and verify data, build a clean data frame. Describe missing values, apply inner joins to merge master data, and impute or remove data to retain about 77 percent.

Perform funding, country, and sector analyses for investments between 5 to 15 million, comparing venture, seed, angel, and private equity using median emphasis and outlier visualization in English-speaking countries.