Data Science : Complete Data Science & Machine Learning

Install Anaconda and Spyder, learn what Anaconda is and how to install it on Windows, Mac, or Linux, and launch Spyder from the Navigator.

Learn Python variable types such as integers, floats, strings, and booleans, declare and print them, and master string indexing, slicing with [0:4], and concatenation using the plus operator.

Explore Python loops and iteration concepts. Learn while loops that run while condition is true and for loops that iterate over a sequence, demonstrated with reading records and loan processing.

Explore Python while loops by iterating i from 0 to just before 10, printing i and incrementing by 2, with a boolean flag to safely terminate.

Explore for loops by printing each character of a string and by using range 0 to 10 with step 2, including an else clause.

Learn Python lists, tuples, and dictionaries as sequences defined with square brackets, with zero-based indexing, and used for data processing alongside libraries like pandas.

Initialize a Python list with three elements, access by index, and print the result. Demonstrate adding with append, updating by index, and deleting with del to manage list elements.

Master python list operations by counting elements with len, concatenating lists with plus, and sorting with sort. Prepare for multi-dimensional lists.

Explore two-dimensional lists in Python by creating a list of lists, accessing elements with row and column indices, and printing nested items like names and IDs.

Discover how Python dictionaries store data as key-value pairs and enable access by keys, then learn to create dictionaries and convert data to JSON.

Learn how to create and manipulate a Python dictionary by accessing, updating, adding, and deleting key-value pairs in an address example, including handling case sensitivity and zip codes.

Learn how to define a dictionary, access, add, update, and delete key-value pairs, and use len, str, keys, and values to work with dictionaries for data science in Python.

Open, read, write, and close files in python, using r, w, and a modes with citytemp.csv; compare readline and readlines and observe the file pointer behavior.

Split the record by comma to extract city, temperature, and unit; convert the temperature to Celsius and write the updated record to a file, then close the file.

Learn to read lines from a file using a for loop, reset the pointer with seek, strip newline characters with rstrip, and apply best practices for clean data in Python.

Compute the average temperature per city from a mixed Fahrenheit and Celsius dataset by reading records, converting Celsius to Fahrenheit, summing values, counting records, and outputting per-city averages.

Calculate and print each city's average temperature in Fahrenheit from citytemp.csv by parsing records, converting Celsius to Fahrenheit, and aggregating sums and counts per city.

Demonstrate how exponents represent repeated multiplication, define base and index, cover positive, negative, and zero powers, and introduce logs with examples.

Define polynomials as sums of terms with exponents, and learn to add like terms and multiply polynomials, preparing for factoring in the next lecture.

Explore factoring by identifying factors that multiply to a number or expression, by 5 and 3 for 15, factoring 3x+9 by pulling out 3, and 2x^3-8x into 2x(x^2-4) and (x+2)(x-2).

Explore quadratic equations of the form ax^2 + bx + c. Study symmetry, x-intercepts, and forms like (x + a)^2 and (x - a)^2, plus difference of squares.

Learn how a function processes input to output, with examples like f(x)=x^2 and 1/(x-1), noting domain limits, undefined values at x=1, and the idea of a continuous function.

Compute delta y over delta x from points on y = 2x + 3. Relate the rate of change to speed, derivatives, and limits as x approaches values.

Explore rate of change and limits to perform differentiation and derivatives, derive the first-order derivative, compute the slope at a point, and see how derivatives optimize errors in machine learning.

Explore how first and second derivatives, including the double derivative, reveal critical points and local extrema, using the second derivative test to identify minima, maxima, and link to gradient descent.

Learn to find local and global maxima and minima via the first- and second-order derivatives, using the double-derivative test and concepts like gradient descent and gradient ascent.

Learn how partial derivatives extend single-variable calculus to multivariable functions, use gradient descent to reach minima or maxima, and treat other variables as constants during optimization.

Explore vectors as arrows that encode data features, with direction and magnitude. Learn Cartesian and polar representations, unit vectors like a-hat, and vector operations behind PCA, SVM, and SMOTE.

Explore vector arithmetic by adding and subtracting vectors, and apply cross product to obtain a perpendicular vector with magnitude |V1||V2| sin theta, equal to the parallelogram area.

Explore matrices as the core data structure behind vectors in machine learning, viewing datasets and images as matrices and feature vectors, and apply methods like SMOTE, PCA, and SVM.

Master matrix arithmetic: addition, subtraction, scalar and matrix multiplication, dot product, and division via matrix inverse. Relate datasets and pixels to matrices, noting SMOTE, PCA, and SVM for machine learning.

Explore the identity matrix, determinant, inverse, and transpose, and learn how the determinant enables the inverse and how transposing reverses rows and columns.

Understand how a 2x2 transformation matrix changes a 2x1 vector via matrix multiplication, producing a new vector and enabling scaling, dimensionality reduction, and graphics applications.

Discover eigenvectors and eigenvalues in linear transformations, showing how certain vectors keep their direction while scaling. Apply these ideas to coordinate changes and principal components for dimensionality reduction.

Explore probability concepts such as experiments, outcomes, events, sample space, and conditional probability, using coin toss and dice to illustrate 0-1 bounds and 50% benchmarks in data science.

Define experiments, outcomes, events, sample spaces, and sample points to understand probability. Distinguish an actual outcome from the event and recognize the full sample space for probabilistic reasoning.

Learn conditional probability and its role in machine learning, with intuitive examples and formulas for P(A|B), illustrated by Venn diagrams and real-world classification scenarios.

Explore random processes and random variables, including discrete and continuous types, with coin tosses and two-dice sums, and learn about outcomes, sample space, and probabilities.

Explore why data science and machine learning are essential as data grows to 175 zettabytes by 2025, with unstructured data driving faster decisions and deeper insights across industries.

Explore analytics types: hindsight, insight, and foresight—and how they fit with BI reports, diagnostics, and predictive analytics. Use machine learning with classification, regression, and clustering to inform prescriptive actions.

Understand what data science is, including domain knowledge, mathematics, statistics, and programming, and contrast traditional BI with the predictive analytics and modern data science embedded architecture.

Explore the data science project lifecycle, focusing on the business case and discovery, stakeholder interviews, data availability, budget, and timeline.

Perform exploratory data analysis and visualization to understand distributions and correlations using line charts and histograms, then select an appropriate algorithm—binary, multi-class, regression, anomaly detection, or clustering—and build a model.

Explore the data science project lifecycle from data processing and model planning to training, cross-validation, parameter tuning, selection, deployment, and presenting results to stakeholders.

Develop essential soft skills like domain knowledge, communication, curiosity, and analytical thinking, and master technical skills in mathematics, statistics, data wrangling, machine learning, programming, and data visualization for data science.

Understand data types and variables in data science: qualitative vs quantitative, with nominal and ordinal examples, discrete vs continuous, plus predictors and the target variable, and sample versus population.

Explore the central tendency of data by calculating mean, median, and mode, and observe how outliers affect the mean more than the median, with practical examples.

Explore measure of dispersion by computing variance and standard deviation to quantify data spread around the mean, using squared differences and the 68-95-99.7% rules.

Demonstrate calculating statistics from a Python list using the statistics library (import statistics as st), including mean, median, mode, variance, and standard deviation, and looping over columns.

Explore dispersion measures by defining percentile, range, and the interquartile range (IQR), and learn to identify outliers using quartiles and data spread.

Learn to visualize discrete and continuous data with frequency tables, histograms, bar charts, and box plots. Discover how visualization reveals maximum and minimum values and trends beyond simple algorithms.

Create frequency tables by binning data into ranges and counting records, visualize with a histogram, compare bar charts by plotting values, and note box plots in financial analysis and IoT.

Plot a box plot to visualize data using the interquartile range, with the box showing quartiles and median, whiskers for minimum and maximum, and outliers beyond 5th and 95th percentiles.

Identify essential plot elements such as the title, x and y labels, ticks, and legends through a stock line plot, prepared for plotting with Python and Matplotlib.

Create a two-series line plot with matplotlib to compare stock prices over week days, including axis labels, a title, and a legend.

Explore plotting with Matplotlib and pyplot. Learn to create a plot with x and y lists, add titles and labels, and use pan, home, zoom, and save as png tools.

Learn to plot a bar chart in Python using matplotlib by mapping cities to temperatures, labeling axes, and adding a title for temperature variations.

Load agedata.csv, convert records to integers, and plot a bar-type histogram of ages using matplotlib in Python with bins 0 to 100.

Learn to create a box plot in Python with Matplotlib, reading daily sales data from SalesData.csv, and visualize outliers using showfliers.

Visualize categorical data with bar charts and pie charts to compare loan applications by gender, marital status, dependents, and property area, and identify dominant values.

Create a Python pie chart from a csv by splitting records into city names, counting occurrences with counter, and preparing city names and values lists.

Plot a pie chart with plt.pie, using city values and city names, display percentages with autopct, and visualize city-wise record distribution.

Explore scatter plots to analyze how cost relates to sales, assess linear correlation, identify outliers, and investigate discrepancies.

Create multiple matplotlib figures in one program and save them as png files. Learn to use plt.figure and label figures like my scatter plot and my box plot.

Create a single figure with four subplots in a 2x2 grid using matplotlib, featuring scatter, box, histogram, and line plots, then save and tighten the layout for clarity.

Learn to customize charts by changing marker shape, size, and color; adjust box plot elements, histogram colors, line styles, and bar colors to distinguish multiple datasets and enhance presentations.

Explore customizing a matplotlib box plot with patch artist enabled, configuring box props, whisker props, median line, cap, and outlier markers to create vivid visuals.

Customize plots in matplotlib by adjusting colors, markers, line styles, and bar colors; rotate x-axis labels by 45 degrees with xticks to prevent overlap and save figures like 01custom.png.

Explore inferential statistics by comparing populations and samples. Learn sampling techniques such as simple random sampling and stratified sampling to infer facts about the population from the sample.

Learn how sample bias distorts population inferences, illustrated by the 1936 Literary Digest misprediction, and how data quality, timeliness, and stratification based sampling help produce representative samples.

Explore how to quantify the strength of relationships between variables with the Pearson correlation coefficient and R values, visualize them with scatterplots, and differentiate correlation from causality.

Explore probability distributions for discrete and continuous variables, use histograms to visualize probabilities, and learn how the probability density function and area under the curve define probabilities.

Explore the normal distribution, its bell curve (gaussian), and how the mean and standard deviation shape probabilities, with empirical rules of 68-95-99.7% under the curve and real-world examples.

Learn the standard normal distribution and how to convert data to z-values. Use the z-score and standard normal table to compute probabilities and compare different distributions.

Explore the sampling distribution by drawing multiple samples and computing their means. See how the distribution of sample means forms a histogram and reflects population parameters.

Explore how the central limit theorem makes the means of large random samples follow a normal distribution, enabling inferences about population means and efficient sampling.

Explore how a sample mean serves as a point estimate of the population mean and how the confidence interval, built from the sampling distribution and standard error, quantifies uncertainty.

Define the confidence interval as the sample mean plus or minus the reliability factor times the standard error, derived from the z scores for a given confidence level.

Explore hypothesis and hypothesis testing, including null hypothesis (h0) and alternate hypothesis, with examples like chocolate weight and call center wait times, and relate to statistical significance and feature selection.

Apply hypothesis testing to determine if the call center waiting time decreased, using null and alternate hypotheses, a 0.05 significance level, and a sample mean of 95 from 100 calls.

Import pandas and load csv and tsv data using read_csv, then access data with iloc and column names to create subsets in a data frame.

Explore basic data exploration in pandas by viewing head and shape, inspecting columns, and identifying missing values, then begin preprocessing by replacing missing values.

Identify and handle missing values in a pandas DataFrame using dropna and subset, then impute with mode for categorical and mean for numerical columns.

Learn how label encoding converts categorical variables into numeric values for machine learning, replacing categories like gender and area with 0, 1, or 2 in Python.

Learn to convert categorical data to numeric using pandas by changing columns to category and applying cat.codes, preparing data for label encoding.

Discover why mapping categorical data to numbers introduces unintended ordinal relationships, and learn how hot encoding creates dummy variables for each category to keep features independent.

Apply one-hot encoding with pandas get_dummies to convert category columns into dummy variables, drop the loan_id column, and understand prefixes, drop_first, and how this reduces multicollinearity.

Learn how data normalization brings numerical features to a common range, enabling fair regression and distance-based algorithms, and improving neural network optimization, through standardization (Z transformation), MinMax, and exponential methods.

Learn to standardize numerical features of loan data using the standard scaler (z-transform), extracting applicant income, co-applicant income, and loan amount, with missing values dropped.

Split data into training and test sets to train your algorithm and evaluate learning with unseen data, using cross-validation and folds for feedback.

learn to prepare data in python with pandas: clean missing values, drop loan ID, create dummy variables, select Y and X, then split into train and test sets with sklearn.