Python for Data Science & Machine Learning: Zero to Hero

Review the core data science libraries and module structure for this course, noting that Anaconda provides the essential libraries so no additional installation is needed for now.

Learn how to import and alias essential data science libraries in Python, using the import keyword and as aliases to call pandas, numpy, matplotlib, and seaborn in your notebook.

Master Pandas, the foundation of data science, for storing, accessing, cleaning, and processing data. Understand series, one dimensional, and dataframe, two dimensional, data structures.

Explore numpy arrays, the multi-dimensional data structures in data science, and the high-level functions that operate on them; see how pandas data frames relate as 2D arrays.

Explore how pandas and numpy work together to handle data organization and processing, with pandas built on numpy and accessible numpy functions for concise computations.

Matt plot lib enables data visualization with line graphs, bar plots, and scatter plots, and its pipe plot module makes it work like Matlab, with a dedicated data visualization module.

Discover Seaborn as a powerful data visualization library built on top of Matplotlib, extending its functionality with additional graph types and simplified syntax.

Explore numpy arrays, a grid of values that can be multi-dimensional, created from Python lists, with more built-in functionality and greater efficiency in memory and speed than Python lists.

Learn to create NumPy arrays from Python lists and tuples using np.array, build 1d, 2d, and 3d arrays, and verify their dimensions with the .ndim attribute.

Learn to index numpy arrays in Python by using multi-dimensional indexing with two indices for 2d arrays and three indices for 3d arrays, demonstrated in a Jupyter notebook.

Explain how to read numpy array shape with the .shape attribute, returning a tuple where the first value is number of dimensions and the second is elements per dimension.

Explore iterating over numpy arrays with for loops, from 2d arrays using nested loops to 3d arrays with three levels, printing rows and elements in a Jupyter notebook.

Explore numpy's array creation features and learn to quickly generate zero-filled arrays with numpy.zeros. Specify dtype to obtain integer zeros and practice with a ten-element example.

Use numpy's ones() to create an array of ten ones. The resulting array defaults to floats, then cast to an integer array.

Use numpy.full to create a prefilled array by specifying the number of elements and the fill value, as in np.full(10, 5); for floats, pass a float as the second argument.

Learn how to add a scalar to a numpy array with the plus operator, adding to each element, unlike Python lists that raise an error in a 2d array example.

Subtract a scalar from a numpy array using the minus operator, applying the value to every element, and contrast with Python lists that raise an error.

Multiply an array by a scalar using the star operator to apply the scalar to each element; contrast with Python lists, which concatenate when multiplied by a scalar.

Divide an array by a scalar using / for float division and // for integer results, while a Python list cannot be divided and raises an error.

Raise each element of an array to a power using the double star operator, and contrast with a Python list that errors, illustrating element operations and list limitations.

Learn how to transpose a numpy array using dot. The caption shows that there is no built-in function for transposing Python lists and highlights errors when attempting it.

Explore element-wise addition in NumPy by adding two arrays with the plus operator, and note that Python lists concatenate rather than sum element-wise.

Explore element-wise subtraction with the minus operator on arrays, and show that python lists cannot be subtracted, causing an error; subtract matrix one from matrix two to reveal element differences.

Perform element-wise multiplication in Python with the asterix star operator to multiply two arrays. The caption shows that Python lists produce an error.

Demonstrate element-wise division on arrays, using the single slash for floor division and the double slash for integers, and show that dividing two Python lists is not possible.

Demonstrates matrix multiplication of two numpy arrays using the built in maximal function, showing results in a Jupyter notebook and explaining print behavior.

Explore essential NumPy statistical functions for data analysis, including min, max, sum, mean, std, and median, applied to arrays and matrices in Python.

Explore the two main pandas data structures: dataframe and series, noting that a dataframe is two-dimensional with labeled rows and columns, and that labels can be numbers or strings.

Learn how a pandas series is a one-dimensional data structure with a single column, index 0 to 9, and its name and data type.

A dataframe comprises one or more series, with each column representing a series. Learn how to extract a series from a dataframe.

Create a dataframe from a list or two-dimensional list using pandas, automatically generating row indices and optional column labels, and demonstrate custom column names with numpy arrays.

Create a dataframe from a dictionary by using each key as a column and its value list, with raw labels automatically assigned.

Load a csv into python with pandas read_csv to create a dataframe in a Jupyter notebook, store it as df, and display columns name, calories, protein, vitamins, and rating.

Switch from numeric to string indices by using the set_index function to make the name column the index of the dataframe.

Pandas functions typically do not mutate the original dataframe; use inplace to modify permanently, demonstrated by setting inplace to true when calling set_index to make NAM the index.

Use head and tail to inspect a dataframe or series. Head returns the first five rows by default and can fetch more; tail returns the last seven rows.

Use the described function to obtain a statistical summary of the entire dataframe, DF, via the describe method, producing a data frame that presents per-column statistics.

Slice dataframe rows with the bracket operator by specifying a start index and an exclusive end, using raw positions not labels such as 1 to 3.

Use the bracket operator to index one or more dataframe columns by labels; when indexing multiple columns, supply their labels as a list, returning a subset while preserving original dataframe.

Use boolean masks to filter dataframes by passing a list of true and false values to the brackets operator, selecting the third row.

Apply a condition on the calories column to filter a DataFrame, using the bracket operator with a boolean list to return rows where calories are greater than 70.

Learn to filter pandas dataframes by combining conditions with the and and or operators, using brackets or variables to require calories > 70 and protein < 4.

Learn to filter data with loc by indexing rows and selecting columns using labels, slicing ranges, and mixing indexing versus slicing to produce the correct dataframe outputs.

Explore filtering data with iloc by indexing rows and columns by position, using zero-based indices to select single values, slices, and dataframes without relying on labels.

Add a row with the lock method using a label and values matching the dataframe, then delete rows and columns with drop and bracket notation (axis 0 and axis 1).

Learn to sort the values of a data frame by a column using the sort_values function, with numbers sorted numerically and words alphabetically, as shown for the calories column.

Learn how to save a pandas dataframe to a csv file using to_csv, including overwriting or creating new files, optionally omitting the index with index_label=false, and verifying the saved data.

Learn to concatenate two or more data frames, stacking one below the other or joining data side by side, with a practical Jupyter notebook demonstration.

Group by the gender column to split a dataframe, apply mean, sum, max, min, median, count, and std to non-grouping columns, then merge results.

Identify and clean real-world data by applying basic techniques to handle inconsistency, missing values, and duplicated information, preparing it for practical analysis and manipulation.

Improve data quality by applying data cleaning techniques to ensure data is fit for operations, decision making, and planning, enabling better decisions.

Identify anomalies, or outliers, in data by recognizing observations that significantly diverge from the norm. Explore real-world data, detect and address anomalies using practical techniques for accurate results.

Learn to detect anomalies in a data series using the median and absolute differences, apply a threshold, and flag outliers with a Python example.

Detect anomalies using mean minus standard deviation and mean plus standard deviation as bounds, showing how outliers lie outside the range and illustrating the method in a Jupyter notebook.

Apply z-score based anomaly detection to identify outliers by measuring how far values lie from the mean in units of standard deviation, using a threshold of 1.5.

Explore detecting anomalies with the interquartile range by computing Q1, Q2, and Q3 from data, using NumPy percentile brackets, and the 1.5 IQR rule in Python.

Learn how to handle missing values in data cleaning using pandas: remove rows with missing data or fill with mean, median, or mode to preserve information and analyze datasets.

Explore regular expressions in Python to match patterns, locate digits, test string starts, and replace text, using the re module methods like findall, search, and sub for data cleaning tasks.

Scale features in pandas using min max normalization to 0 to 1 and standardize by subtracting the mean and dividing by the standard deviation.

Launch this course section by applying exploratory data analysis techniques to analyze data and gain a clearer understanding for Python-based data science and machine learning.

Learn how to conduct exploratory data analysis with Python to identify patterns and anomalies and summarize the main characteristics of a data set, often using graphs.

Zoom into univariate analysis by examining one variable at a time, contrasting continuous and categorical data to reveal patterns in the data frame with continuous features and a categorical output.

perform univariate analysis of continuous data with scatterplots and strip plots, visualize distributions via histograms and distribution plots, and summarize iris petal lengths with a box plot and five-number summary.

Perform univariate analysis of the categorical Verity column using count plots and a pie chart to show equal proportions across the three categories.

Learn bivariate analysis of two continuous features with scatterplots and correlation. Use seaborn and pandas to plot and compute correlation, confirming low link between fare and edge, with heatmap.

Analyze two categorical variables by plotting survival rates across p class, using group by and sum on survived to compute survived_ratio and visualize with Seabourne dot bar plot.

Perform bivariate analysis of a continuous edge variable and a categorical survived variable using box plots and bar plots in a Jupyter notebook, showing younger Titanic passengers had higher survival.

Identify outliers and anomalies in data using z-scores, remove values with high z-scores (above 1.5), and apply median imputation to replace remaining outliers.

Learn categorical variable transformation by converting the sex category to numbers, using labeling coding and frequency encoding in a dataframe DF within a Jupyter notebook.

This final chapter introduces time series, showing how to analyze them, identify patterns, and develop forecasting models with machine learning using pandas, numpy, matplotlib, and Python's date time tools.

Learn to download stock data from Yahoo Finance using yfinance in a Jupyter notebook, import the library, and inspect a pandas time series dataframe.

Convert string date values in a time series to timestamps with pandas to_datetime, using a format to avoid parser errors. Change the DataFrame index to timestamp and verify the dtype.

Learn to work with time series in Python by treating each column as a time series, converting dates, indexing by date, and handling weekends and timestamp ranges.

Visualize a time series of Apple stock prices over five years using math plot lib in a jupyter notebook, plotting the adjusted close column and labeling the axes.

Learn to visualize data in Python by drawing visuals in a Jupyter notebook using mock plot lib, and explore the different types of graphs it can produce.

Learn to set up matplotlib in Python, use its pyplot module with a MATLAB-like interface, and adopt the common import pattern matplotlib.pyplot as plt to create graphs.

Plot line plots with Matplotlib by passing x and y lists to plt.plot in a Jupyter notebook, and change the line color with a third argument like 'r' for red.

Learn to label and title matplotlib graphs, add x and y axis labels, plot multiple series in one chart, and use legends to distinguish exponential and linear plots.

Plot histograms using the dot hist function with a value list, visualize frequencies on the y-axis, and customize color and bar width with color and width parameters.

Plot bar charts with the dot bar function in my plot lib, distinguishing bars from histograms by treating x values as discrete, and adjust heights and width in a notebook.

Create a pie chart using the dot pie function in the map plot lib, passing a values list, optional labels, and adjust the explored distance and colors.

Plot a scatter plot with x and y data using the dot scatter function, and customize colors with color, c, and cmap color maps.

Switch to a logarithmic y scale to handle data with exponential growth, using the dot y scale function to reveal differences hidden on a linear plot.

Create a polar plot using a mock plot library, mapping distance r and angle theta; generate theta from 0 to 2 pi in 0.01 steps and set r to 2.

Learn to improve x-axis readability by rotating the x ticks using its rotation parameter, such as 90 degrees, to differentiate dense data.

Create multiple subplots in one figure using the subplot function to arrange plots in a row or column with individual titles and labels, and apply tight layout to adjust spacing.

Explore what machine learning is as a form of artificial intelligence, why we need it, how data guides models to predict events, and improve with time using Python libraries.

Explore how machine learning powers diverse applications across health, transportation, finance, and language translation. See it enable cancer detection, self-driving cars, Gmail spam filtering, and personalized product recommendations.

Explore the two broad machine learning categories, supervised and unsupervised learning, defined by how learning is received, with deeper study to follow in upcoming lectures.

Supervised learning trains the algorithm by comparing outputs with labeled data to learn from errors and patterns, labeling unlabeled data and predicting classes like square or triangle.

Uncover patterns in unlabeled data by applying unsupervised learning, as algorithms group inputs by similarity. Leverage data that are abundant relative to labeled data, making unsupervised methods particularly valuable.

Explain how supervised learning uses labeled training data to classify and guide predictions, while unsupervised learning discovers patterns and clusters without labels.

Explore regression as a statistical method to predict a continuous outcome from one or more input variables, covering linear and non linear approaches, including simple and multiple linear regression.

Explore how linear regression uses a scatterplot of house prices versus area to fit lines, minimize squared errors, and select the best fit for price prediction.

Explore line representation with the y = mx + b model to predict price from surface area, estimating the best fitted m and b.

Learn to set up python for home price predictions by importing pandas, numpy, matplotlib, and linear regression, and loading home prices.csv into a pandas dataframe with pd.read_csv.

Visualize data distribution with a scatter plot of area vs price using matplotlib, label axes, customize color and marker, and note points move up and right for linear regression.

Create a linear regression object from the Square Learn library, fit area and price data, predict a 3300 square foot house price, and inspect coefficients and intercept.