Pandas for Data Wrangling: Core Skills for Data Scientists

Explore NumPy, a numerical Python library for high-performance multi-dimensional arrays, and compare it with pandas, showing when to use NumPy arrays versus pandas dataframes in data science.

Explore numpy arrays and how they differ from lists, learn various import styles (import numpy, as np, from numpy import array), and use np.array to create ndarrays.

Compare numpy arrays with lists to illustrate memory efficiency and speed, showing numpy uses less memory and runs faster for data processing.

Explore how numpy arrays compare to Python lists in terms of execution time, using arange to build inputs, zip for pairing, and time.time to benchmark performance.

Compare lists and numpy arrays for numerical operations, then show how numpy enables BMI calculations from height and weight data.

Explore NumPy ndarrays, contrast them with lists, and perform elementwise addition and subsetting. Learn slicing, boolean indexing, and basic operations on 1-D arrays like BMI data.

Create two-dimensional numpy arrays from nested lists, convert them with np.array, and inspect shape (four rows, two columns) and indexing from zero, while noting dtype consistency when decimals are included.

Master numpy array subsetting by selecting specific rows and columns, using indices and ranges, to extract elements, all rows, or all columns in 2d arrays.

Learn to compute descriptive statistics on numpy arrays, including mean and median, and when to use median for outliers. Explore correlation coefficients and standard deviation in 2D arrays.

Build and update numpy arrays by creating a base array x, forming an update array, and computing the total with element-wise addition to practice array updating.

Learn to create NumPy arrays from lists, multiply elements, and update arrays. Concatenate arrays using vstack and hstack, and apply operations like dot, min, max, and ravel.

Learn to use pandas for data wrangling by building data frames and series, handling scalars, dicts, and numpy arrays, and creating a sample student data frame.

Learn to build pandas data frames from series and dictionaries, assign column headers with columns, manage indices, and convert series into data frames for data wrangling tasks.

Learn to build a Pandas data frame from a dictionary, map country names to driving side and CPC scores, and set custom row labels for a clear data view.

Learn how to concatenate data frames with pandas using the concat function, create frames from lists or dictionaries, and ensure matching column headers for seamless data wrangling.

Explore how to join pandas data frames using pd.merge, performing inner, left, and outer joins on name, and handle missing values when combining salary data with employee records.

Learn how to unpivot a dataframe with Pandas melt, converting wide to long format, setting id variables, and naming value and variable columns for clear data wrangling.

Explore dataframe operations in pandas, printing top and bottom rows, inspecting index and columns, checking dtypes and shape, and selecting single or multiple columns with intuitive syntax.

Slice and dice data by rows with iloc and loc, noting ix is deprecated, then apply boolean filtering on age to select a column from your filtered results.

Master advanced filtering in pandas by applying age and gender conditions and selecting columns like gender and occupation. Use is in for doctor or architect and debug dicing errors.

Sort data frames with pandas using sort_values to order by age in ascending or descending order. Learn multi-column sorting by occupation and age, and use inplace for direct modification.

Master descriptive statistics in pandas by using df.describe for numeric and object columns, exploring counts, mean, std, min, max, and per-group averages by occupation.

Learn to remove duplicate rows in Pandas data frames by appending sample rows, testing duplicates with the duplicated method, counting them with sum, and dropping duplicates with drop_duplicates.

learn pandas-based data wrangling in Python by importing numpy and pandas, loading income.csv, inspecting data types and shape, and cleaning data by handling missing values and duplicates before modeling.

Learn to view top and bottom rows with head and tail, inspect unique values with index.unique, and build cross tabs and frequency distributions with value_counts and pd.crosstab.

Access and sample data with pandas using income.sample, including n and fraction options. Learn to select specific columns with loc and iloc for focused analysis.

Learn to rename pandas DataFrame columns, rename specific or all columns with df.rename or df.columns, and use string replace to convert y to year; set a column as index.

Learn to reset the index, drop rows and columns with pandas drop and axis, and sort by state and year, while managing multiple variables and adding new columns.

Explore core pandas techniques for descriptive statistics and data wrangling. Learn to compute year-to-year differences, use describe for numeric and object data, and handle spaces in column names.

Group by functions guide you to aggregate a data frame by index, computing min, max, mean, and count of income across years such as 2004 and 2005.

Learn practical pandas filtering to select rows by index patterns, filter by conditions on income and state, and combine criteria with loc and isin for data wrangling.

Explore how to use Jupyter notebook for data wrangling with missing values, learning to run code, save notebooks, and switch between code and markdown while working with NumPy and pandas.

Learn to create a crops data frame, detect missing values with is null, count them with sum, drop them with dropna, update in place, and discuss imputing missing values.

Learn imputation techniques in pandas to replace missing values with unknown, mean, median, or mode, and apply data frame checks, type changes, and Jupyter notebook workflows.

Create a pandas data frame of names and occupations, then use numpy where to flag self-employed; apply multiple conditions with numpy select to assign color bands.

Install Ubuntu (or run it in VMware) and set up Miniconda to manage NumPy, pandas, and other data-analysis packages on Linux.

Install numpy and pandas with pip or conda, learn basic numpy commands for 2D arrays, and set up a GitHub portfolio while exploring Kaggle's Titanic dataset.

Install numpy and pandas, set up Jupyter Notebook, and learn from Kaggle datasets and kernels, building a GitHub portfolio to showcase data-wrangling skills for data science jobs.

Learn to download a data set and load it into numpy with genfromtxt. Then compare numpy and pandas for reading csv or excel files and creating data frames.

Explore the wine data set from the UCI repository, a multivariate classification task with 178 instances and 13 attributes, and learn loading, checking shape, and inspecting it in numpy.

Perform slicing and dicing in pandas to view specific data points, such as selecting all rows of the first column using colon notation and specifying ranges starting from zero.

Explore pandas for retail data wrangling by reading csv files, filtering and aggregating data, and visualizing sales, inventory, and returns across regions and products.

Display the first and last five products with tail and head, sample ten, sort by returns to identify across region, subsidiary, and stores, then group by region and subsidiary.

Group by region and subsidiary on the retail data frame to compute sums and means for sales, inventory, returns, and total stores.

The lecture demonstrates aggregating retail data by region and region plus subsidiary, computing sums and means for sales, inventory, and returns, and ranking regions by returns.

Leverage Pandas to group by specified variables, compute sums or means across region and subsidiary, and build reusable utility functions that simplify future aggregations.

Sort the retail data frame by returns to identify the top 20 products, reset indices, and compute a tiered discount using conditional np.where based on return thresholds.

Learn to create discount factors with np.where and pd.cut, apply nested conditions, compute adjusted sales, and merge results into the retailer data frame for top 20 products.

Merge the retail data frame with the top 20 products using a left join via pandas' merge, fill missing values with zeros, and analyze adjusted sells after discount.

Learn to import and append India retail records to the main data frame, then analyze the full dataset using a reusable pandas aggregation utility by region and subsidiary.

Explore pandas data analysis techniques with a focus on calculating total sales, mean sales, and standard deviation at regional level, using data frames and grouping by region.

Analyze how to compute and compare standard deviation and coefficient of variation across regions in a retail data frame, using pandas to reveal the most variable regions.

Learn to compute descriptive statistics with pandas using the describe function, inspect mean, standard deviation, min, max, and quartiles, and tailor outputs by dropping columns and transposing results.

Rename variables and round values to two decimals to produce a clear report, then analyze sales, inventory, and returns across 400 samples using n, mean, std, and quartiles.

Analyze hypothesis testing with one-sample t tests in pandas, using SciPy stats to test a population mean against a sample, with null hypotheses and interpretable results.

Learn to perform a one-sample t test in pandas on the sales column, interpret the t and p values, and use a 95% significance level to assess the population mean.

Set an alpha, compute population and sample means, and interpret the t value and p value to decide whether to reject the population mean of 80,000.

Learn how to perform two-sample t tests in pandas for comparing means across regions using hypothesis testing, check variance with an F test, and apply descriptive statistics to business data.

Explore data visualization with pandas, plotting line and bar charts from retail data, and learn to aggregate by region to reveal sales trends across regions.

Explore how to use scatter plots to examine the relationship between sales and returns, identify outliers, and assess correlation, with histograms and box plots for distribution and normality in pandas.

Analyze white wine quality data with numpy, mastering broadcasting, slicing, and concatenation, perform matrix operations, and implement gradient descent on a 100-observation dataset of physiochemical factors.

Analyze a white wine quality dataset by loading a semicolon-delimited file with numpy genfromtxt, inspecting dimensions, and saving and loading arrays as .npy and .npz formats.

Explore pandas data wrangling with slicing and broadcasting: select the first five rows, pick columns by index, and apply vectorized operations across 2d arrays for efficient data analysis.

Split and stack arrays efficiently using NumPy functions like vsplit, hsplit, and array_split to divide arrays into equal or adaptive parts along vertical, horizontal, or specified axes.

Practice indexing and slicing a data array to select the first five rows and specific columns, and apply broadcasting for vectorized arithmetic in pandas data wrangling.

Learn how to sort numpy arrays without breaking row integrity by using arg sort to obtain sort indices and apply them, including multi-column sorts via index tuples.

Apply gradient descent to fit a linear model y = m x + c by iteratively updating slope and intercept using a learning rate and the squared error cost.

Run a gradient descent loop to update and print m and c per iteration, illustrating convergence toward y = 3x + 0 with sample data x and y.

Master numpy linear algebra by computing rank, determinant, trace, eigenvalues, and inverse; solve linear equations, perform least squares, and use matrix operations for data wrangling.