Modern Data Analysis Masterclass in Pandas and Python

Code along through 12 tasks to master pandas series fundamentals, including defining series with default and custom indexes, from dictionaries, and applying attributes, methods, sorting, math operations, indexing, and slicing.

Define a Pandas series from a Python list using the Pandas constructor to see the default numeric index. Observe text data as object and numeric series as int64.

Define a Pandas series using the series constructor with a movie list and the default numeric index, then confirm its object data type in this mini challenge solution.

Define a Pandas series with a custom index by passing data and index to the series constructor, illustrated with stock tickers and custom labels.

Define a Pandas series named my series containing your top three movies with custom indices movie one, movie two, and movie three using the series constructor.

Define a Pandas series from a Python dictionary by mapping keys to indices and values to data, then apply a mini challenge with stock prices.

Define a pandas series from a Python dictionary of three stocks—S&P 500, Apple, and Tesla—and their prices, using a custom index and int64 data.

Learn to use pandas series attributes to access values, index, and data type without parentheses. Distinguish attributes from methods and indexers with a practical stock example.

Use the size attribute to reveal the length of a Pandas series. Compare size with shape and see how a series differs from a data frame with rows and columns.

Learn how to use pandas methods with parentheses to transform series data, including sum, product, mean, and head. Create new series subsets and evaluate memory usage.

Explore pandas methods by using head and tail to fetch first and last rows, and measure a series' memory usage with memory_usage to reveal 168 bytes.

Import a one-dimensional CSV as a pandas series with read_csv and squeeze=true, using the S&P 500 prices CSV. Compare with squeeze=false to see the data frame result and type differences.

Import a one-dimensional CSV with pandas read_csv, using squeeze to switch between a series and a data frame. Compare types to understand basic pandas data structures in practice.

Explore pandas built-in functions and Python built-ins, convert CSV data to a pandas series with squeeze, practice length, max, min, and a mini challenge converting positives to negatives and deduplicating.

Define a pandas series with integers, convert values to absolute with abs, remove duplicates with set, and review built-in functions like type, length, max, and min.

Sort pandas series by values and by index, using in place updates. Explore memory-aware sorting of the S&P 500 series with min and max checks.

Sort the S&P 500 values in descending order using sort_values with ascending=false and in_place=true to update the Pandas series in memory, and reference sort_index to revert.

Learn to perform math operations on pandas series, including sum, count, max, and min, and use describe for a statistical summary of the S&P 500 prices.

Learn to compute the average of a pandas series using two methods: the mean and sum divided by count. Validate results with describe and S&P 500 data examples.

Learn to check whether a given element exists in a Pandas Series, by querying values or the index using S&P 500 price data, and understand Pandas' default search behavior.

Explore how to check whether a stock price exists in a Pandas series using the in operator, then round S&P 500 values to the nearest integer to verify again.

Learn zero-based indexing in pandas to access specific elements of a series, including the first, fourth, and last items, using square brackets with the element's index.

Explore pandas series slicing to extract multiple elements using index-based positions. Learn the difference between square brackets for indexing and colon notation for end-exclusive ranges, with practical examples.

Master series slicing in pandas by extracting all elements except the last three using the colon syntax and minus three, as shown in the mini challenge solution.

Recap the pandas series fundamentals: define series with default and custom indices, explore attributes and methods, and perform core operations like sorting, indexing, and slicing.

Define a pandas data frame from a Python dictionary to model bank clients with id, name, net worth, and tenure, then inspect its structure with head, tail, shape, and info.

Define a pandas data frame named portfolio_df with three stock tickers, their prices per share, and shares, then compute the total portfolio value by multiplying and summing.

learn to read csv and html data with pandas, creating and inspecting data frames, and using read_csv and read_html for tabular web data, including multiple tables.

Learn to read tabular retirement data with pandas by loading csv and html tables into a data frame, creating retirement_df with pd.read_html.

learn how to write a pandas data frame to a csv file, compare index true versus false, and explore exporting to html, json, and excel formats.

Export Pandas data frames to csv files with or without index, and explore compression options like zip and gzip to reduce file size.

Read a bank client CSV into a Pandas DataFrame, set and reset the index with a chosen column, and use read_csv with an index column for one-shot indexing.

Load the bank client csv into a pandas DataFrame and display it. Set the last name column as the index in bank_df (inplace), and learn to access rows and columns.

Master selecting single or multiple columns in pandas DataFrame, using dot and bracket notation, and handle spaces in column names while viewing results as Series or DataFrames.

Learn to select columns in pandas by extracting net worth, years with bank, and postal code from a data frame; single columns yield a series, multiple yield a data frame.

Learn to load bank client information from bank_client_information.csv, add a has mortgage binary column, and create a mortgage value column in dollars for clients who have a mortgage.

Explore label-based data selection with pandas .loc, including indexing by last name, sorting, slicing with inclusive label ranges, selecting by list, and random sampling.

Explore label-based element selection with .loc by loading a csv, setting the first name as the index, and using sample to randomly pick two rows, reinforcing inclusive indexing.

Learn integer based indexing with iloc in pandas to select rows and columns by numeric indices, perform slicing, and contrast with label based indexing using loc.

Explore two .iloc() techniques to select the last rows of a data frame, using negative indices and starting from a specific index, and see they produce identical results.

Master pandas broadcasting to update bank net worth and create new computed columns. Practice reading CSV, applying label and integer indexing, and currency conversions.

Learn to sort pandas data frames by a chosen column, use in-place updates, and rank customers by net worth, adding a rank column for easy analysis.

Sort a pandas DataFrame by net worth in memory using sort_values with inplace for mini challenge, explore ascending and descending orders, and preview results, including sort index and rank concepts.

Learn to define Python functions and apply them to pandas dataframes, performing value updates with 1.1x, calculating name length, and solving a double-and-add-100 net worth challenge.

Define a function that doubles a balance and adds 100, then apply it to the net worth column in a Pandas DataFrame and sum the updated totals.

Learn to filter data in pandas dataframes using masks and conditions, combine criteria, handle duplicates, and perform selective queries with between and where.

Practice pandas dataframe filtering with masks to select high net worth records, sum net worth values, and manage duplicates, missing data, and feature engineering with multiple criteria queries.

Learn to clean messy real-world data with pandas by detecting missing values, dropping or imputing them (mean or median), and applying feature engineering to ready data for modeling.

Identify missing values with isnull and fill them with the median monthly rate in a Pandas data frame, illustrating feature engineering.

Convert the business travel column from object to category to reduce memory usage in a pandas data frame. Showcases memory savings from category dtype.

Recap how to create and manipulate pandas data frames, read and export data, index and select columns, apply functions, filter, handle missing values, and optimize memory with category data type.

Explore dataframe concatenation, merging, and appending in pandas by combining two dataframes, using pd.concat and append, controlling the index with ignore_index, and validating with length; complete hands-on mini challenges.

Walks through creating a new data frame with bank client IDs and names, building it from a dictionary, and concatenating it to the master data frame with ignore_index.

Learn to concatenate dataframes with multi indexing using keys to create a multi-level index and use loc to access data by customer group one and two.

Explore multi-indexing in pandas by concatenating three bank data frames into a single master list and accessing the third customer group with a simple lookup.

Use pandas to concatenate bank data with pd.concat, build a salary dataframe, then merge on bank client id with pd.merge to enrich the master dataframe.

Merge new data into the bank data frame using pandas, aligning on bank client ID to add credit card debt and age.

Explore real-world e-commerce sales data by applying pandas multi-indexing and groupby to create a multi-index dataframe and perform advanced multi-indexing operations.

Learn how to import a large e-commerce dataset with pandas, encode text with Unicode escape, convert invoice date to datetime, check for nulls, and identify unique countries.

Explore an e-commerce dataset by extracting the country column, using pandas unique and nunique to reveal 38 unique countries, and setting the stage for the next lesson on group by.

Learn how to group data with pandas groupby, compute average, minimum, and maximum unit prices by country and by invoice date, including multi-index options.

Group data by invoice date to compute the mean, min, and max of unit price. The example shows min 1.65, max 9.95, and mean about 4.8.

Create a multi-index dataframe by reading csv data, setting multiple indices (country and invoice date), and sorting and accessing data using the multi-index structure.

Sort a multi-index dataframe in descending order using sort_index with ascending set to false and inplace set to true, demonstrated on sales_df in the mini challenge.

Master multi-indexing in pandas by creating a two-level index from country and invoice date, accessing index levels by name or position, and renaming index labels in a sample dataset.

Learn to build a multi-index from invoice date and country, rename the index names to date and location, and verify the new order in a pandas data frame.

Master multi-indexing in pandas by setting a two-level index, accessing data with lock and iloc, transposing, and swapping levels to analyze unit price trends.

This mini challenge demonstrates using pandas multi-indexing to filter United Kingdom transactions on a given invoice date, extract unit prices, and compute their average.

Recap the pandas multi-indexing and group by techniques on the e-commerce sales data set. Learn how to read csv files, handle null values, and summarize by country and invoice date.

Explore data visualization with Matplotlib by plotting cryptocurrency prices using line plots. Learn to download data from Yahoo Finance, read CSVs, and create subplots, scatter plots, pie charts, and histograms.

Create a basic line plot using pandas and matplotlib to visualize crypto prices, plotting date on the x-axis and bitcoin price on the y-axis, with labels, title, grid, and legend.

Plot an Ethereum price line in red with a thicker line and a clear title, showing 2018 peaks and movements toward two to three thousand via Yahoo Finance.

Learn to download market data directly from Yahoo Finance using the Yahoo Finance library, specify tickers like BTC-USD and date ranges, load into a Pandas dataframe, and reset the index.

Learn to download crypto and stock prices from Yahoo Finance by listing multiple tickers in a single query, fetching open, high, low, adjusted close, and volume.

Learn to plot multiple crypto prices on a single figure using pandas, plotting BTC and ETH by date, with customized line width, figure size, y label, title, and grid.

Solve mini challenge three by adding Litecoin price data to the plot alongside Bitcoin and Ethereum, noting scale differences. In the next lecture, learn to create subplots for multiple lines.

Plot subplots in Pandas to display crypto price series on separate graphs from an investment data frame, then experiment with subplots set to false to observe outputs.

Create scatter plots in pandas by plotting Bitcoin daily returns against Ethereum returns, using a CSV data source. Learn to customize axes, grid, and figure size to visualize relationships.

Plot bitcoin versus litecoin prices to complete mini challenge five using a simple copy-paste workflow and a litecoin column header. The lecture previews plotting pie charts in the next session.

Plot a crypto portfolio pie chart using pandas and matplotlib, building a data frame with BTC, ETH, LTC, XRP, and ADA to visualize allocations and explode options.

Master the pie chart solution by setting crypto allocations to 60% for Ripple XRP and 10% for others, and use explode to highlight Ripple with pandas data frame and matplotlib.

Master how to create histograms from crypto daily returns, compute the mean and standard deviation, and plot with matplotlib using bins, grid, alpha, and a title showing mu and sigma.

Plot a histogram for Ethereum returns as in the mini challenge, compute its mean and standard deviation, and compare its distribution to Bitcoin and Litecoin, using 30 red bins.

Complete the final breakout room challenge by plotting Apple, S&P 500, and Google prices from the CSV on a single graph and in subplots, with legends in the upper center.

Load stock daily prices from a CSV, plot Apple, S&P 500 and Google with Matplotlib, then create subplots with labels, grid, and a legend to show Covid-era trends.

Recap data visualization with Matplotlib notebook, including single and multi-line plots, subplots, scatter and pie charts (explode). Learn to download Yahoo Finance data with start and end dates via yfinance.

Explore Seaborn scatterplots and count plots to visualize cancer data with hue by target. Use simple code to reveal how mean area and mean smoothness separate cancerous from benign samples.

Explore Seaborn scatter plots and count plots, using hue by target to show mean radius versus mean area; review mini challenge results.

Explore seaborn pair plots and heatmaps to visualize multiple features, select variables, and color by class. Learn to interpret correlations, distributions, and KDE with distplot and annotated heatmaps.

Explore Seaborn pair plots and heatmaps while splitting the cancer data into class zero and class one, and compare mean radius with distplots showing histograms and kernel density estimates.

Cover the basics of the Python date time module and pandas date time handling. Code along with skeleton and solution notebooks to explore the avocado prices dataset and date visualizations.

Explore Python's datetime module to create date and date time objects, inspect year, month, and day, convert to strings, print a calendar, and convert pandas series to datetime.

Use Python's date time module to construct birth date and time with year, month, day, hour, minute, and second, import date time as dt, and convert result to a string.

Master handling dates and times with pandas by creating timestamps, comparing to Python’s datetime, building date time indexes, generating date ranges, and extracting business days for time series.

Learn to use pandas to generate a date range from 2020-01-01 to 2020-04-01 and filter for business days with the B frequency.

Load avocado price data with pandas read_csv, convert the date column to date time, and set it as the index for powerful data analysis.

Explore pandas date time handling with the avocado data, converting the date column to datetime, checking original types, and preparing to group and aggregate by year and month.

Learn date-time analysis in Pandas by setting a date index and using resample, truncate, and offsets to explore avocado prices by year, month, and quarter.

Compute the average avocado price per quarter end by resampling the data with a quarterly rule and mean. Preview upcoming violin plots to visualize price progression across months and quarters.

Learn to visualize data with pandas by resampling avocado prices—monthly, quarterly, and yearly—and plot insights with matplotlib and seaborn using violin, dist, and cat plots.

Plot weekly avocado average prices using pandas resample('W') and mean; visualize trends with a fixed 10x5 plot, then compare organic prices by region with a hue by year.

Recap covers using Python datetime and pandas for date time indexing, resampling, and visualizing avocado prices with Matplotlib, including violin plots, distribution plots, and categorical plots.

Explore pandas and text data by loading, cleaning, and feature engineering with Amazon reviews. Learn binary classification with a feedback column, text processing, and techniques like tokenization and stop words.

Load a 3000-review amazon dataset, inspect ratings, date, variation, and the text column to derive sentiment and prepare clean data for model training.

Explore loading text data and performing basic data exploration with pandas, using describe to summarize ratings, count unique variation classes, and inspect memory usage, while reviewing the mini challenge solution.

Learn to convert a data frame's verified_reviews column to uppercase and lowercase using str.upper and str.lower. Also turn headers to uppercase and apply a title case challenge.

Explore how to convert text data in pandas to title case using the .str.title method on a verified_reviews column, contrasting it with .str.upper and .str.lower transformations.

Explore text data with pandas by measuring string length, creating a reviews_length column, and identifying shortest and longest reviews to reveal customer insights.

Learn how to identify the longest customer review in a text data column using pandas, filter by maximum length, and view the full verified review for analysis.

Learn to replace text in a Pandas DataFrame, filter rows by ends with, starts with, or contains love, and tokenize reviews for sentiment analysis.

Apply a lowercase transformation and use str.contains to filter a pandas dataframe for the word love, displaying matching rows. The next lecture covers text data cleaning by removing punctuation.

Clean text data by removing punctuations using Python and pandas. Use the string module, list comprehension, and a remove_punk function applied to a dataframe column.

Remove stop words and punctuations using nltk and gensim, extend the stop word list, and apply a custom preprocessing pipeline to clean text for sentiment analysis and word cloud visualization.

This mini challenge solution demonstrates adding 'really' to stop words and applying a preprocess with a length filter to keep tokens with three or more characters.

Learn how to tokenize text and pad sequences, convert words to numbers, and use count vectorization to create a feature matrix for text data, including stopword removal and vocabulary building.

Change the index and run the code to verify text tokenization, confirming that the word love appears three times among 4000 unique words as a sanity check.

Visualize text data by tokenizing, computing review length, and plotting distributions with Seaborn, including count plots for feedback and bar plots of variations versus ratings.

Plot the rating distribution with a seaborn count plot using Alexa_df. The chart shows many five-star ratings and fewer low scores.

Create word cloud visualizations from cleaned customer reviews in pandas, joining tokens into a single string and exploring positive and negative ratings.

Develop a pandas workflow to filter negative ratings and prepare text data for a wordcloud visualization. Render a figure to reveal common complaints like screen issues and refurbished devices.

Master practical text data analysis in pandas by loading, cleaning, tokenizing, stopword handling, counting words, and visualizing with histograms and wordclouds for all data and positive/negative subsets.

Master python basics from variables and assignment through data types, operators, loops, and functions. Learn to print, get user input, and work with files, pandas, numpy, and data visualization.

Learn how to perform math operations in Python, including addition, subtraction, multiplication, division, and compound assignments, using stock examples to compute portfolio value and returns.

Master the Python order of operations and precedence using parentheses to control calculations. Explore examples like 3+4*5 vs (3+4)*5, abs(x-y)*(x+y), and basic stock profit calculations.

Master how to use Python's print function to display strings, define strings with quotes, and format output with placeholders to show shares and tickers like Apple Inc and AAPL.

Learn to get user input in Python with the input function, print and format responses, and build simple interactive programs that collect name, age, and net worth.

Learn booleans in Python, representing true or false as constant objects, and use them in conditional statements and comparisons (==) versus assignment (=), with stock price examples.

Learn how Python lists work as ordered, changeable collections, including indexing, nested lists, negative indexing, and slicing, with practical examples and mini challenges.

Learn how Python dictionaries store key value pairs and access values by keys, with a practical client assets example including real estate, stocks, savings, and crypto.

Learn how to define and manipulate strings in Python, including concatenation, case conversion, splitting, and parsing emails to extract first names for practical data tasks.

Explore tuples in Python as immutable sequences defined with parentheses, unlike lists. Learn indexing and slicing, demonstrate immutability via assignment errors, and concatenate tuples while checking their type and length.

Explore sets in python as an unordered collection of unique elements, created with curly braces or set() function, and learn to remove duplicates from lists by converting to a set.

Explore comparison operators, logical operators, and conditional statements using stock price examples, boolean outputs, and string comparisons, with hands-on challenges and code-along practice.

Master logical operators and conditional statements in Python by exploring and, or, boolean truth values, and how comparisons combine to drive code flow.

Explore conditional statements in Python, including if, else, and elif, with indentation and boolean outputs. Learn how to combine comparison and logical operators for input-driven access control and case-insensitive checks.

Solve conditional statements in Python by validating sign of input, checking divisibility by three but not seven, and determining even or odd, with emphasis on full test coverage.

Master for loops, range, while loops, and nested loops in Python, exploring break statements and list comprehension while practicing with lists, tuples, strings, dictionaries, and sets.

Master while loops in Python, including condition-based execution, incrementing i, and printing, and build a rolling dice game with two dice, random numbers, and keep playing prompts.

Master breaking and continuing loops in Python using break and continue, with for and while examples, including running average from user input and exit handling.

Explore nested loops by embedding a slow loop inside a fast loop using range to print a multiplication table and color-phone combinations, and preview list comprehension in the next lecture.

Show how list comprehension replaces for loops, transforms a list in one line, squares elements, and filters even numbers.

Explore the basics of Python built-in functions, learn to define and call functions, and use built-ins like length, min, max, sum, map, filter, and type with lists, tuples, and ranges.

Define and call custom Python functions using def, pass arguments like shares and price, and return values; set default values and compute total account balance.

Apply the map function to lists using named functions and lambda to produce new lists, as shown with summation, squaring, and cubic mapping across ranges.

Master Python data filtering using map and filter with lambda on lists to extract even numbers, values greater than or equal to 100, and ranges 200–250.

Master text file handling in python by opening, reading, writing, appending, and closing in text mode; practice line-by-line reading, splitting into words, and basic csv file operations.

Learn to read csv files with Python and pandas, inspect headers, print the first five rows, and manipulate data by combining names and extracting dollar values.

Explore numpy basics and import numpy to create one- and two-dimensional arrays. Understand shape, length, type, reshaping, and maximum and minimum values, and practice with slicing and indexing.

Explore numpy built-in methods and functions, including rand, randn, randint, arrange, ones, zeros, and identity matrices, to generate random arrays and prepare for Monte Carlo simulations.

Learn to inspect numpy arrays by obtaining shape, length, and type; reshape them into new dimensions, and compute max, min, mean, and arg max and arg min.

Explore numpy mathematical operations by creating two arrays with arange, performing element-wise addition, squaring, square roots, and exponentials, then compute the Euclidean distance using sqrt(x^2 + y^2).

Explore numpy slicing and indexing to access and modify 1d and 2d arrays, select rows and columns, broadcast changes, and manipulate a 5x5 random matrix and its mini matrices.

Explore numpy element selection by filtering a 5x5 matrix to keep elements greater than three. Practice replacing negatives with zero and odd numbers with 25.