Pandas with Python for Data Science

Master the Jupiter environment, including a windows-like interface, kernel widgets, and options, and use command mode, edit mode, and shortcuts to manage cells and run pandas code.

Learn to read data with pandas using read_table and handle separators like tab and pipe. Explore dataset structure by inspecting head and shape, and assign names for Chipotle orders.

Explore the difference between a series and a data frame. Learn to extract columns from a data frame using bracket and dot notation, and rely on intellisense to identify columns.

Practice basic data frame operations in pandas, including string concatenation and type errors with mixed ints and strings. Create a new column by concatenating occupation and gender with proper formatting.

Explore pandas functions for data science by reading csv data with read, specifying separators, and using describe and data types to summarize numerical and string columns, including object types.

Learn how to view and rename dataframe columns in pandas, use data.columns to inspect, assign new names, rename in place, and apply string replacements to handle many columns.

Use drop to remove columns and rows in a pandas data frame, axis 1 for columns and axis 0 for rows, drop columns with Python list, and update in place.

Discover how to sort a pandas data frame by duration or journal, in ascending or descending order, perform multi-key sorts with a Python list, and expect non in-place results.

Learn to filter a data frame by multiple criteria in pandas using and with parentheses and the isin function to select crime, action, and drama genres.

Learn to read a dataset with Pandas by selecting specific columns with usecols and limiting to the first five rows with nrows for a quick data and schema glimpse.

Explore how to iterate through city data and a data frame, print titles and their corresponding indices, and address row-wise data access.

Explains the axis parameter in Pandas and how axis=0 (default) computes the mean for numeric columns across rows, while axis=1 computes across the columns like rating and duration.

Discover how to use pandas string methods with the str accessor to transform dataframe columns, such as converting text to uppercase, replacing characters, and checking for substrings with contains.

Learn how to change a column's data type to float in pandas, either by converting a selected series in place or by specifying dtype during read with a dictionary.

Learn how to change data types in pandas by converting text numbers to numeric values, removing dollar signs, and casting to float to compute mean for data science and ML.

Learn to group data by categorical columns in pandas and compute min, max, mean, and median durations for each category, using groupby and aggregate functions across genres.

Explore pandas series methods like value_counts, normalize, and unique to reveal genre frequencies and top values. Use crosstab to analyze distribution of genres and content ratings.

Learn to plot time series in pandas by visualizing city duration values with matplotlib inline in a notebook, highlighting how to create city-based plots.

Identify and count null values in each column, use sum to convert booleans to ones and zeros, and filter rows to keep non-null data in a Pandas data frame.

Explore how pandas uses an index to identify rows, select data, and align data in data frame. See how a city can become an index and how labels accompany rows.

Explore how to use a pandas column as an index, set the index name to city, reset to a default range, and apply describe to the index output.

Explore how the look method selects data by row and column labels in Pandas. Use lists, slices, and colon syntax to specify rows, columns, and label strings.

Learn to use the IEX function to mix labels and positions for selecting values, understand inclusive versus exclusive indexing, and note that this method is not widely recommended.

Master the in place parameter in pandas drop operations, dropping a column with axis=1 and choosing in place=true for permanent changes or assigning the result back for a temporary change.

Inspect the space occupied by a pandas data frame by examining column types and memory usage, and use memory_usage(deep=True) for real usage of strings and references.

Learn how to reduce data memory usage in pandas by converting high-cardinality columns to categories and using integer codes via a lookup approach, dramatically cutting space.

Learn how converting a country series to a category affects memory in pandas, with 193 unique values and a lookup table. Always check unique counts first.

Create a manual data frame in pandas by defining a dictionary of columns with lists of values, such as city and customer name columns, then extend with more columns.

Use pandas' sample method to perform random sampling on a dataframe. Specify n for the number of rows, or frac for a fraction, and use random_state for reproducibility.

Explore dummy coding to convert categorical variables into numeric values, using mapping and get_dummies, illustrated with the Titanic data and practical column consolidation.

Learn to create dummy variables in one pandas command by passing the data frame to get_dummies, specifying columns, and using drop_first to drop a level for two-category columns like gender.

Create a data frame with duplicates and use the duplicated function to flag subsequent occurrences as duplicates. Drop duplicates with drop_duplicates and choose keep='first' or 'last' to retain specific records.

This lecture demonstrates converting a city daytimes string column to pandas datetime, enabling dt-based operations to extract year, hour, and filter records by date efficiently.

Learn how to avoid the setting with copy warning in pandas by using the look method instead for safe updates, illustrated with a two-step selection and update.

Explore how to avoid the setting-with-copy warning in pandas by explicitly copying data frames with copy, using loc for safe selection, and creating readable, separate copies.

Learn to adjust pandas display options with get_option and set_option for display.max_rows to control how many rows a dataframe shows, and use reset_option to revert to defaults.

Format data in pandas by applying thousands separators with display.float_format, adjust maximum column width, and explore display options and describe to read datasets clearly.

Explore tricks to customize pandas display options, search within methods, and reset settings, then learn three data frame creation methods: from dictionaries, from lists of lists, and from arrays.

Build large data frames by generating columns from lists and ranges, create data frames with specified indices and dictionaries, and concatenate them to align countries and capitals.

Convert a data frame into a datetime column with pd.to_datetime on month and year data, after installing the latest pandas and restarting the notebook.

Discover how to merge two data frames in pandas using the merge method, join on a key column such as state, and explore left, right, inner, and outer joins.

Explore how to merge data frames using pandas, including inner, left, right, and outer joins, handling nulls, and using suffixes and an indicator to track data origin.

Shape a data frame by melting wide data into long format with pandas melt, using id_vars and custom var_name and value_name for temperature and humidity values.

Learn how to fill null values in a data frame by passing a dictionary to specify per-column fill values, such as 'knuckler' for cities and 'no shape' for other columns.

Import a time series dataset with Pandas and explore a daily temperature series. Fill missing values using forward fill, backfill, and limited filling strategies to preserve data realism.

Learn how the interpolate method fills missing values in time series data by using a datetime index and converting the date column to datetime before interpolation.

Master stacking and unstacking in pandas to move data between rows and columns. The lecture uses a dataset with Chennai and Mumbai across milk, cheese, and butter to illustrate levels.

Explore stacking and unstacking in pandas with three levels, moving from level zero to level two to reveal cities, states, and product prices in flexible data views.

Learn how crosstab creates contingency tables to display frequency counts in pandas, using Titanic data to compare survival by class and gender.

Explore crosstab to reveal distributions from repeating columns in a data frame, using Titanic data to uncover insights about survival by sex, class, age, and other factors before modeling.

Explore advanced crosstab options in pandas, including margins for row and column totals, index or columns normalization, and the values parameter with an aggregation function to compute averages.

Pivot pandas data frames by specifying index, columns, and values to reshape city and day data, showing windspeed and precipitation, and select one or both via a Python list.

Explore the pivot_table method to generate a spreadsheet-style pivot table from duplicate weather data. See how aggregation like mean or max and margins reveal daily trends for city and day.

Explore an advanced pivot table example in pandas, using the grouper function to group data by week and apply aggregation for clear, readable management reports.

Replace values in the data frame in place using the replace function, then write the result to a csv file with options to exclude the index and select specific columns.

Write a data frame to an Excel file with to_excel, set sheet name, remove index, place at the third row and fourth column, and write data frames to sheets.

Explore the world of pandas with Python for data science by learning basic operations to analyze datasets, build confidence for future machine learning and visualization with matplotlib.