Data Cleaning With Polars

Learn to detect outliers using a box plot, with focus on the total bill column, highlighting lower and upper quartiles and the 25th, 50th (median), and 75th percentiles.

Detect outliers in the total bill column by plotting a histogram with a Polars workflow, converting to pandas for visualization, and noting right-skewness indicates outliers.

Compute the mean and standard deviation to define the upper bound (mean plus three times the standard deviation), create an outliers column, and filter non-normal values.

Learn to detect outliers with the interquartile range in Polars by computing Q1 and Q3, deriving IQR, and flagging outliers in the total bill data.

Learn to detect outliers with the z score method by computing (value minus mean) divided by standard deviation on the total bill column, flagging values above a 2.5 threshold.

Detect outliers with percentile thresholds using polars and numpy, setting 5th and 90th percentile bounds, creating an outliers column and filtering results to refine data.

Learn how binning converts the total bill from continuous to categorical values to reduce outliers, create cut points and labels, and visualize distributions with a bar plot.

Clean the club column in the FIFA data by removing leading newlines with a regex replace all. Apply the change using the idiomatic Polars with_columns workflow.

Clean contract column in a fifa data frame and create contract start and end year columns using polars with_columns, string.split, and alias, handling on loan and free values as null.

Learn to create a contract status column in Polars using with_columns and conditions for on loan or free. Compute contract length from start and end.

Learn how to clean a height column in Polars by converting strings to numeric, removing cm, handling feet values, and unifying into a single column with values in centimeters.

Clean the FIFA data set weight column in Polars by removing kg, extracting digits with a regex, casting to numeric, and computing the mean.

Create a function that encapsulates the long cleaning chain in Polars, converting height and weight to float, normalizing contract columns, and returning a clean FIFA data frame.

Clean the FIFA data frame's weak foot column by removing the star symbol, converting to an integer type, and computing the average rating.

Convert the meal price column from float 64 to integer 16 to reduce memory usage, using numpy to inspect bounds and Polars to cast the data type.

Learn how to extract and create date-based features in Polars, including day, month, weekday, leap year flags, and filtering by leap year to enrich your data frames.

Learn data cleaning with Polars by filtering a data frame with starts with, ends with, and contains on the type of meal column, then count and sort values.

Learn to reshape a data frame with the melt method, transforming 50,000 rows into 100,000 by melting the type of meal and meal price, creating a longer data frame.

Learn to reshape data in polars with pivot, using meal price as values, company ID as index, and meal type as columns to compute mean spend by company.

Learn to clean a microplastics data set by loading it with polars read csv, listing the columns, and selecting ten targeted columns to shrink a 20-column data frame.

Learn to select specific columns by excluding unwanted ones using the plot exclude method in Polars, then use a set difference and df.select to display the remainder.

Count total particles per sample ID in polars using count and over, then filter out blank rows and plot lab id versus total particles after converting to a pandas dataframe.

Group by lab ID, filter for blank rows, and count total blank particles per lab; rename the column to total particles blank and cast to integer 64 for math.

practice joining two Polars data frames with a left join to combine spiked particle counts with blank particle counts on lab ID, preserving left-frame rows.

Learn to compute the count of correct values with polars by creating a 'Total Particles Corrected' column as total particles minus total blank in the count joined data frame.

Learn to replace negative values in the total particles column with zero using a when condition in polars, then visualize lab id versus total particles with a scatter plot.

Replace outliers in the CR column using a 0.995 quantile; visualize with seaborn box plots to identify extremes, then substitute high values with the quantile value.