Data Cleaning & Preprocessing in Python for Machine Learning

Explore a dummy dataset that mimics website demographics, identify missing values and incorrect formats, and learn how to fix issues using python for machine learning and analytics.

Learn how to perform data quality checks in Python by identifying missing values, duplicates, and data type issues, using Pandas and NumPy to inspect a data frame's shape and info.

Inspect unique values with the unique() method to detect anomalies in dataset columns. For example, reveal country inconsistencies like us versus usa, and note 18 unique countries.

Detect missing values to assess data quality by counting missing values per column, tallying rows with missing data, and computing the overall missing percentage relative to the dataset length.

Detect duplicated rules to assess data quality in raw data. Use the duplicated rules method to report any duplicates, ensuring data confidence and preventing downstream costs.

Load the data into a notebook, import pandas and NumPy, read the Website Visitors Excel file, inspect it with head, and replace error values with nulls to clean the dataset.

Learn to handle missing values by imputing with column means using a simple imputer, preserving data rather than dropping rows in data cleaning and preprocessing for machine learning.

Drop all rows with missing values in non-numeric columns using the drop method with in-place modification to keep the dataset clean for machine learning workflows.

Apply the strip method to remove leading and trailing white spaces from text columns like gender and country, ensuring consistent data entries for machine learning preprocessing.

Transform date columns from strings to datetime using pandas to_datetime, after inspecting dtypes with info, and fix inconsistent entries like 15th March 2021 to enable reliable machine learning preprocessing.

Identify and fix data types by converting the age column from float to integer with astype, verify with info, and convert other columns to string as needed.

Detect anomalies in the country column using the unique method, then apply the replace method to convert US to U.S. and prevent duplicate country entries.

Learn to convert categorical values to numeric using a map and a Python dictionary, mapping gender from male and female to 0 and 1 for machine learning models.

Group data by country using the groupby method to compute total sales per country, with options to sum or mean the values for analysis in Python data frames.

Learn to use the applied lambda method in pandas to classify monthly visits as normal or active by applying a function to each element and creating a new label column.

Learn to convert non-numeric columns into numeric features by mapping binary labels to 0 and 1 and applying one-hot encoding with pandas get_dummies for multi-category data.

Learn to detect and remove outliers in Python data cleaning by using z-scores and absolute values on value, monthly visits, and item purchased, while preserving customer ID and gender.

Scale all features to a 0–1 range using a min-max scaler after dropping customer ID and date, then fit and transform the data for machine learning.

Learn to clean and preprocess text for machine learning by tokenizing, removing stop words, and stemming, turning text into numeric features for natural language processing tasks like tweet sentiment.

Clean and preprocess textual data for NLP by loading a corona tweet dataset with pandas, handling encoding, and extracting original tweets and sentiment for NLP tasks using NLTK.

Learn how tokenization converts tweets into tokens for sentiment analysis by breaking text into words and numbers, filtering out special characters, and applying a tokenization function to a data column.

Remove stop words in Python data preprocessing by filtering out english stop words with a stop words removal function and applying it to text data to speed up machine learning.

Learn how stemming reduces word forms to a root using the Porter stemmer, lowering dimensionality in text data by applying a stem_text function to the tweet column with list comprehension.

Learn how to combine tokenization, removal of stop words, and stemming into a single preprocessing function for text data, and apply it across an entire dataset with a lambda function.