GPT 3.5 & 4: Data Analysis and Machine Learning in Python

Discover how data analysis inspects, cleans, transforms, and models data to uncover insights and support decision making, using descriptive, diagnostic, predictive, prescriptive, and exploratory approaches.

Navigate the complete data analysis workflow from cleaning raw data and handling missing data to hypothesis testing, including exploratory data analysis, outliers, and distribution analysis.

Explore inferential data analysis through hypothesis testing, distinguishing population from sample, and learning to collect, organize, analyze, interpret, and report data to infer effects and draw conclusions.

Explore confidence level as the probability that an estimate contains the true parameter, and use significance level and p-value to decide in hypothesis testing between null and alternative hypotheses.

Execute the complete hypothesis testing workflow: define null and alternative hypotheses, set 0.05 significance, assess normal distribution, apply one-way ANOVA, and interpret p-values to reject or fail to reject.

Explore the core of machine learning as a subset of AI that learns from data to make predictions or decisions, covering supervised and unsupervised models, feature engineering, and future applications.

Navigate the complete machine learning workflow from cleaning data, handling missing data and outliers, through manipulation, scaling, normalizing, encoding features, to model development, evaluation, and deploying the best models.

Open a Jupyter notebook to start learning Python for data analysis, create a beginner environment named Python for beginners, and print hello world as first code using the print function.

Learn to create and use variables in Python, assign data with the assignment operator, and follow naming conventions: characters, no leading numbers, underscores for spaces, case sensitivity, and descriptive names.

Explore Python data types including integers, floats, strings, and booleans, and learn how to inspect and differentiate numeric versus text data using the type function.

Learn how to convert data types in Python for data analysis, using int(), float(), and str() to switch between integers, floats, and strings, and handle type mismatches when combining values.

Explore Python arithmetic operators like addition, subtraction, multiplication, division, modulus, and exponentiation with concrete examples such as 45 + 67 and 2 ** 3.

Explore how Python uses comparison operators such as greater than, less than, greater than or equal, and not equal. Return boolean results and distinguish assignment from comparison using double equals.

Explore how Python logical operators and, or, and not combine conditions to produce boolean results. See practical examples with 45 > 35 and 35 < 45.

Explore lists in Python by creating lists of strings, numbers, and mixed data, learn indexing from zero, slicing ranges, and modifying lists with append, insert, remove, and replace.

Learn about sets and operations in Python, including union, intersection, difference, and how to add or remove elements and multiply values by two.

Explore Python dictionaries, which store key-value pairs as items, and learn to create, modify, and access them using keys(), values(), and items(), including lists as values.

Master Python conditional statements with if, elif, and else to evaluate conditions, print outputs, and control true or false results in data analysis workflows.

Explore integrating logical operators in conditional statements to classify ages (young, adult, senior), use or with temperature, and not with graduation status to determine eligibility.

Master looping structures in Python using for loops to iterate and print list items, and while loops to run on conditions, including building an even-number list and breaking at five.

Define, create, and call a custom Python function using def to compute percentages from total respondents and a category count, demonstrated with male and female examples.

Learn how to load an Excel dataset in a Jupyter notebook with pandas, install and import pandas as pd, use read_excel, and preview the first five rows.

Identify missing values in a dataframe using Python and pandas with ChatGPT. Impute with simple imputer strategies—mean or median for numeric data, most frequent for categorical data—via scikit-learn.

Identify and fix inconsistent data types in a pandas data frame, converting misassigned numeric fields like cost to float and validating dates and categories for accurate analysis.

learn to clean miss-identified data types by removing inconsistent values, filtering with a mask and tilde, and casting to float and datetime in pandas.

Identify and remove duplicate rows in a data frame using the duplicated and drop_duplicates functions, producing a clean data set and viewing the first rows with head.

Learn to sort a dataset by a numeric value in ascending or descending order using pandas sort_values, with examples showing costs across games, books, and other categories.

Learn to filter a dataset in Python by conditions, such as country France and product category games, using multi-criteria filters and not equals to remove values.

Merge extra variables into the cleaned data by loading the refund variable, aligning on the common order_id, and using a match/merge operation to add refund to the dataset.

Learn how to perform data concatenation by rows in pandas using pd.concat, load extra_data.xlsx, and append it to masked_df to form final_df for exploratory data analysis.

Explore feature engineering in Python to create median sales, median cost, and purchase frequency via a pivot table, then identify top ten customers as loyal or regular.

Extract day, month, and year from a date variable in a dataframe using pandas by converting to datetime and using dt.day, dt.month, and dt.year to create features.

Apply feature encoding to map loyal to 1 and regular to 0 in a customer type variable, using Python and pandas.

Create dummy variables to encode categorical features as 1s and 0s for machine learning, using pandas get_dummies, and replace original variables with these indicators in the data frame.

Scale your data for machine learning by normalizing features with a standard scaler, balancing variable magnitudes, separating the target y from features x, and improving model performance.

Split the scaled data into train and test sets to measure model accuracy, using the refund as the target and sklearn's train_test_split with 20% test size and a fixed random_state.

Build and evaluate a linear regression model in Python to predict refund amounts, using train and test data, measure accuracy with mean absolute percentage error, and visualize predictions and residuals.

Develop a decision tree regression model in Python to predict refund amounts, compare against linear regression, and evaluate with MAPE while visualizing predictions and residuals.

Develop a random forest regressor model to predict the refund amount and compare its performance with linear and decision tree regression, achieving a 9.92% mean absolute percentage error.

Compare regression models - linear, decision tree, and random forest - and implement a support vector regressor to predict refunds, using Python and sklearn, evaluated by mean absolute percentage error.