Understanding New Data - Exploratory Analysis in R

Learn to perform initial data analysis by preparing and validating data quality, importing and restructuring oddly shaped data, and applying duplicate removal, missing value imputation, and outlier detection in R.

Learn the succession of data preprocessing steps in R, from importing and tabular restructuring to type classification, duplicate removal, missing value imputation, and outlier handling.

Import tabular data into R using the reader package, skip the x1 column, and classify fields as factors or dates.

Learn how to reshape oddly shaped data in R by using tidyr's gather and spread to convert between long and wide formats, and separate concatenated variables with separate functions.

Apply sampling methods in R base to create representative subsets, addressing sample size and sampling error. Use stratified sampling to preserve population proportions and ensure reproducibility with seed.

Learn stratified sampling in R with slice_sample and group_by to achieve proportional representation across iris species, using seeds and pipe syntax for reproducible, streamlined sampling.

Explore core data types and structures in R, vectors, lists, data frames, and tables, and how class and structure affect analysis and visualization.

Identify and categorize missing values as MCAR, MAR, or MNAR. Apply deletion, hot deck, mean imputation, interpolation, and multiple imputation by chained equations to mitigate bias.

Explore visual tools from the debt package to detect patterns in missing data, using color-coded and greyscale plots to relate missingness to data types and overall missing value percentages.

Learn simple missing-value handling in R: delete with na.omit, and impute with super package tools (na.fill, na.locf, na.approx, na.spline) and na.aggregate for group-wise means.

Explore missing data with the mice package, performing multiple imputation by chained equations using 24 methods for diverse data types, and inspect patterns with the MBD pattern tool.

Assess missing values in profit and sales to decide an appropriate imputation strategy, considering outliers and avoiding mean-based or ordered methods, and apply random forest imputation.

Explore a random forest-based multiple imputation method using mice with three imputations and a fixed seed, then validate data for sales and profit with no new outliers.

Validate numeric variables before outlier detection by ensuring positive sales and profit and that sales exceed profit, then flag validity and remove invalid observations for reliable analysis in R.

Identify outliers and invalid values after imputing missing data, and assess their impact on analysis accuracy. Explore delete, substitute, or preserve strategies, with one-column and multivariable approaches and plausibility checks.

Explore univariate outlier detection with descriptive statistics, six sigma methods, visual tools, and box plots in a car parts dataset, imputing missing values and examining profit and sales.

Learn how the boxplot method detects outliers in a single variable using Q1, Q3, and 1.5 times the intercoastal distance, with whiskers and the out element.

Apply the Six Sigma Ed Rule to identify outliers using mean plus or minus three standard deviations, with attention to normal distribution and imputation.

Explore outlier detection with hypothesis tests in R using Grub's test and the outliers package, including one- and two-sided options, test types, p-values, and practical interpretation of results.

Explore multivariate outlier detection by contrasting model-based and proximity-based approaches. Learn methods that transform data along eigenvectors to account for covariance and high-dimensional data, and compare points to a distribution.

Apply the robust principal component algorithm with the pc out function to detect outliers in numeric car parts data, then assign a binary normal/outlier classifier and visualize the results.

Apply the Box-Cox transformation to right-skewed data using lambda to assess normality and detect outliers. Compare methods after transformation and decide to keep, remove, or impute identified observations.

Explore plausibility checks for non-numeric data using a summary to spot typos and misclassified categories, then reclassify and drop empty levels; adjust dates with the lubricate package year function.

Learn initial data analysis in R by importing, cleaning, and converting data to tables, then handle duplicates, missing values, and outliers with simple and multiple imputation.

Master exploratory data analysis with a reusable blueprint, using data visualizations and chart types to reveal factors influencing price and arrival delays, including time-series summaries and a generalized linear model.

Explore how data visualizations empower exploratory data analysis by enabling interactive, rapid insights, telling the analytical story through charts like box plots and a dashboard.

Explore the ggplot2 diamonds data set in R by inspecting its structure, summary statistics, and potential outliers, duplicates, balance, and missing values for a quick quality check.

Explore numeric variable distributions using non-parametric methods, compare histograms and box plots to reveal skewness, center, and variability without assuming a specific distribution.

Compare numeric variable distributions to theoretical models using the Cucu plot to assess normality and fit. Explore parametric options like lognormal, gamma, and weibull, with data transformations and outlier considerations.

Explore how numeric variables relate using scatterplots to reveal linear trends, correlation, and regression concepts, and understand why correlation does not imply causation in exploratory data analysis.

Explore the nycflights13 flights data with 336k observations and 19 variables, focusing on delays and fragmented date-time fields. Learn when to use correlation and hypothesis tests, plus sampling.

Analyze the dataset summary and fix variable classes and date time components from the 2013 data with 24-hour time, remove incomplete rows, and prepare for the exploratory phase in R.

Explore grouping variables using count summaries to reveal unbalanced carrier groups and evenly distributed origin airports, then visualize with a bar chart using reorder for clear ordering.

Assemble summary tables using group by and summarize to compute flights, total distance, arrival delays in minutes, and arrival delay per 1,000 miles by carrier, then compare punctuality across carriers.

Analyze numeric distributions with histograms and box plots to compare flight distance, air time, and departure and arrival delays, reveal skewness, and guide nonparametric testing.

Explore a year's flight data by merging date and time components into a daily timeline, then analyze daily flights and delays with histograms, box plots, and scatterplots to reveal irregularities.

Explore time-based data in R by visualizing daily flights against the date, and analyze weekly and seasonal patterns with line and bar plots, delays, and derived weekday and week variables.

Examine how to assess linear relationships between numeric variables using correlation coefficients and scatterplots, with Spearman and Pearson comparisons showing near zero distance–delay and strong departure–arrival delay correlations.

Learn to measure the association between departure and arrival delays using the odds ratio with binary delay indicators and relative risk, and interpret the results.

Explore how statistical models in exploratory analysis identify independent and dependent variables, using correlations and regressions to assess relationships, select features, and quantify uncertainty with coefficients and random error.

Explore logistic regression for binary outcomes with grouping variables, using a binomial GLM to model arrival delay. Learn that departure delay and carrier matter, while distance and origin/destination may not.

Investigate the factors behind arrival delays in the flights dataset through exploratory analysis. Highlight carrier and departure delay as influential predictors, and note weather context on extreme days.

Celebrate completing the data science course and apply techniques from data analysis courses in Python and Tablo. Explore the portfolio and instructor profile for latest news, courses, monthly promotions, discounts.