Data Science with Julia: Data Analysis & Visualization

Develop foundational skills in constructing and manipulating data frames with the Julia DataFrames package, using the DataFrame function, dot notation, symbol and string column access, and mutating operations.

Learn to index and slice data frames with the DataFrames package, using iris data to extract subsets, select columns by name, and reorder columns with Cols and All.

Delve into conditional filtering in Julia by applying element-wise conditions with the broadcasting operator on iris data, combining predicates, and using subset or filter with missing value handling.

Select and transform columns in a data frame using mutating and non-mutating forms, including renaming, computing new columns, and applying ByRow, All, and AsTable functions.

Select and transform columns in a Julia data frame using transform versus select, create new columns for min, mean, and max, and handle missing values with skipmissing, describe, and combine.

Learn how to split a data frame by origin using groupby and GroupedDataFrame, apply mean calculations to arrival and departure delays, and combine results with AsTable in Julia.

Explore how to join data frames in Julia using inner, left, right, outer, semi, anti, and cross joins, with practical library database examples and key-column handling.

Explore importing and exporting data in formats from txt and csv to excel, xml, json, Stata, SAS, SPSS, and hdf5, using Julia libraries and custom functions for data workflows.

Explore flat files, including text, delimited, and spreadsheet formats, and learn core Julia file operations, reading, writing, appending, and managing folders and paths.

Learn to read and save delimited files in Julia using the CSV package, handle comma or semicolon delimiters, and export the first 12 columns of a data frame to CSV.

Master reading and writing excel files in Julia using XLSX packages, including readxlsx and openxlsx lazy loading, converting sheets to data frames, and saving multiple sheets with writetable.

Learn how hdf5, an open source hierarchical data format, manages large, complex data with groups and datasets, and use Julia's hdf5 package to create, save, and read arrays and attributes.

Json explains the JavaScript object notation used in web apps and APIs, and demonstrates importing, parsing to dictionaries, and converting to a data frame.

Explore Extensible Markup Language (XML), a human- and machine-readable format for data storage and transfer, learn to create, manipulate, and read XML documents with the EzXML package in Julia.

Explore relational databases by building and querying a Northwind sample in Microsoft SQL Server Management Studio, using ODBC with Julia to connect, retrieve, and convert results to data frames.

Learn to import SAS, SPSS, and Stata data into Julia using StatFiles package. Use the load function to obtain a StatFile, then convert to a DataFrame with the pipe operator.

Explore web scraping to automate extracting data from websites by parsing html with xpath, fetching pages via http, and building structured data from headers, lists, and movie figures.

Import the input and output data files as dataframes, remove duplicates, convert non-numerical data to numerical form, and summarize, handle missing values, detect outliers, standardize, and analyze correlations.

Analyze a hypothetical credit dataset to understand default probability by collecting, cleaning, and preparing input features like credit limit and overdue payments, using credit_input_data.csv and credit_output_data.csv, with binary default indicators.

Import input and output data from csv files, inspect data frames for shapes, names, and types, and validate date columns with dateformat while preparing final datasets.

Import and clean data in Julia by removing duplicates. Apply duplicates removal using custID and appDate for input, and default date with custID for output, to reduce bias and cost.

Merge input and output data to flag defaults within one year of each application date using customer IDs, and implement the process in Julia.

Describe function to get statistics for numerical and categorical data, build frequency tables with groupby and combine, and replace future dates with missing to compute group-wise averages and default rates.

Learn how to handle non-numerical data by converting categorical variables to numerical formats via one-hot and label encoding, so statistical and machine learning models can use these features.

Explore methods to manage missing data in Julia, including removing rows or columns, imputing with mean or median, interpolation, and simple predictive models, while noting when to keep missing values.

Explore how outliers influence central tendency and spread, and learn univariate outlier detection and treatment using histograms, box plots, iqr thresholds, and context-driven decisions.

Standardize and scale data using log normalization for skewness and z-scores; handle zeros with log(x+1), apply min-max scaling to 0-1, and cap outliers beyond three std deviations with sine-based transformation.

Explore correlation analysis as a practical method for variable selection, using Pearson correlation to measure linear relationships, identify redundant features, and filter inputs by thresholds before modeling.

Convert a numeric input into a nominal category by binning into adjustable bins, compute and plot default rates per bin, and assess monotonic relations to the target variable.

Learn to tell data stories with Julia visuals, from single plots to dashboards, using World Bank data with 51 indicators for 217 countries (1960–2022).

Download data from databank.worldbank.org and export to Excel, clean notes, read into Julia as a data frame, handle missing values, use stack and unstack to convert wide and long formats.

Master line plots to visualize how a continuous variable changes over time, customize with titles and axis labels, and compare multiple countries by plotting separate lines.

Learn to create scatter plots to explore relationships between GDP per capita and life expectancy, customize marker size and transparency, and distinguish groups by color.

Use bar plots to compare mean population growth across countries from 2000 to 2022. Filter with subset, compute means, sort descending, and render the bar plot with the bar function.

Explore how histograms reveal distribution by binning a continuous variable. Use the plots package for auto or manual bins with Freedman-Diaconis or Scott, and compare frequency, probability, and density.

Compare distributions with box, dot, and violin plots to visualize summary statistics, central tendency, and outliers across GDP growth data for selected countries.

Explore three dimensional plots, including heatmaps and surface plots, to visualize relationships with inflation data across years and countries, using filtering and unstacking.

Explore quick interactive visualizations in Julia using StatsPlots with Interact and Blink to create an interactive data viewer for small data sets.

Explore building dashboards with Makie, a collection of backends like CairoMakie and GLMakie, by creating figures, axes, and plots, with mutating additions and optional direct scatter for single plots.

Learn to create dashboards in Julia using Makie grid layouts to arrange scatter, density, bar, and line plots of macroeconomic variables across multiple countries, highlighting storytelling through data.

Explore observables in Julia, where variables update automatically through on blocks, lift functions, and the lift macro, enabling reactive updates and chained changes for interactive dashboards.

Create interactive dashboards in Julia with GLMakie, using observables and menus to select GDP per capita and life expectancy, powering linked scatter, density, bar, and line plots.