Machine Learning with Javascript

Explore how the k-nearest neighbor works using a thought experiment with drop positions. Compute distances, sort observations by closeness, and predict the most common bucket.

Learn lodash basics in javascript: review sort by, map, and chain, explore the lodash documentation, and see concise examples that support building k-nearest neighbor data operations.

Apply a k-nearest neighbor algorithm using Lodash chain, map, sortBy, and slice to compute distances to a 300-pixel prediction point and select top K buckets.

Finish the K nearest neighbor implementation by counting the top K labels with lodash countBy, then use twoPairs, sortBy, last, and parseInt to select the most frequent bucket.

Move the K nearest neighbor algorithm into the Plinko project, refine the distance function, run analysis, predict a bucket for a point, and validate results with simulated drops.

Explore interpreting bad results of a k-nearest neighbors model by adjusting parameters, adding features, and automating tests to improve accuracy and understand data correlations.

Assess the accuracy of the k nearest neighbor model by splitting data into training and test sets and evaluating predictions across many points.

Learn how to shuffle a data set and split it into test and training sets using slice, ensuring a randomized split that reflects both sides of the data.

Generalize knn by creating a reusable k and n function using a training dataset, a configurable prediction point, and a distance function to measure test-set accuracy.

Split the data into test and training sets, then use the k n algorithm to predict each test point and evaluate overall accuracy with a consolidated metric.

Compute and print model accuracy by tallying correct predictions from the test set, divide by test set size, compare to the baseline of 10%, and experiment with varying k values.

Refactor accuracy reporting by replacing an archaic loop with lodash chain, filter, size, and divide to compute accuracy as the ratio of correct predictions in the test set.

Investigate optimal k values by looping k from 10 to 15 and testing accuracy for each, then select a practical k around ten based on dataset size and spacing.

Update the KNN algorithm to incorporate multiple features such as drop position, ball bounciness, and ball size by extending the distance calculation with a multi-dimensional distance using the Pythagorean theorem.

Learn to compute multi-feature distances in k-nearest neighbors using the Pythagorean theorem, expanding from two to three features like drop position, ball bounciness, and ball size, with JavaScript examples.

Refactor the distance function to handle any number of features in n-dimension data, using the pythagorean theorem, lodash zip, and destructuring while excluding the label.

Refactor the K and N algorithm to support arbitrary feature spaces by separating features and label with lodash initial and last, avoiding array mutation, preparing data for training and testing.

Refactor the k-nearest neighbor analysis for multiple features and higher dimensions, noting slower performance. Show that the bounciness feature contributes little to distances due to axis scaling.

Apply normalization to scale each feature between 0 and 1 using min-max, one feature at a time. Show that k-nearest neighbor outputs stay unchanged by this scaling.

Develop a min-max normalization function in JavaScript, handling multiple features via a feature count, while cloning data to avoid mutation and normalizing only the first few columns, not the label.

Show a min-max normalization function in JavaScript with a browser-based data demo. Normalize the first feature, keep labels unchanged, and wrap normalization before splitting the dataset of three features.

Explore feature selection in k-nearest neighbors by comparing drop position and ball bounciness, improving accuracy from near 8% to 30% when using the right features.

Evaluate features one at a time with a nearest neighbor model (K=10). Iterate over drop position, bounciness, and ball size, map data, and apply min-max normalization.

Evaluate how drop position, ball bounciness, and ball size affect accuracy in a ball-drop analysis, with k=10, noting drop position yields 32% accuracy while bounciness and size stay near 10%.

We review fundamentals like features, labels, train/test splits, and normalization from the k nearest neighbor project, then pivot to TensorFlow.js for fast numeric calculations and a Lodash transition.

Preview how to use the TensorFlow library (and TensorFlow.js), outline fundamental concepts, and practice with small exercises before re-implementing k-nearest neighbor to demonstrate dataset splitting and test set.

Explore TensorFlow JS fundamentals by understanding tensors, their dimensions, and shapes, including 1D and 2D tensors. Learn the dot length trick to determine shape and distinguish low-level from higher-level APIs.

Create and manipulate tensors with TensorFlow's tf.tensor, perform elementwise operations such as add, subtract, multiply, and divide, and understand how tensor shape governs these operations.

Learn how broadcasting lets two tensors with different shapes participate in element-wise operations by matching shapes from right to left, using ones as needed, with practical examples.

Learn to inspect tensor contents in TensorFlow by using the data.print method to display the data inside a tensor, not console.log.

Access tensor data using the git method for 1D and 2D tensors, following row and column order. Remember that the print method is for debugging, and tensors are immutable.

Learn to extract the center column of a tensor using the data.slice method with a start index, a size, and negative one to include all rows, contrasting with lodash approaches.

Explore tensor concatenation in TensorFlow with the concat method, understand axis 0 versus axis 1, and see how shapes change for k-nearest neighbor applications.

Demonstrate summing each athlete’s three jump distances along axis one, then concatenate the result with athlete ids and heights, and handle a rank mismatch during tensor concatenation.

Fix tensor dimension mismatches when concatenating jump data with player data by using the keep dimension option or expand dims to align shapes along the correct axis for TensorFlow.

Apply a modified k nearest neighbors regression to predict house prices from location and features by computing distances, selecting the top k neighbors, and averaging their prices.

Reframe the K nearest neighbor from classification to regression by averaging the top-K labels. Use two-feature house data with longitude and latitude, built as separate features and labels tensors.

Learn to implement k-nearest neighbors in TensorFlow.js by building feature and label tensors, computing distances for a prediction point using broadcasting, and applying subtract, pow, sum, and sqrt.

Explore how to maintain correct distance-to-label pairing while sorting tensors in TensorFlow.js by using concat, expand_dims, and axis-aware operations to extract the k lowest records.

Sort a 2d tensor by unstacking into tensors, sorting with a custom comparator on the first value, then take the top k for averaging.

Sort tensors by distance to the prediction point, then take the top K records using a vanilla JavaScript slice. Compute the average of the corresponding labels with a reduce operation.

Move the k-nearest neighbor regression from the JS playgrounds to the code editor, wiring it to a real housing dataset and loading CSV data with Load CSV.

Wire up TensorFlow in Node.js to load a CSV dataset, shuffle and split train and test sets, and extract latitude, longitude, and price for a KNN model.

Implement a k-nearest neighbors analysis in JavaScript by loading csv data, converting features and labels to tensors, and predicting a test point.

Compute the error as (expected minus predicted) divided by the expected value, times 100, then evaluate this percentage across all test features to assess model accuracy.

Add the square foot lot as a feature and compare normalization versus standardization to scale data, showing how min-max distorts outliers and standardization better handles extreme values.

Discover how to standardize features in TensorFlow by computing per-column means and standard deviations with tf moments, then normalize values by subtracting the mean and dividing by the standard deviation.

Apply standardization in a k-nearest neighbors algorithm by computing mean and variance, scaling the prediction and features, and integrating the standardization step into the distance calculation.

Debug calculations with a node debugger and chrome devtools to verify feature scaling and standardization. Inspect inputs and scaled predictions, and verify the K and N function stays near zero.

Tune the k nearest neighbors model by adjusting the K value and adding square foot living to improve accuracy. Learn to connect the algorithm to a front-end app.

Learn how linear regression models the relationship between dependent and independent variables and enables fast predictions once a model is trained.

Explore how linear regression derives an equation and coefficients to relate multiple independent variables to a dependent variable, offering insights beyond a spreadsheet trend line.

Learn how gradient descent solves linear regression by using mean squared error to evaluate price predictions from square footage, and see how lowering the error improves the model.

Investigate how minimizing MSC guides estimating the linear regression coefficients M and B, critique brute-force searches, and lay groundwork for gradient descent in learning M and B.

Explore how mean squared error guides choosing B without brute force by analyzing the error trend's slope, revealing when guesses approach the optimal value.

Use derivatives to estimate the slope of the B–MSC relationship and make educated guesses of B without exhaustive search, illustrated by the derivative of x squared plus five.

Learn how to minimize mean squared error by computing the derivative with respect to B, using gradient descent and a learning rate to iteratively update B toward an optimum.

Review how gradient descent uses mean squared error to guide updates of B via the slope and derivative. Apply learning rate to iteratively adjust B until the slope becomes small.

Explore how the learning rate controls updates to B in gradient descent, taming adjustments to avoid overshoot, and how slope sign and mean squared error guide convergence.

Learn why the derivative guides gradient descent, not just comparing MSC values, and how focusing on B with zero MSC slope finds the optimum.

Extend gradient descent to jointly update m and b using two slope equations of the mean squared error. Demonstrate updates with a learning rate and a spreadsheet walkthrough before coding.

Apply gradient descent with mean squared error to tune parameters M and B, adjust using the learning rate, and address scaling with min-max scaling or standardization before implementing in TensorFlow.

Implement a linear regression gradient descent in a JavaScript project, using the cars.csv dataset to relate horsepower to fuel efficiency, and build LinearRegression class with train, test, and predict methods.

Load car data from cars.csv in index.js, prepare features (horsepower) and label (mpg), and set up a two-file workflow with a linear regression model trained via gradient descent using TensorFlow.js.

Learn to build a linear regression module in JavaScript by loading a dataset, creating a linear regression class, and applying default options with a learning rate of 0.1 using TensorFlow.

Formulate the training loop for a linear regression model in JavaScript, implementing a train method that runs gradient descent with an iterations option (default 1000) to update M and B.

Implement the initial gradient descent in linear regression class to compute the slopes for M and B and update their guesses, then the train method iterates toward an optimal solution.

Calculate the b slope by comparing each mx plus b guess to the actual mpg values. Compute the m slope using horsepower, preparing to update the current mpg guesses.

Finish implementing gradient descent by updating M and B from the computed slopes using the learning rate, and prepare to train linear regression with features and labels.

Train a linear regression model with gradient descent on features and labels, tuning learning rate and iterations. Use min max normalization or standardization to stabilize training before refactoring to TensorFlow.

Learn how matrix multiplication underpins refactoring gradient descent with TensorFlow, covering matrix and tensor concepts, shapes, inner dimensions, and why multiplication order matters.

Determine whether matrices A and B can be multiplied, determine the shape of the output matrix C, and compute its values by hand to simplify gradient descent logic.

Explore how matrix multiplication refactors gradient descent by computing the slopes with respect to M and B in one equation using a features matrix, weights, and tensor operations.

Learn to condense equations using matrix multiplication by turning features into a tensor, adding a column of ones, and combining weights M and B to yield MX plus B.

This lecture explains how to combine features and weights into a single equation using matrix operations, transposes, and multiplies to compute m and b slopes for linear regression.

Refactor the linear regression class to use TensorFlow matrix operations. Convert features and labels to tensors, append a column of ones, and replace gradient descent with the new single equation.

Refactor weights into a single tensor and initialize to zero, then compute mx plus b with matmul and differences, and update via gradient descent divided by the number of observations.

Vectorize the gradient descent by computing slopes for M and B, multiplying by the learning rate, and updating the weights tensor, with TensorFlow concepts guiding the refactor.

Examine how the vectorized gradient descent differs between the old and new implementations, and how learning rate adjustments reveal when the two approaches are identical, guiding robustness testing and optimization.

Train a linear regression model to estimate M and B, then evaluate test predictions with r squared, computing sum of squares residuals and sum of squares total to gauge accuracy.

Implement a test method to compute the coefficient of determination. Use test features and labels as tensors, add a ones column, and predict with weights to gauge accuracy.

Compute the coefficient of determination (R-squared) by comparing predictions to test labels, using sum of squares, residuals, and mean with TensorFlow; negative R-squared signals poor accuracy, prompting future strategies.

Compare normalization and standardization, scale features using mean and variance within negative one to positive one standard deviation, and apply the same preprocessing to training and test data.

Learn to create a dedicated process features helper to convert feature arrays to tensors, append a ones column, and concatenate features, with plans to add standardization.

Implement process features standardization by computing and saving mean and variance on first use, then reusing them for test features. Validate by testing results and debugging as needed.

Investigate why standardization yields a poor r-squared by showing the ones column is standardized, then fix by applying concatenation after standardization to preserve a true ones column.

Massaging learning rates demonstrates adjusting learning rate and iterations in gradient descent, uses standardization, and adds features to improve R-squared and model accuracy.

Extend linear regression to multivariate models using weight, displacement, and horsepower to predict mpg. Implement vectorized gradient descent with matrix multiplication for multi-feature analyses.

Refactors the linear regression JS to support multivariate analysis by aligning the weights with feature columns, adds horsepower, weight, and displacement, and tunes learning rate and iterations to improve R-squared.

Learn to optimize the learning rate in gradient descent for linear regression by tracking mean squared error and adjusting the rate: up on improvement, down on worsening.

Learn to compute a vectorized mean squared error, record its history during gradient descent, and use MSE trends to adjust the learning rate in training.

Update the learning rate by examining the two most recent mean squared error values in the MSC history, shrinking when MSE rises and expanding when it falls.

Observe a learning rate schedule during gradient descent, updating the rate and tracking the mean squared error. Starting at ten, it decays toward smaller updates, with an R-squared of 0.65.

Learn to visualize mean squared error and MSC history with a simple plot using node remote plot, label axes, and interpret how learning rate and iterations affect convergence.

Plot mean squared error against evolving B values in a gradient descent–driven linear regression model, recording B history to visualize convergence and the impact of B on mean squared error.

Compare batch and stochastic gradient descent with full gradient descent in linear regression, updating parameters B and M (and M2, M3) using batches or single observations to speed up convergence.

Refactor linear regression to implement batch gradient descent and expose features and labels as batch inputs, revealing equivalence to stochastic gradient descent with batch size one.

Set up mini-batch gradient descent by implementing a configurable batch size, calculating total batches with floor division, and updating the train method to pass batch data to gradient descent.

Add a second inner loop to process mini-batches within each iteration, using tensor slice for features and labels, and feed slices to gradient descent.

Assess batch gradient descent performance on 300-400 observations, noting faster results with fewer iterations and notable mean squared error reduction, and discuss switching to stochastic gradient descent.

Train and test a linear regression model, then use the predict function to convert new car observations (horsepower, weight, displacement) into mpg estimates via tensor operations.

Introduce logistic regression as the next algorithm after linear regression for binary classification, predicting one of two categories, with examples like pass/fail, spam vs not spam, and Apple vs Android.

Compare logistic regression to linear regression, using age as the single feature to predict a binary preference (read books vs. watch movies) after encoding labels to 0/1.

Explore why linear regression with MX+B can't bound predictions to 0–1 and trial alternative equation forms to map age to binary activity preference in a JavaScript machine learning course context.

Explore how the sigmoid equation enables logistic regression by mapping age to an activity preference between 0 and 1, using gradient descent to optimize M and B.

Understand how the sigmoid outputs probabilities between 0 and 1 in logistic regression and how a 0.5 decision boundary assigns binary labels like reading books or watching movies.

Compare linear and logistic regression via gradient descent, noting encoding labels as 0 or 1 and using a sigmoid in place of MX plus B, with learning-rate updates.

Build a logistic regression model to predict smog pass/fail from weight, horsepower, and engine displacement, and reorganize the project into data, linear regression, and logistic regression folders.

Load the car data CSV into a logistic regression workflow, using weight, displacement, and horsepower as features and past emissions as the label, with shuffle and test split options.

Encode the past emissions label by converting true/false to 1/0 with a converters option, then prepare features and labels for logistic regression in JavaScript.

Reuse linear regression code for the logistic regression class, rename it, and apply the sigmoid to MX+B. Switch to cross-entropy while keeping gradient descent in a vectorized form.

Implement logistic regression by applying the sigmoid to the mx plus b output from features times weights in TensorFlow, using tensors, matmul, and element-wise operations.

Refactor the logistic regression class to use the sigmoid in training and prediction, instead of the MX plus B values, and adjust tests to reflect logistic accuracy with manual predictions.

Learn how to gauge classification accuracy for logistic regression by converting probabilities to binary predictions with a 0.5 threshold and counting incorrect predictions to compute accuracy.

Update the test function to compute logistic regression accuracy using the predict function on test features, apply a 0.5 decision boundary, and return the percentage correct from test labels.

Move rounding into the predict method for a JavaScript logistic regression class to return label predictions. Introduce a configurable decision boundary (default 0.5) with TensorFlow casting for reliable results.

Discover how mean squared error history informs learning-rate updates in gradient descent, and why cross entropy is used for logistic regression to ensure a convex optimization landscape.

Refactor the record cost to compute cross entropy using a vectorized, matrix-based approach with features, weights, sigmoid, and log operations.

Finish refactoring the record cost function to compute cross-entropy, rewrite term two as negative actual plus one, and apply log, transpose, and matmul to update cost history.

Validate updated learning rate metric and gradient descent effects in logistic regression. Plot the cost history over iterations, showing batch size influences convergence.