Create a Neural Network in Java

Build a custom matrix class in java that uses a one-dimensional double array to store values, enabling efficient matrix multiplication of weights and inputs for neural networks.

Initialize a matrix with values by adding a producer interface and a constructor that assigns each element using the producer; illustrate with a lambda and a JUnit test.

Implement matrix toString by using a string builder to format each value with five decimals, iterating rows and columns, and appending a newline per row, with test output.

Develop and validate a neural network in java by testing the toString output of a matrix, parsing rows and values with regex, and asserting numeric accuracy.

Add a new matrix apply method that uses a value producer to perform diverse mathematical operations and ensure the sign is always shown in the result.

Create and run a test to verify the scalar multiplication using the apply method, and prepare the equals method to compare the resulting matrix with the expected values.

Create an equals method for matrices using a tolerance for floating point values, replace array equals with elementwise comparison, and test equality with unit tests.

Demonstrates using the equals method to validate matrix multiplication via tests, building an expected matrix and asserting results, with a get method for value checks.

Explore adding two matrices of the same size and dimensions, verify results with a test, and discuss implementing matrix methods and performance considerations for Java.

The lecture motivates matrix multiplication with a hands-on table problem about how many meat and fish cans to bring to two houses, using dogs and cats to illustrate.

Explore filling a result table from two input tables via matrix multiplication, matching rows to columns, and interpreting dog and cat data as meat and fish.

Learn the dot product and matrix multiplication by combining vectors and matrices, computing entries as row-by-column products to build a result matrix.

Learn the matrix multiplication rule with A, B, and C; compute cij as dot product of the ith row of A and the jth column of B, and implement it.

Explore matrix multiplication via dot products of vectors and determining the result shape. Compute each entry as a row-by-column dot product, noting non-commutativity and Java-like array concepts.

Master matrix multiplication by exploring dot product and its related shapes and rules for multiplying matrices and vectors, with examples that prepare you for coding a neural network in Java.

Implement a Java matrix multiply in a new matrix class, returning a new result, enforcing dimensions with assertions, and testing with JUnit while treating matrices as immutable.

Implement matrix multiplication with a nested loop over result rows and columns, using row-major 2D-to-1D indexing (row * number of columns + column) and zero-based indices.

Learn to implement matrix multiplication by iterating over multiplicand rows with nested loops, determining the start of each row using row times columns, and outputting results for debugging.

Complete the matrix multiplication in java by iterating over columns and the multiplier matrix, validate results, and prepare for optimization with external libraries.

Learn to test and time a Java matrix multiplication, validating results with expected values and timing runs on large matrices (500x500) to compare performance and identify optimizations.

Optimize matrix multiplication in java by reordering nested loops to improve cache locality. The video tests six loop orders and shows row-column order as the fastest.

Create a JUnit test class to build a simple neural network in Java, using input and weight matrices to compute neuron outputs, with biases planned for the next step.

Modify matrices in a neural network by introducing a modified method that uses a row co-producer to apply biases to a result matrix via nested loops, returning the updated matrix.

Add biases to the neural net by applying the bias vector to each neuron's weighted sum, using a modify lambda to retrieve the bias and verify with a dot product.

Organize the input as a matrix with multiple columns, each column a vector, so matrix multiplication yields a corresponding output column. Demonstrate that the same bias applies across input columns.

Explore activation functions and learn how a nonlinear function like relu enhances neural networks by converting negative weighted sums to zero and preserving positive values.

Develop a ReLU test in Java by configuring neuron counts, input sizes, and Gaussian-initialized weights and biases; verify that negative values are zeroed while positives pass through.

Add a value producer to set matrix values and a new index value producer, then implement a modify method and a for each using an index value consumer.

Learn to implement the ReLU activation in a Java neural network, test its nonnegative outputs by comparing results, and prepare to use soft marks for probabilistic output representations.

Apply softmax as the final-layer activation to convert weighted sums into probabilities, normalizing so all outputs sum to one for accurate multi-class classification.

Explore a practical softmax worked example by exponentiating output values with Java's Math.exp and normalizing by their sum to obtain probabilistic class predictions.

Add a new method in the matrix class to sum each column, returning a single-row matrix of column sums, and validate with tests and debugging insights.

Implement the soft marks activation function as a matrix method in Java by exponentiating values, computing column sums, and normalizing each column to produce probability distributions for multi-column inputs.

Test the softmax implementation in a Java neural network by using a row-column-value consumer and a for-each method, verifying outputs are in [0,1] and column sums near one.

Create a neural network engine in Java by defining a Transform enum for dense and activation layers, wiring weights and biases, and chaining transforms within an engine.

Build a hand-built three-layer neural network by defining the input layer, layer one with weights and biases, and layer two using neuron counts and previous layer outputs.

Assemble weights and biases to form an untrained two-layer network, multiply input vectors by weight matrices, and apply relu and softmax activations to generate output.

Configure engine to support dense layers adding weight and bias matrices with optional params, derive neurons from params, determine weights per neuron from previous layer, and initialize with Gaussian randomness.

Enhance the neural network builder by implementing a toString method with a show values toggle, then add and configure multiple layers including dense, relu, and softmax while validating matrix dimensions.

Implement a forward run of a Java neural network, applying dense layers with weights and biases, handling transforms, and preparing for training through backpropagation concepts.

Learn how to evaluate neural network outputs by using mean squared loss for numeric targets and categorical cross entropy for one-hot class predictions in classification tasks.

Explore categorical cross entropy and information theory, defining information as surprise and showing how to encode it into bits to compare a neural network's output with ideal result in Java.

Explore how symbol spaces determine information per symbol, showing that four equally likely symbols require two bits each, and how compression and encoding schemes like ASCII relate to conveying data.

Learn how yes-no questions determine a four-symbol space, measure entropy, and produce a 2.25-bit average using a non-optimal strategy, with smarter encoding to be explored next.

Explore an optimal encoding strategy that uses yes-no questions to halve the symbol space, achieving two bits per symbol for four options and revealing entropy concepts.

Analyze how unequal symbol probabilities shape entropy and the average surprise per symbol, using an A/B book example to illustrate compression and encoding strategies.

The lecture defines information content in a nonuniform symbol space using probability and log base 2, noting rare symbols carry more, with I(s) = -log2 p(s) and a 1/8 example.

Calculate entropy for unequally probable symbols using the formula H=-sum p log p, illustrating how non-uniform probabilities yield 1.9 bits per symbol and a compression lower bound.

Introduces cross entropy by comparing theoretical English symbol probabilities to a real message's symbol distribution, showing why actual bits per symbol exceed entropy and how cross entropy measures their difference.

Explore cross entropy with symbols A, B, C, D and distributions P and Q. Compute bits per symbol and show how cross entropy measures P versus Q for neural networks.

Apply cross entropy as a loss function to train a neural network that classifies handwritten characters, comparing the predicted probability distribution to the true distribution, and minimize cross entropy.

Implement cross entropy as a loss function for a neural network, using categorical cross entropy with expected and actual matrices to compute per-column and average network loss.

We write a cross entropy test in Java for a neural network, building a 3x3 expected one-hot matrix and a softmax-processed actual matrix to test distinct columns.

Finish the cross entropy test by comparing expected and actual values and confirming that when expected is one, the loss equals the negative logarithm of the actual.

Train a neural network by applying back propagation and gradient descent to minimize the loss function, systematically updating weights and biases rather than random tweaks.

Explore gradient descent as a method to train a neural network, using larger steps on steep gradients and smaller steps as the surface flattens, aiming for local minima.

Explore how the loss of a neural network changes with respect to a single weight or bias, and use gradients to decide how to adjust to minimize the loss.

Explore programmatic differentiation by implementing a Java function differentiator to compute gradients, explore linear and nonlinear functions, and simulate rate-of-change with finite differences.

Implement a differentiation function by measuring the gradient at x with (output2-output1)/dx, and observe gradients for linear and quadratic functions across a range.

Translate programming concepts into basic mathematical notation, showing f(x)=a x, dy/dx as the gradient, and x^2 with a derivative of 2x.

Explore gradient descent by analyzing how the loss changes with respect to a single weight using partial derivatives, including exact and approximate gradients and cross entropy considerations.

Explore a three-layer neural network architecture, starting with input X, applying weighted sums with biases and ReLU activations, then softmax and cross-entropy to generate loss matrix for each input column.

learn to compute gradients of the loss with respect to weights and biases using partial derivatives and the network's layered weighted sums—the error.

Develop an approximate gradient tool for a neural network by iterating an input matrix to estimate the rates of change of a loss matrix, and verify results with tests.

Create a matrix of expected values for the lost function, initialize it with zeros, and set one random row per column to one for neural network testing.

Implement a transform that takes an input matrix and returns a one-row loss matrix across columns, enabling rate-of-change checks for losses. Test with neural net tests and assertions.

Explore cross-entropy loss on a zero-layer neural network by comparing input, output, and loss matrices, and use soft cross-entropy to ensure column sums to one and enable gradient approximation.

Implement an addIncrement method in the matrix class to return a duplicate with a specified element incremented by a value, and validate via unit tests.

Increment each element of the input matrix slightly, recompute the loss, and compute the rate of change with respect to that input element to help finish the approximator.

Apply cross entropy by matching a one-in-every-column expected matrix to the input and compute the negative natural log; determine the derivative of ln(1/x) to validate the approximation.

Finish the approximator test by comparing results to expected values using a small 0.01 value via math.abs, and verify cross entropy and approximation logic in the neural network.

Learn backpropagation to compute the loss gradient with respect to the network output after the weighted sums and before soft marks, using soft marks and cross entropy.

Derive and verify a fast softmax–cross-entropy gradient for an input matrix, showing gradients equal softmax output minus the expected value, and discuss backpropagation implications.

Apply backpropagation to compute how losses change with respect to activations in the second layer, propagating errors through weighted sums using the chain rule.

Learn the chain rule in calculus through currency exchange rates, showing how the rate of change of euros with respect to pounds equals the product of euro-dollar and dollar-pound rates.

Program the chain rule by composing functions and computing derivatives, gradients, and rates of change for x, y, and z, then apply dz/dx = dz/dy times dy/dx.

Apply the chain rule for functions of multiple variables using partial differentiation and summation notation, relating z to y1 and y2, with altitude as a function of latitude and longitude.

Explore the multivariable chain rule in Java by differentiating z = f3(y1, y2) with respect to x. Use two-variable, lambda-based differentiation and compare with a direct derivative.

master matrix multiplication notation and vector concepts, using w, a, b and subscripts to compute products via summation, and practice deriving results with examples.

Learn how matrix transpose affects multiplication, swapping indices of W to produce B from A, and see how transposing changes rows to columns and matrix shape.

Implement a Java matrix transpose, swap rows and columns, and verify results with tests that compare the transposed matrix to the expected values.

Explain backpropagation through weights to compute the input error, applying the chain rule to a single-column input with weights and biases, and tracing losses through the network.

Compute the rate of change of loss for each input using the weights matrix. Propagate the output error to the input via the chain rule with the weight transpose.

Create a backprop weight test in Java, restructure tests, compute the calculated result as the difference between softmax output and expected, and align input, output, and weight matrices.

Create and initialize a weights matrix and bias vector for a neural network, determine shapes from inputs and outputs, initialize with gaussian values, then apply soft marks and cross entropy.

Multiply the input by weights, add biases, apply softmax, and compute cross-entropy loss while backpropagating errors through the transpose of the weights; implement a one-layer neural network in Java.

Improve the weight backprop test by introducing a functional neural net interface and a lambda apply method to reduce repetition, then reuse the same code with softmax and cross-entropy.

Apply backpropagation through the ReLU activation by passing gradients to inputs where x >= 0 and zeroing gradients where x < 0, then propagate the loss gradient onward.

Implement backpropagation through ReLU by adding a backward input stage, propagate gradients through weights and biases, and refine with a softmax and cross-entropy step before wiring into the network.

Add a run backwards method to perform backpropagation in the neural net engine, using an error matrix and reverse iteration through transforms to adjust weights and biases.

Create a utility class to generate mock input and expected matrices for testing a neural network in Java, using static methods and a random generator.

Create a batch result class to store inputs and outputs from forward runs for batched processing with multi-threading, enabling backward passes using a linked list of matrices.

Configure the neural network engine by creating batch-friendly battery cells and wiring inputs and outputs through transforms, then run forwards to produce a batch result and backwards to backpropagate errors.

Ensure the neural network uses cross entropy loss and the final transform is softmax, with configurable loss and transform checks that enable safe code changes and clear error handling.

Store errors at the outputs of weight transforms for gradient descent and the input error for testing in the battery cell.

Initiate backpropagation by computing initial input error from cross entropy with soft marks, and iterate backward through batch outputs to obtain the soft maxed results, optionally storing the input error.

Verify backpropagation by extending the test engine, compute the approximated error from network output, compare with calculated error after forward and backward passes, and ensure consistency through assertions.

Implement backpropagation through weights by traversing weights backwards, computing error via weight transpose multiply error, and adjust testing tolerance to compare matrices with realistic precision in a neural net.

Explore backpropagating through the ReLU activation by wiring an engine, propagating errors through inputs and weights, and updating weights and biases to train a neural network in Java.

Use a store input error flag to turn off unnecessary backpropagation to input, propagating errors for the first dense layer when storage is on and for subsequent layers otherwise.

Calculate the weight gradients by applying the chain rule; show that the gradient matrix equals the error vector times the input transpose for the dense layer.

Practice practical weight gradient calculation in a neural network by testing a weights matrix, computing error with soft max and cross entropy loss, and deriving the weight gradients.

Implement a weight gradient method for a simple neural network in Java, perturb each weight, measure loss changes, and validate the approximate gradient with a tolerance based test.

Show how matrix multiplication with transposed inputs generalizes to multiple columns, yielding a sum of column-based matrices, and use this to compute weight gradients for a dense layer.

Compare online training, which processes one input at a time with gradient descent and learning rate, to batch training that averages gradients over batches, for example 32 images.

Train the network by computing the weight gradients as the error times the input transpose. Update weights and biases using the learning rate times the average gradient over the batch.

Write a unit test for the Matrix class method that averages the error matrix columns to compute bias adjustments, validate with a 3x4 matrix, and implement the average column function.

Learn to implement a public average column method in the matrix class that returns a new matrix with the same rows and one column, averaging each row.

implement an engine.evaluate method to compute the average cross-entropy loss from batch results and expected values, updating the batch result with a single loss value.

Develop a training step by adding an adjust method that uses a learning rate, gathers weight inputs and output errors, and supports backpropagation through dense layers in Java.

Generate a test to create input data and a trainable expected matrix for a neural network in Java. Use the input column sums to place ones, enabling structured training data.

Add code to test train engine for neural net, configure transforms and dense layers, run forwards, evaluate batch results, and train by running backwards with a learning rate of 0.01.

Finish the adjust method to train the neural network by computing weight and bias adjustments from error and input, applying the learning rate. Results show the loss drops after updates.

Implement a matrix method to return the greatest row numbers for all columns, use it to compute accuracy by selecting the column's top value, and test the result.

Calculate the percentage of inputs correctly classified by a Java neural network, comparing predicted versus actual outputs and iterating through batches to report percent correct during testing and training.

Run forward and backward passes to train a neural network with repeated cycles, updating weights and evaluating loss and accuracy across data batches and circles with different radii.

Generate input vectors as coordinates on a circle or hypersphere using gaussian draws, scale to a radius, and train a neural network to classify radii.

Generate training data where each input column is a coordinate on a hyper sphere surface with a random radius and the output encodes that radius.

This lecture wires a training matrix into a neural network, tests with more data, and tracks accuracy from 28% to around 68% with three outputs, noting possible architecture tweaks.

Implement a running averages utility in the neural network project to smooth output by averaging recent loss and percent correct, with a non-overlapping, callback-driven interface and a configurable window size.

Implement a running averages class in Java by adding an ad method that updates sub arrays and computes averages over a sliding window.

Discover how to train a neural network in java by using a linearly decaying learning rate, tuning iterations and layer size, and achieving high accuracy on digit recognition.

Turn test train engine into a test, set the window to two, and run 500 iterations to assess loss, then package into a neural network class with multithreading for speed.

Define a metadata interface for the data loader, encapsulating items and batch counts. Implement a loader interface that supplies batches and a simple neural network API.

Create a batch data interface to describe and access a single data batch from the loader, using double arrays for input and expected data, with generated getters and setters.

Design a loader interface in Java that opens data, provides a close method, retrieves metadata, and reads data batches sequentially.

Create a neural network class with an engine, constructor, a wrapper method, and a toString method, set up a main method, initialize transforms, and run to see the output.

Update the matrix class to initialize from a double array, enabling neural networks to convert a double array to a matrix and arrange each image as a column by transposing.

Implement a new Java method in the neural network package, generate training arrays, refactor training matrix to training arrays, and switch to double[] input and output while validating tests.

Create abstract loader and data classes to simplify implementing loader interfaces, enabling a test loader that uses util methods for the neural network API.

Create a simple Java test loader for a neural network, with test meta data and test batch data under a loader interface, refactoring to input size and output size.

Finish implementing the test loader by calculating batch item counts, handling the last batch, generating test data for input and expected output, and updating total items read in metadata.

Write a JUnit test for the test loader in a Java neural network project, configuring number of items and batch size, reading batches via metadata, and asserting batch data integrity.

Add a simple sum method to the Matrix class that iterates the internal array and returns the total of all elements. Use it to verify matrix contents and support tests.

Finish the loader tests for the neural network in Java, validating the last batch size via number items mod batch size and ensuring items read match expectations.

Develop a neural network class with a fit method, implement an epoch-based training loop, apply a learning rate schedule from initial to final, and evaluate using an eval loader.

Create a train loader with 60,000 items and a test loader with 10,000, batch size 32, then call neural network fit with trainLoader and testLoader, and set epochs to 20.

Implement and run the epoch workflow for a neural network in Java by creating and consuming batch tasks with a thread pool, using futures to manage training and evaluation.

Implement create batch tasks in a single-threaded mode by loading metadata, determining the number of batches, and running each batch with a simple run batch method.

Implement run batch to process a data batch for training or evaluation, using the loader, with a synchronized lock to update weights and compute loss across epochs.

Add multithreading with a fixed thread pool, manage a linked list of futures for batch results, and submit tasks using a lambda to process batches.

Learn to consume batched tasks with futures in a Java multi-threaded workflow, including setting a default two thread setup, batch iteration, and progress dots during training.

Fix a multithreading bug in a Java neural network by synchronizing the read batch method to prevent index out of bounds and execution exceptions.

Learn to print average loss and average percent correct during training and evaluation, accumulate metrics via a loader, and prepare saving, loading, and a predictor for categorizing data in Java.

Add a scale initial weights parameter to the neural network engine to prevent not a number in the loss and overflows in image data by scaling initialized weights and transforms.

Learn to implement and refine the neural network's toString method in Java, using a string builder, format outputs, and expose engine configuration details like learning rate, epochs, and threads.

Learn how to save a trained neural network in java by implementing serializable, marking transient fields, and using object output streams to write to a .net file.

Implement a load method to read a neural network from a saved file using object input streams, handle exceptions, and initialize or reuse the model with debug output.

Fix a null transient lock after deserialization by implementing a readResolve method that reinitializes the lock, and demonstrate saving and loading training state across epochs in a Java neural network.

Add a public double[] predict method to the neural network, feed input data, run forwards, and return the prediction for use with the handwritten digits dataset.

Load the MNIST binary dataset of handwritten digits 0–9, train a neural network on it, and evaluate with separate test data to monitor generalization and avoid overfitting.

Load the neural network data by passing a data directory as a command line argument, then validate the directory and print a usage message if missing.

Create an image loader class to manage image and label file names and batch size, with a constructor and separate train and test loaders for training and evaluation data.

Open and assemble file paths for MNIST data in Java, using cross-platform separators. Implement a loader exception and data input streams to safely read image and label files.

Explain the MNIST file format, including 32-bit magic numbers, item counts, and labels as unsigned bytes 0–9, with images, rows and columns, and big-endian data input stream order.

Read label metadata by implementing a private readMetadata method that reads a 32-bit magic number and the label count from a data stream, validates against 2049.

Learn to read image metadata, verify item counts, and extract the height and width from image files to prepare a Java image loader for 28x28 handwritten digit images.

Store and manage image metadata for a neural network in Java by creating an image metadata class, setting width, height, item counts, and batch size.

Structure the read batch workflow by adding a read lock for thread safety, replacing prints with logging, and integrating image batch data with input and expected batch methods.

Learn how to read a batch of image data for a neural network by determining items and bytes to read. Convert bytes to doubles and handle io errors during read.

Convert bytes to doubles by reading bytes into a double array, mask to interpret as unsigned, and divide by 255.0 to produce 0–1 values, then feed into the input batch.

Demonstrates invoking the code, loading a test loader with batch data, and validating inputs before reading labels and resolving a mismatch between images and labels.

Read label data from the input batch, convert to one-hot vectors for digits 0-9, and batch them for neural network training, while verifying reads and visualizing images and labels.

Develop a Java image writer class to save loaded images to files, fix a directory validation bug, and run the writer from a main method to verify outputs.

Write grayscale images in Java by creating a 900x900 buffered image, and save montages as jpeg files using image IO with a montage path and batch naming.

Determine the montage canvas width by calculating horizontal images per row with a square root, adjusting for even division, then compute vertical count and canvas height.

map each pixel from the input batch to its image in the montage by computing image number from the pixel index and image size, then determine pixel position.

Write grayscale images into a montage by converting pixels to rgb in Java, debug byte-range issues, and visualize tiled results from the analyst dataset to evaluate a neural network.

Learn to convert a ten-element one-hot vector into a digit label using a private int method, applying an offset and validating the one-hot factor across batches.

Write the image labels to text files by computing label sizes, looping through labels, and appending formatted entries with a string builder, then write with a file writer.

Bring together the neural network, run it, and analyze training and performance using loaders, batches, and input size and expected size to configure the first hidden layer in Java.

Fix a bug in the neural network that yielded disappointing results; after correcting the average weight calculation, 100 epochs reach about 90% accuracy.

Experiment with the neural network by setting the initial weight scale to 0.2 before adding transforms, a trial-and-error optimization. Observe first-epoch accuracy improve from about 90% to 91.32%.

Integrate a trained neural network with an image writer and improve one-hot conversion for AMS dataset labels, using the max prediction index (values > 0.5) to colorize images.

Wire a neural network into an image rater, load a trained model, and generate predictions for image batches, converting outputs to labels and comparing them to actual labels.

Colorize images by using neural network predictions to set pixel colors, comparing correct and incorrect results with red and green highlights, and examining montages to understand model errors.