A deep understanding of AI large language model mechanisms

Discover how text becomes numbers through tokenization and embeddings, and how tokens, embeddings, subwords, and context windows drive modern language models.

Explore building a vocabulary for tokenization and the role of token ids. Implement encoder and decoder mappings with Python dictionaries to tokenize text and recover words.

Explore code challenges that teach encoder and decoder functions, one-hot encoding, and tokenization by converting text to token indices and back, with vocabulary, context, and visualization.

Explore one hot encoding and tokenization by building a sparse token matrix from a sentence and its vocabulary, then transpose and visualize the encoding as an image.

Import text from the web into a Python session, clean it with regular expressions, and parse it into words to build a word-based vocabulary for tokenization.

Learn tokenization skills by building vocabularies and encoder–decoder functions, handling unknown words with an unk token, and exploring Brownian noise and token length distributions.

Compare character, word, and subword tokenization for language models, with subword tokenization, especially byte pair encoding, balancing vocabulary size, context, and semantics.

Explore how byte-pair encoding builds tokenization vocabularies by merging frequent character pairs into new tokens, implemented in Python and iterated to a target vocab size.

Tackle a code challenge to implement byte-pair encoding with a loop until a vocab size of 25 tokens, reusing functions from the previous video and exploring subword tokens and spaces.

Explore how the GPT four tokenizer encodes text into tokens, including spaces and new line characters, and learn to encode and decode tokens in Python.

Analyze how tokens map to word length using the GPT-4 tokenizer on real text, splitting by whitespace and punctuation, and measuring token efficiency.

Explore how word frequency relates to token length through a tokenizer and GPT four encoding, and examine confounds like punctuation that affect word versus token interpretation.

Explore how models tokenize text; strawberry becomes three tokens, not a single r. Decode with Python to count, and recognize embedding vectors govern model answers.

Join a code challenge to generate an algorithmic rapper name by tokenizing your name and color, sorting tokens, and decoding three numbers with a GPT-4 tokenizer in Python.

Explore the Bert tokenizer and how it differs from GPT, including uncased vs cased vocab, vocab size, special tokens like CLS and SEP, and encode methods.

Count character occurrences in BERT tokenizer tokens, excluding unused tokens, and visualize the results with a bar graph of digits and letters using Python and numpy with the uncased tokenizer.

Learn how GPT-4 and Bert tokenizers differ and how to translate between them using an intermediate text, while analyzing encoding, decoding, special tokens, and whitespace effects on compression.

Practice translating Bert and GPT-4 token indices to illuminate tokenization, tokenizers, and how embeddings and llm mechanisms work, highlighting encoding, decoding, and inverse relationships.

Explore how tokenization compresses text into fewer tokens to extend a model’s context window, and follow hands-on exercises measuring token-to-character compression ratios with the GPT-4 tokenizer.

Explore how tokenization differs across languages, comparing GPT-4 and Bert tokenizers, and learn why token counts can expand or compress text depending on language and training data.

Explore Zipf's law, the log-log relationship between word frequency and rank, and apply it to characters and token IDs through a Python code challenge using numpy.

Explore how the Claude tokenizer handles spaces, misspellings, punctuation, and multi-language code, illustrating subword tokenization, token IDs, attention masks, and a 65,000 token vocab.

Explore how embedding spaces and vectors power language models by comparing Word2Vec, GloVe, GPT, and BERT embeddings, and learn how static and contextual matrices are learned and used.

Explore glove pretrained embeddings by inspecting the 400,000 word vocabulary and 50-dimensional vectors, computing cosine similarity to find most similar words and nonmatches using Python.

Explore glove embeddings trained on Wikipedia versus Twitter text, visualize 50-dimensional vectors, compare embedding matrices, and preview representational similarity analysis using cosine similarity.

Explore representational similarity analysis to compare Wikipedia and Twitter word embeddings, compute inter-word cosine similarities, and visualize deviations from the line of unity to interpret word relationships.

Explore GPT-2 and BERT embeddings by importing entire models, compare word embeddings and token lengths, and analyze embedding distributions to reveal model-specific vector characteristics.

Explore how tokens and embeddings represent numbers and arithmetic, explain why token-based math is unreliable, and show how embeddings support meaningful arithmetic through unembedding.

Explore cosine similarity as a practical measure of relationships between vectors, including embeddings, and learn its formula, interpretation, and Python implementations, with a comparison to Pearson correlation.

Explore cosine similarities in the GPT-2 embeddings matrix by tokenizing words (including multi-token words) and implementing both loop-based and vectorized calculations, including plotting the results.

Explore how embeddings matrices convert token indices to vectors and back to text, and compare trained versus tied unembedding. Observe how real and random embeddings affect generated text coherence.

Explore how position embeddings provide temporal weighting to link current tokens with earlier context, using learned or sine-cosine predefined vectors, and examine cosine similarity matrices.

Explore position embeddings through cosine similarity analyses, extract unique upper-triangle elements, and compare real versus shuffled embeddings to reveal structure and identify high-similarity vector pairs using numpy tricks.

Learn how embeddings matrices are trained from data using gradient descent, and how to initialize, train, and use an embeddings matrix inside a non-language model for educational purposes.

Create a PyTorch data loader for language model training by formatting text into input-target token sequences, with context length, stride, and batch size in a dataset class.

Build a helper model to train the embeddings matrix that maps tokens to dense vectors. Understand how sequence length and dimensionality shape these embeddings.

Explore loss functions in training embeddings and language models, from mean squared error to negative log likelihood, and learn how log softmax and cross entropy drive gradient descent in PyTorch.

Train an embeddings model on the GPU with a batch size of 32 across epochs to minimize loss. Extract pre- and post-trained embeddings and analyze cosine similarity for upcoming code challenges.

Explore how embeddings evolve with training by comparing pre and post training distributions via histograms. Analyze time and machine embeddings and their cosine similarity, with direct visualizations of embeddings matrices.

Train ten reinitialized embeddings for 16 epochs on identical data to test stability, compare losses and vectors, and analyze cosine similarities among time, machine, and she to reveal embedding relationships.

Skip building a model from scratch for real use due to high costs and complexity, and instead leverage free pre-trained models while learning transformer architectures and attention mechanism.

Build a small from-scratch language model, from embeddings to unembedding, matching the GPT-2 small. Learn text generation using logits, softmax probabilities, and probabilistic token sampling.

Learn how nn.Embedding and nn.Linear act as wrappers around fundamental PyTorch weights, showing their shared under-the-hood structure, with differences in sizing, indexing, and initialization.

Compare gelu and relu activation functions, exploring their non-linearities and exact versus approximate formulas, and their impact on gradient descent in deep learning.

Explore softmax, its temperature parameter and logsoftmax with numpy and pytorch demos, and learn how these transformations create probability distributions for ai large language models.

Explore how torch.multinomial samples from softmax probabilities, compare with NumPy's random choice, handle PyTorch errors, and understand replacement vs non-replacement in language model token selection.

Explore greedy, top-k, and top-p token sampling methods and their trade-offs from deterministic output to probabilistic choices, clarifying when randomness enhances or hinders accuracy in various tasks.

Demonstrate how softmax sharpens probability distributions, explore iterative softmax and the role of data range and temperature, and explain why normalization prevents vanishing or exploding values.

Explore how layer norm stabilizes deep learning by standardizing data to zero mean and unit variance, then scale with gamma and shift with beta, applied across columns or entire matrices.

Explore how position embeddings are added to token embeddings in the GPT-2 model. Learn how layernorm and tied output reduce parameters, and how temperature controls generation in model two.

Explore causal attention and time-based masks, softmax dynamics, and the GPT-style decoder versus Bert encoder distinction for training and generation.

Explore causal masking in token prediction by implementing a lower-triangular softmax mask to ignore future tokens, and see how past activations influence future predictions through matrix operations.

Explore the attention mechanism in transformers, including query, key, value, and softmax scoring. See how embeddings are adjusted in single-head attention, with the causal mask and scaling.

Implement the attention mechanism in PyTorch, including q, k, v and a pass from tokens to embeddings. Compare attention with PyTorch scaled dot-product attention and measure CPU versus GPU performance.

Explore how a single attention head updates token and position embeddings within a transformer block, combining attention and an MLP to refine language model representations.

Explore how a transformer block uses attention and an mlp sublayer with residual connections and layer normalization. Learn how dimensionality expansion and contraction with a non-linear activation enables richer representations.

Translate the transformer block into code by building modular classes for attention and the transformer, illustrating forward passes, layernorm, residual connections, and the MLP expansion to model language.

Expand to 12 transformer blocks in a GPT-style decoder by using a flexible sequential arrangement, reusing a single transformer block with multi-head attention, layer normalization, and feedforward sublayers.

Explore multi-head attention by slicing Q/K/V into equal heads processed in parallel, and learn the math and implementation, including head dimensionality, scaling by d/h, and final W0 mixing.

Demonstrates why shifting from cpu to gpu speeds up matrix multiplications in deep learning, shows how to move data and models with PyTorch in Colab, and discusses transfer overhead.

Build and run a complete GPT-2 style language model on the GPU, using a fused query–key–value weight matrix for efficient attention and transformer blocks to reveal trainable parameters.

Compare model five initialization, forward pass, and backprop times on cpu and gpu, revealing gpu speedups and discussing higher model demands and gpu safety regulation implications.

Explore a pre-trained GPT-2 small model from Hugging Face, using its tokenizer and auto model for causal LM to inspect embeddings, transformer blocks, attention, and text generation in Python.

Examine how token embeddings transform through the transformer's attention heads and the mlp block, using equations to map inputs to logits via embedding and projection matrices.

Explore nano GPT architecture from token embeddings and position embeddings through three transformer blocks, attention with causal mask, and MLP blocks to produce logits for next-token predictions.

Count trainable parameters across GPT-2 variants by importing models from Huggingface, organizing them in a dict, and looping through weights and embeddings to compare totals.

Continue the code challenge by comparing attention and MLP parameters in GPT-2 models, visualizing a bar plot of the percentage of total weights, and exploring layernorm contributions toward mechanistic interpretability.

Investigate weight distributions in GPT-2 small pre-trained model by extracting token and position embeddings, creating histograms, and comparing attention and MLP weight ranges across transformer blocks using density plots.

Explore causal mechanistic interpretability by replacing the Q matrix with noise in a GPT-2 code challenge, comparing CPU and GPU behavior, and examining targeted, selective manipulations of transformer weights.

Learn how large language models transition from random weights to a pre-trained base model by learning textual statistics in unsupervised pre-training, and distinguish it from fine tuning and instruction tuning.

Explore Hugging Face’s open source NLP resources, including models and datasets, and learn how to access free resources via Python. Discover their YouTube channel for tutorials and practical AI development.

Explore how AdamW decouples weight decay from the gradient-based update, applying L2 regularization after the Adam update to improve generalization in large models.

Compare sgd, Adam, and AdamW in a PyTorch mini-training loop that tunes a single weight toward pi, highlighting gradient accumulation and the impact of zeroing gradients.

Train model 1 demonstrates building a tiny language model from embeddings and a non-linearity, training with negative log likelihood on PyTorch for 25 epochs to illustrate loss and text generation.

Add a test set and train/test split to assess a language model's generalization, using a 90/10 split on the Time Machine book with 21,000 training sequences and 2,500 test sequences.

Copy GPT-2 embeddings (50,000 by 768) into a smaller model, freeze those weights during training, and compare frozen versus trainable embeddings for practical insights.

Train model five with modified sampling to pre-train on Gulliver's Travels using the GPT-2 small tokenizer, compare GPU versus CPU runtimes, and evaluate losses after 500 samples.

Learn to design and implement custom loss functions in PyTorch, compare L1 and L2 losses, and understand how loss guides parameter learning and model training.

Train a simple language model on a GPU to prefer tokens containing the letter X, using a KL divergence loss, and examine the ethical implications for AI safety.

Explore how softmax sensitivity to data scale affects attention by scaling Q transpose matrix by the square root of dimensionality, and examine GPT-2 layernorm parameters for stability and token selection.

Discover how weight initialization aids training stability in very large models by keeping weights small, with three methods—Chi Ming, Xavier, and a code-based normal distribution example.

Apply a language model init method across all modules, initialize nonlinear layers with a normal distribution and zero biases, use Xavier for embeddings, and track attention weight distributions during training.

Explore dropout as a regularization technique in deep learning and PyTorch, including class and function implementations and the training versus evaluation effects on activation and generalization.

Explore when to output raw logits versus log-softmax in language models. Learn how temperature affects generation, training, and classification tasks.

Explore the fine web dataset from hugging face, a vast deduplicated corpus of 15 trillion tokens for training large language models, and learn to import and tokenize portions in python.

Learn to implement dropout regularization in a transformer language model during training, covering embedding, attention, and MLP dropout. Explore handling raw logits and temperature in the generation step.

Train model five with dropout on the final token loss. Use logsoftmax inputs, switch to eval mode for testing, then return to train mode.

Explore how a language model forms internal representations for tokens across frequency categories. Analyze frequency, never-used tokens, and logsoftmax learning dynamics using four exercises and Gulliver's Travels.

Explore pre-training concepts and practical strategies to accelerate training of large models. Discover how batch size, precision, GPUs, gradient techniques, and fused attention algorithms reduce training time without sacrificing learning.

Fine tuning large language models uses a pre-trained base model and a domain-specific dataset to tailor performance, with small learning rates and selective layer freezing.

Fine-tune a pretrained GPT-2 on Gulliver's Travels using PyTorch, explore tokenization and frequent tokens, adjust learning rate, monitor loss, and evaluate generation quality and overfitting.

Compare three learning rates for fine-tuning a GPT-2 model on Gulliver's Travels, analyzing train loss and token-based evaluations to balance specialization with preserving base knowledge.

Explore how tokenizers and pre-trained models like GPT-2 handle padding, end-of-sequence tokens, and attention masks in batched inputs. Learn to parameterize generate with input IDs, max length, top-k, and top-p.

Explore a pre-trained GPT-2 medium model, inspect its 24 transformer blocks and embeddings, and implement a KL-divergence loss by converting raw logits to log probabilities to maximize token x selections.

Fine tune two GPT-Neo models on Alice in Wonderland and Edgar Allan Poe texts, compare architecture and tokenizer, and assess outputs through a qualitative prompt.

Fine-tune two models to adopt Alice in Wonderland or Edgar Allan Poe style, then quantify and qualitatively evaluate their generated tokens before and after training, including a crossover analysis.

Train the Alice and Edgar models, set up two optimizers, and run a back-and-forth chat that expands the token context window with each exchange.

Explore how to freeze weights in large language models using PyTorch, selectively freezing transformer blocks to reduce training costs, preserve syntax, and tailor fine-tuning.

Explore precision freezing and targeted fine-tuning of large language models by training two GPT-2 Neo variants on Moby Dick, and measure training loss, token usage, and compute time.

Explore fine-tuning and targeted freezing in language models through hands-on matplotlib visualizations comparing frozen versus fully trainable models, losses, token sampling from Moby Dick, and computation time.

Explore parameter-efficient fine-tuning (peft) techniques that freeze most parameters and train only a small set of adapters, prefix modules, or bias terms for large pre-trained models guiding narrow tasks.

Discover how Codegen enables code completion using Salesforce's pre-trained base models, from 350 million to 16 billion parameters, and how to download, inspect, and fine-tune them with Hugging Face.

Fine-tune the CodeGen model on calculus Python code and assess post-training outputs. Explore learning rate choices, training epochs, tokenizer setup, and qualitative evaluation, plus instruction tuning concepts.

Fine-tune a pre-trained BERT for classification by adding a two-output head to predict movie reviews as positive or negative; cover IMDb data from Hugging Face and batching.

Explore IMDb sentiment analysis with bert in a code challenge, training a classifier on movie reviews and experimenting with freezing embeddings and attention while training mlp and final classifier.

Master gradient clipping and learning rate scheduling for training large models. See how gradient norms are clipped to a threshold and how warm-up with cosine or linear schedulers shapes learning.

Explore how to pair an optimizer with a gradient clipping and learning rate scheduler. The demo covers cosine and linear schedulers, warm-up steps, and training cycles.

Explore fine-tuning a pre-trained BERT model for sentiment classification, using gradient clipping and learning rate schedulers, with visualization of gradients, loss, and accuracy.

Learn to save and load models across Hugging Face and PyTorch, using folder-based saves with config and safe tensors, or PyTorch state dictionaries, and how to download or upload them.

Train a Bert classifier to distinguish Alice versus Edgar texts on GPU, compare mean smoothing vs raw loss, and save the model for the next code challenge.

Train and evaluate a Bert classifier with two Eleuther GPT Neo models in a single Python session, managing memory and token translation between Bert and Eleuther for 128-token sequences.

Fine-tune alice and edgar models on their texts with 121 training samples and a 1e-5 learning rate, tracking Bert classification accuracy over the training.

Fine-tuning your own model can offer greater control, ownership, and task-specific performance for narrow applications, even when state-of-the-art models exist; assess practicality, data access, and on-device speed.

Discover how instruction tuning shapes chatbots to interact with users like humans, using next token prediction, negative log likelihood, and gradient descent, while aligning with human scripts and constraints.

Explore three public datasets used for instruction tuning to shape model interactions. Learn how data format, references, and lightweight preprocessing influence model behavior.

Learn how base language models are trained into chatbots using next token prediction and gradient descent, with thousands of curated, human-crafted examples built from system, user, and assistant prompts.

Fine-tune a GPT-2 model on a question-answer dataset using tokenization, batching, and GPU training to build a chatbot, then examine its authoritative tone and AI safety implications.

Apply instruction tuning to GPT-2 large, analyze token distributions in Q&A data, and compare padding strategies with 256-token sequences, attention masks, and start-of-question formatting.

Continue the code challenge by fine tuning GPT-2 large and evaluating results. Explore batch size, learning rate, and GPU RAM management, including intermediate variables and logits, to understand training dynamics.

Reinforcement learning from human feedback uses human ratings of model outputs, via a separate reward model, to align language models with human values and desires beyond token-based losses.