Applied Generative AI and Natural Language Processing

Explore natural language processing foundations through word embeddings, from one-hot encoding to glove, visualize word vectors, and build a simple sentiment model with pre-trained embeddings.

Explore natural language processing, a field using deep learning to understand and generate language from text data. Tokenization and word embeddings power neural networks for sentiment analysis and translation.

Explore word embeddings by converting text to tensors in a high-dimensional space, starting with one-hot encoding and frequency-based methods, and capturing word relationships.

Explore sentiment analysis with one hot encoding on a 27,000-tweet Kaggle dataset, building a simple neural network. Review the confusion matrix and misclassifications, and preview future word embeddings.

implement sentiment analysis using one hot encoding with a simple neural network in pytorch, leveraging sklearn's countvectorizer, a 50/50 train-test split, and evaluation showing 63.6% accuracy.

Word embeddings built with neural networks, including Word2Vec CBOW and Skip-gram, GloVe, BERT, and GPT, capture semantic and syntactic meaning while reducing dimensionality.

Load glove embeddings with torchtext, select the sixb model in a 100-dimensional space, download and cache the models, then map words to indices to retrieve their vectors.

Learn to obtain a word's embedding with GloVe and compute distances to other words to return the top five closest words.

Explore GloVe word embeddings by solving analogies through vector arithmetic, predicting the fourth word and retrieving closest words like king to queen, man to woman.

Explore GloVe word clusters by reducing 100-dimensional embeddings to two dimensions, map closest words to categories like algebra, music, and numbers, and visualize effective word embeddings.

Explore glove word embeddings, build a word cloud of closest words for categories like algebra, music, science and technology, and visualize 100-dimensional vectors with t-SNE in ggplot.

Develop a sentiment model using pre-trained word embeddings on Twitter data, comparing to one-hot encoding, and modify code to build a neural network with 100-dimensional embeddings.

Apply sentiment analysis with 768-dimensional sentence embeddings from a transformer to boost accuracy while reducing data size. Learn practical steps with Hugging Face, PyTorch training, and comparison to one-hot encoding.

Explore transformers as a breakthrough deep learning architecture for natural language processing, introducing positional encoding, attention and self-attention to overcome RNN limitations and enable large-scale parallel training.

Apply pre-trained networks from Hugging Face to common NLP subtasks using open source models, and explore the range of tasks available for practical use.

Explore Hugging Face, a platform to build, train, and deploy models with infrastructure to run them in live applications, and discover nlp tasks like text classification, translation, summarization, and fill-mask.

Explore how to use Hugging Face pipelines for model inference with simple APIs. Learn task-based and model-based pipelines, default models, and flexible input options.

Explore how text classification assigns predefined categories to text, enabling tasks like spam detection, topic categorization, and sentiment analysis, with models producing positive or negative probabilities.

Learn to set up a transformers text classification pipeline, choose a task, load a default or specific model, and interpret label and score outputs.

Explore named entity recognition (NER) for structured and unstructured data, tagging entities such as organization, people, date, and location in sentences like Apple Inc. founded by Steve Jobs.

Apply named entity recognition using the Transformers pipeline by setting the task, creating an instance, and extracting entities with start, end, entity, and score from a sample sentence.

Explore the question answering task, learn how to input documents and questions into a pipeline to extract information and obtain answers.

Demonstrate question answering with a transformers pipeline by supplying context and a question; the model retrieves 'the Big Apple is a nickname for New York City' as the answer.

Explore how text summarization distills essential information while preserving meaning and key details, and apply it to documents, social media, and news articles via a summarization pipeline.

Configure a transformers text summarization pipeline with a fine-tuned t5 model, set min and max lengths, and summarize articles like the Hugging Face overview to produce concise results.

Explore how translation pipelines convert a document from a source language into one or multiple target languages. Specify the source and target languages to achieve accurate translations.

Set up a translation task using a transformers pipeline, specifying the source and target languages. Translate English to German with a default model and see the resulting translated text.

Explore fill-mask, a technique that predicts missing words in a sentence. See a pipeline fill blanks with contextually suitable tokens, demonstrated by predicting that the capital of France is Paris.

Explore fill mask in language processing, using a pipeline to predict a masked word in a sentence for data augmentation, with scores indicating confidence, and Paris as the likely token.

Explore zero shot text classification and how pre-trained models apply to unseen classes without retraining. See how candidate labels, attributes, word embeddings, and natural language inference enable class probabilities.

Use a hugging face zero-shot classification pipeline with bert-large trained on the Multinli dataset to classify texts into entailment, neutral, or contradiction, applying candidate labels like crime, fantasy, and history.

Fine-tune a pre-trained network to adapt to a custom data set or a different task, such as regression or film mask, for significantly better results.

Train a simple model by using features from a DistilBert base uncased last hidden layer. Tokenize inputs, generate input IDs and attention masks, then train a classifier on 768-dimensional features.

Load the Yelp review full dataset from the Huggingface hub, inspect train and test splits, labels, and text; assess balance and visualize review lengths with seaborn.

Set up a simple model, load and tokenize the Yelp data with DistilBERT, extract last hidden states, and evaluate classifiers on train-test splits.

Fine-tune a Distilbert base uncased with Hugging Face trainer, using a train/test split on a dataset dictionary, to replace fill mask with a 5-class sentiment output.

Explore the Hugging Face trainer, a simple PyTorch API for fine-tuning transformer models with training and evaluation datasets, including distributed training on GPUs or TPUs.

Fine-tune a DistilBert base uncased model for a five-label sequence classification using the Hugging Face trainer, with a prepared tokenizer, dataset handling, and tailored training arguments.

Log in to hugging face via the CLI, create a token, push a Distilbert base uncased model to the hub with a model card, then load and run it.

Explore how vector databases store high dimensional embeddings for fast querying and similarity analysis across unstructured text and image data, enabling NLP, image search, and real-time analytics.

Master tokenization, breaking text into words, subwords, or sentences to form tokens for embeddings in NLP tasks. Explore recursive character text splitters and practical tokenization approaches for vector databases.

Explore how the tokenizer maps words, characters, and tokens with a practical demo, showing punctuation and multi-token words and a rule of thumb of about four characters per token.

Apply tokenization for vector databases by using a sentence transformers token text splitter to create model-friendly chunks up to 256 tokens, then embed and store tokens in a vector database.

Explore the full workflow of building a Bible vector database: prepare the underlying corpus as text, tokenize, set up the vector store, add tokenized data, and query for relevant documents.

Build a bible vector db data prep pipeline from project gutenberg text, applying recursive character splitting and tokenization to produce 384-dimensional sentence-transformer embeddings on CPU.

Set up a vector database with chroma db, create a Bible collection using an embedding function, tokenize text with sentence transformers, load tokens, and run fast queries for Noah references.

Create a Python function to get the title by description from a movie dataset using a vector database, retrieving titles by semantic matching of plots.

Import and clean a subset of movie data, compute description word counts with regex, visualize distribution with seaborn, and prepare sentence-transformers embeddings for a vector database.

Set up a persistent chroma vector database, create a movies collection with embeddings, insert the movies in batches, and verify the collection contents.

Create a get title by description function using a chroma collection to query and return top three movie titles, recap pandas import and vector database setup for retrieval augmented generation.

Explore multimodal vector databases using clip embeddings to combine image and text representations, enabling image and text based nearest-neighbor queries.

Set up a multimodal vector db with chroma db and open clip embeddings, create a docs collection, and embed six images with unique IDs and URIs.

Build a single one-liner to show multimodal query results using a helper function, querying texts and images with open clip and chroma, and visualize the results.

Survey the OpenAI API and its models, and learn to set up an API key. Discover how REST APIs and Python interfaces let you interact with OpenAI models.

Explore OpenAI's ChatGPT, a stateful GPT-3.5 model with 170 billion parameters, fine-tuned by supervision and reinforcement learning, and its applications, limitations, and developer use cases.

Explore the OpenAI API with GPT-4 and GPT-3.5, covering transformer-based models, embeddings, and parameter controls like temperature, max length, penalties, and top-p sampling in the OpenAI playground.

Start by creating and copying an OpenAI API key, then save it as a Windows environment variable and use it in Python.

Install OpenAI Python package, fetch the API key from an environment variable, instantiate a client, and create a GPT-3.5-turbo chat completion with a user message, reading the first choice's content.

Set up a Python chat bot in Visual Studio Code using chat.py, retrieve the API key from environment variables, create an OpenAI client, and test with GPT 3.5 turbo.

Learn how REST APIs enable apps to talk to web services, based on representational state transfer, sending get or post requests that return json via http or https endpoints.

Explore OpenAI's web interface and playground to configure system and user prompts, adjust model, temperature, token length, top_p, and repetition penalties, then monitor usage, cost, and billing settings.

Explore cost differences across generative AI models and note prices may change; GPT-3.5 turbo balances performance and price, while GPT-4 costs significantly more.

Explore prompt engineering: improve results, control large language models, and compare broad LLMs with task-specific models. Learn iterative refinement, clear instructions, use of delimiters, personas, examples, and output control.

Craft clear coding instructions by specifying output format, complexity, and level of detail. Learn through examples that contrast vague prompts with precise Python tasks, including docstrings and inline documentation.

Learn how to define personas by separating system prompts from user prompts to guide model behavior, set context, and use delimiters in cmd and shell environments.

Explore delimiters to help the model separate sections using triple quotes and XML tags, illustrate with a system prompt, and compare Hamburg in Germany versus Paris in France.

Divide a task into subtasks using a step-by-step, delimiter-based approach, showing how to translate text between German and English and produce explicit substeps, and previewing examples.

Show how providing concrete examples can steer a model's output, using system prompts and emoji-based results to illustrate tone and behavior.

Explore how to control a model’s output through prompts, including format, length, style, tone, and context, with practical demo prompts and variants.

Explore advanced prompting techniques, including zero-shot prompting, train-of-thought, self-consistency, and ensemble approaches to improve coherence and avoid correlation-causation traps.

Master few-shot prompting by providing question-answer examples to teach the model patterns and enable adaptation to new tasks, illustrated with multiplication and color mapping, noting risks of inconsistency and overfitting.

Explore chain-of-thought prompts to reveal intermediate steps and improve reasoning, using train-of-thought and zero-shot variants with few-shot exemplars. Understand the benefits, limitations, and trade-offs, including transparency, interpretability, and higher costs.

Explore chain-of-thought prompting by guiding a model through step-by-step reasoning to solve a weight-average problem, including substeps and zero-shot prompts for clarity and accuracy.

Develop a train of thought workflow in coding with OpenAI, using a zero-shot chain-of-thought to solve the game of 24 via step-by-step prompts, hints, and arithmetic operations.

Explore self-consistency chain-of-thought to boost coherence by running diverse chain-of-thought experiments and using a voting algorithm to pick the most common final answer, mirroring ensemble learning in large language models.

Explore self-consistency chain-of-thought using a math reasoning example about Michael and his brother, where ages relate as two thirds, showing how multiple runs and a majority vote converge on answer.

Implement self-consistency chain-of-thought in this coding lecture to generate multiple reasoning paths for the 24 game, and return the most common final equation.

Learn prompt chaining, a technique that breaks a complex task into subtasks and uses each output to build the next prompt, enabling incremental solutions in chat, writing, coding, and planning.

Explore prompt chaining in generative ai and natural language processing, starting with a high-level outline for a magical adventure and then generating opening paragraphs that establish characters and conflict.

Explore reflection techniques that incorporate retrospective evaluation of prompts and responses, using iterative self-questioning to improve outcomes in generative AI and NLP.

Explore tree of thought, a method that generates multiple solutions, builds a complete tree of thoughts, and self-evaluates steps by combining a language model with a search algorithm.

Apply self feedback to iteratively refine a model's outputs, using repeated refinements, stop criteria, and a maximum number of iterations to prevent infinite loops, with future code implementation.

Design a system prompt and evaluation workflow to explain the key events and significance of the American Civil War. Assess breadth, depth, and impacts, then seed and revise outputs.

Explore self-feedback, where a model self-evaluates its answers, rates performance, and produces revised outputs through iterations, prompts, and feedback, guided by stop criteria, using an American Civil War example.

Explore self-critique workflows with ideation, critique, and resolution steps, where a language model analyzes ideas, selects the best one, and refines it into a final output.

Explore self-critique in coding prompts by using smart lm chain with Lange chain, installing link chain experimental, loading prompts, and evaluating multiple ideas to select the best solution.

Explore the retrieval augmented generation workflow using embeddings and a vector database to find top two relevant texts, then pass them with the user query to a large language model.

Learn how retrieval augmented generation uses a vector database and a large language model to turn three closest film results into a nicely shaped, readable answer.

Set up a vector database for retrieval augmented generation with chroma db and a movies collection, run a sample query, and format semicolon-separated titles and descriptions for a language model.

Set up a system rule and prompt to constrain the model, design user queries and structured information, and parse OpenAI chat responses to extract the desired final content.

Implement a rec function using retrieval augmented generation by querying a vector database, building system and user prompts, and using OpenAI with a model to return movie names and plots.

Develop a climate change chat bot web app using vector databases and RAC in Streamlit, where user questions trigger a large language model to answer from a parsed PDF dataset.

Build a retrieval augmented generation pipeline by extracting IPCC PDF text, cleaning headers and footers, and creating vector chunks with LangChain and ChromaDB for OpenAI-powered web apps.

Create a persistent chroma vector db, set up an IPCC collection, and index documents with unique IDs and embeddings; query the climate corpus to retrieve the top five results accurately.

Build a retrieval augmented generation webapp for climate questions by querying a vector database, joining retrieved documents, and prompting OpenAI's GPT 3.5 turbo to answer only from the provided information.

Build a climate change chatbot web app using a Python Streamlit interface connected to a prebuilt rec system and vector database, displaying raw documents and structured answers.