Generative AI for NodeJs: OpenAI, LangChain - TypeScript

Explore OpenAI services with a focus on text APIs using gpt-4 and gpt-3.5, translations, image models like Dall-E, and audio options such as Whisper.

Set up your development environment for JavaScript and TypeScript, configure your api key, and learn core concepts of artificial intelligence—models, tokens, and roles—before coding.

Set up a nodejs ai project by obtaining an OpenAI key, configuring npm and dotenv, installing the OpenAI library, and running a basic chat completion.

Install TypeScript, ts-node, and node types, create a tsconfig.json with strict mode and commonjs, configure root and dist folders, and run ts-node via npm start.

Demystify how to call the OpenAI api by using the official NodeJS client or any http request, and explore parameters like model, temperature, top, and tokens.

Identify how OpenAI models power text generation, from GPT-4 and GPT-4 turbo to GPT-3.5 turbo, and how token-based context windows shape predictions.

Learn how tokens drive billing, explore tokenization with OpenAI and Hugging Face, and count tokens in prompts using the tk token library in a TypeScript workflow.

Explore OpenAI chat message roles, including user, system, and assistant, and learn how to configure system prompts to shape responses across multi-message chat completions, including JSON-structured outputs.

Explore other OpenAI chat API parameters—temperature, top p, max tokens, n, frequency penalty, and seed—and how they shape randomness, diversity, determinism, and response length.

Implement a basic console chat app that mimics ChatGPT using the OpenAI API, with history. Learn how context and token limits shape the conversation.

Set up a node TypeScript npm project for a chat app, install ts-node and OpenAI, tic token, configure .env for the API key, and create index.ts to load env vars.

Build a chat app in Node.js by importing OpenAI, creating a client, and sending user input to the OpenAI chat completions API with a system message.

Configure context by maintaining a messages history for OpenAI chat completions in a TypeScript project, using an outer createChatCompletion function to update user and assistant messages with a system prompt.

This optional lecture shows how to debug Node.js with TypeScript in VSCode, including setting up Launch.json, breakpoints, and inspecting local state with a .env file.

Manage OpenAI request context by enforcing a token limit, counting tokens, and trimming older messages to stay under thresholds like 4000 tokens for GPT-3.5 Turbo and 16000 for newer models.

Explore how OpenAI tools, or functions, extend the OpenAI API to access real-time data by invoking user-defined functions, enabling a flight assistant that can check status or reserve flights.

Create a tools folder with index.ts, configure OpenAI chat tools, and perform the first OpenAI call. Then decide to call a tool or function and issue a second OpenAI call.

Configure a tool entry in the create request and define a get time of day function. Enable auto tool calls so OpenAI decides when to invoke it.

Learn to call functions with parameters using OpenAI tools and prompts. Build a get order status function with structured parameters, argument parsing, and tool invocation to query a secure database.

Build a flight assistant dialogue with history that finds flights between two airport codes and reserves seats, returning a reservation number or not possible.

Demonstrates implementing a flight booking flow with dummy get available flights and reserve flight functions, integrated into an OpenAI chat completions workflow with tool calls and a context-driven assistant.

Explore image generation with DALL-E 2 and DALL-E 3, and work with text models like Whisper and text-to-speech to transcribe and translate or have AI read text.

Compare the two OpenAI image generation models, Dall-E 2 and Dall-E 3, and learn how to manage credits and billing in settings to access higher resolution and quality.

Learn to generate images with OpenAI in a Node.js project, using Dall-E 2 and Dall-E 3, crafting prompts, choosing styles and sizes, and saving results as URLs or local images.

Create and edit image variations using the OpenAI API in a NodeJS setup, applying the Dall-E model to generate variations and edits with masks.

Explore OpenAI's audio capabilities with Whisper for speech to text and a text to speech model, including an HD version, and how to apply them in a Node.js app.

Explore OpenAI's Whisper model for transcription, translation, and text-to-speech: transcribe audio to text, translate non-English audio to English, and generate mp3 speech from text.

Explore embeddings and their math foundations, including dot product and cosine similarity, learn to generate embeddings with OpenAI, perform a similarity search, and build a movie recommendation system.

Embeddings convert data into numerical vectors in an embedding space. Use dot product and cosine similarity to measure similarity, and explore OpenAI text embedding tree small.

Explore the OpenAI embeddings API in a TypeScript node project, using text-embedding-3-small to generate embeddings and handling single strings or string arrays, then view and save them to disk.

Generate embeddings from a json data file and save them to data_with_embeddings.json using node fs, path, and typescript generics.

Calculates similarities between embedding vectors using dot product and cosine similarity, loading embedding data, computing input embeddings, and ranking results by similarity.

Explore how embeddings measure similarities with cosine similarity and dot product in a Node.js TypeScript project, powering a basic chat assistant with OpenAI embeddings.

Design a movie recommender project that accepts a list of movie titles with descriptions and user preferences via the command line, then outputs tailored recommendations using Kaggle data.

Build a console-based movie recommender in TypeScript that uses movie embeddings to generate recommendations, loading or generating embeddings from a json cache and preparing for future vector databases.

Explore vector databases, compare chroma db and pinecone, and learn to build a data retrieval app that stores vectors and uses OpenAI to answer questions.

Explore vector databases for scalable ai data processing, learning how embeddings enable similarity search and comparing options like Pinecone, Chroma, PG vector, Redis, and Milvus.

Explore chroma db, an open source vector database written in Python that runs via Python or Docker, and learn to instantiate its server with basic Docker commands and vector operations.

Install a chroma db server using docker by pulling the chroma image, running a container with port 8000 forwarded, and verifying a nanosecond heartbeat at localhost:8000/api/v1.

Build a chroma db client in TypeScript, connect to a local server, create collections, add data with optional embeddings, and explore the API docs, preparing for OpenAI-powered embeddings.

Learn to generate OpenAI embeddings inside ChromaDB by configuring an embedding function in a new collection, using the OpenAI embedding function, npm start, and testing with data.

This lecture proposes an embeddings-based app that uses a vector database and similarity search to answer questions by injecting private data context, chunking data, and leveraging OpenAI for responses.

Build a chat app with your own data by embedding entries in a Croma DB, querying collections, and using OpenAI chat completions with context.

Pine Cone is a managed vector database with a serverless, pay-as-you-go option and metadata support. It offers a free tier and a pricing calculator to estimate cheaper costs.

Explore how to set up and configure pinecone indexes, including creating an index with 1536 embeddings and cosine metric, and using a serverless, code-driven workflow with AWS us-west-2.

Learn pinecone index operations, including listing and getting indexes, and using namespaces to partition data. Upsert vectors and run id-based queries with optional metadata for embeddings.

Build a small info chat app with pinecone using openai embeddings to store and query relevant data in a vector db.

Explore LangChain, a popular framework for building language-model powered apps, with JavaScript and TypeScript, and learn to integrate external data sources and vector DBs like Croma DB.

Explore LangChain as a framework for building language model applications, with chains and agents. Learn how it integrates OpenAI, Hugging Face, and vector databases to enable chat with private data.

initialize a separate npm project, configure tsconfig, gitignore, dot env, and vscode debug, then install lang chain to explore its complex dependencies and multi environment import system with OpenAI.

Create your LangChain application in TypeScript by wiring a chat OpenAI model, configuring gpt-3.5-turbo, temperature 0.8, max tokens 700, and invoking prompts like 'give me four good books to read'.

Explore building complex prompts with LangChain templates and messages, connect them to models via a chain and the pipe method, and handle typed inputs and errors for robust OpenAI requests.

Explore how to parse model outputs with LangChain output parsers, including string, comma-separated, and structured parsers, and learn to build chained prompts in TypeScript.

Explore retrieval augmented generation (Rag) apps to chat with private or new data using LangChain. Build pipelines with data sources, chunking, embeddings, vector databases, and relevance querying.

Build a basic rag application using lang chain with a memory vector store, openai embeddings, and a chat prompt to answer questions from hard-coded data.

Learn to load web content with LangChain's cheerio web loader, install cheerio, split documents with a recursive text splitter, and store chunks in a vector store for retrieval.

Demonstrates loading data from a pdf with LangChain’s pdf loader, creating paragraph-based chunks from a three-book pdf and answering questions from the document.

Switch to ChromaDB with LangChain in node js, installing via docker, and build a Chroma vector store from documents. Persist the books collection for reliable querying.

Explore alternatives to OpenAI using Hugging Face for open source models, including self-hosted and offline options, to avoid vendor lock-in and support text embeddings, text, image, and audio tasks.

Hugging Face is an open source machine learning and data science platform for building, deploying, and training models across multimodal tasks, with datasets, spaces, and an API.

Set up a node project with Hugging Face, obtain an access token, and generate embeddings using the Hugging Face JavaScript inference and feature extraction for model selection.

Explore how to use the Hugging Face inference API for translation and question answering, experimenting with t5-base and nlb-200 models, specifying languages and parameters, and evaluating model behavior.

Explore generating images from text using the Hugging Face API's text-to-image, selecting the stability AI stable diffusion two model, tweaking prompts and parameters, and saving results as image files.

Download and run local models with transformers js using the senova transformers package, adjust the project for ES modules and onnx runtime, and verify setup with a simple test.

Learn to run local embeddings and text generation with Xenova models, and perform offline speech recognition using whisper small, showing caching and offline computation.

Explore AI limitations and opportunities in NodeJS, review core AI concepts, and get an introduction to LangChain and vector databases to guide your next steps.