Generative AI: OpenAI API, Gemini, DeepSeek, and ChatGPT

Learn to use the OpenAI chat completion API in Python, from signup and API keys to environment setup, library use, pricing and cost estimation, and configuring system prompts and temperature.

This update introduces GPT four mini, cheaper and more capable than GPT 3.5 turbo, with image support, and shows how to use GPT four mini for everything by model name.

Apply an evergreen approach to future model updates by identifying GPT models, compare cost and performance, and plug their names into your code while consulting model pages for details.

Sign up for OpenAI and sign in, then open the API keys page. Create and copy a new secret key, store it securely, and never share it.

Set environment variables on Windows using setx to store your OpenAI API key, then access them in Python with os.getenv.

Set and verify openai api key as an environment variable on linux or mac using export and echo, and make it permanent by adding to bash_profile or bashrc and sourcing.

Install the openai and tiktoken libraries in your chosen environment using pip, with a beginner-friendly walkthrough in an Anaconda setup.

Explore OpenAI API pricing, compare per 1000 vs per 1 million tokens, and learn cost implications across GPT-3.5 turbo and GPT-4 variants, including input vs output tokens and context windows.

Learn to add billing details for the OpenAI API and top up your credit balance to avoid insufficient quota errors.

Quickly learn to call the OpenAI chat completions API in Python using GPT 3.5 turbo, manage API keys via environment variables, and build system and user prompts to retrieve content.

Learn how tokenization converts strings into tokens for NLP models, using split functions and delimiters, and explore word, character, and subword approaches including byte pair encoding and embeddings.

Use tiktoken to estimate prompt tokens for the ChatGPT API, calculate costs in dollars per token, and respect the maximum sequence length with GPT-3.5 Turbo.

Explores reproducibility in generative AI using a seed parameter and system fingerprint to achieve mostly consistent outputs. Note that occasional differences may occur despite the same inputs.

Explore how system prompts guide ChatGPT behavior in the OpenAI API workflow, with practical examples, code demonstrations, and prompts you can tailor.

Explore turning the ChatGPT API into a memory-enabled chatbot by managing a history of system, user, and assistant messages for persistent conversations.

Learn how the temperature parameter controls randomness in language models, from deterministic 0 to creative 2, via softmax and the Boltzmann distribution analogy to physics.

Explore frequency and presence penalties passed to the create function like temperature to control repetition and topic diversity in ChatGPT, with values from -2 to 2 and practical testing.

Explore how large language models like ChatGPT and GPT-4 drive dramatic NLP gains across extractive and abstractive tasks such as question answering, text summarization, and information extraction.

Explore how LMS change the machine learning workflow by reducing data collection, labeling, and model training, and weigh cost versus performance against traditional models like Naive Bayes.

Explore OpenAI's universal and API-specific usage policies to ensure safe, lawful, and ethical use across OpenAI API, ChatGPT labs, and related services, with privacy, safety, and safeguards enforced.

Explore China's deep seq models and the open source Deep Sea API, comparing pricing and performance with OpenAI, and learn to use chat and reasoning models in Python.

Share your feedback via suggestion box to help improve this course. Note your background, course, difficulty, missing explanations, and topics you want, including future courses like gradient boosting or transformers.

Explore prompt engineering strategies in generative ai, including unstructured to structured data, json mode, classic nlp tasks, prompt chaining, chain-of-thought prompting, tone adjustment, and sentiment analysis.

Learn to convert unstructured text into structured JSON using the ChatGPT API, system prompts, and Python tools, with prompts, delimiters, and JSON validation to automate data extraction.

Convert structured data to unstructured text with ChatGPT via the OpenAI API, automating earnings reports and product descriptions while handling JSON, markdown, and formatting challenges.

Explore JSON mode to force chat completions to return JSON by specifying response format as a JSON object and using a system prompt, while noting limitations.

Explore translation, language detection, and tone enhancement using ChatGPT and LMS, with practical prompts for multilingual customer service and stylistic tweaks like nerdy scientist or pirate styles.

Discover how sentiment analysis in natural language processing informs stock trading, using OpenAI prompts and Python functions like get_sentiment and get_trade to automate decisions.

Learn how to perform text summarization with ChatGPT using prompts, max tokens, and explain-like-I'm-five variants. Explore practical examples, including stem cell articles, for concise, readable summaries.

Demonstrate an eli5 style explanation by letting ChatGPT read a medical image and translate complex findings into simple terms, highlighting differences between gpt-4 and gpt-3.5 and the role of OCR.

Explore question answering in nlp by using a system prompt to guide ChatGPT toward truthful, concise answers, and distinguish extractive vs abstractive qa through context-driven prompts.

Demonstrate chain of thought prompting in code by breaking problems into steps, showing explicit reasoning, and comparing prompts and model behavior across math and logic tasks.

Explore semantic search by converting text, images, and tabular data into vectors, then use nearest neighbor search in embedding space to retrieve relevant documents and power recommendations and chatbots.

Learn to install and use FAISS for vector similarity search, build an index with L2 distance, add vectors, and query nearest neighbors, while noting alternatives like scikit-learn and Pinecone.

Explore OpenAI's embeddings endpoint and learn to convert text into vectors by passing a list of strings with text embedding three small model, returning a list of floats to arrays.

Leverage the embeddings API for retrieval-augmented generation, converting documents to vectors, storing them in a vector database, and using retrieved text as ChatGPT context.

Learn to create multiple text embeddings with a text embedding model, validate vector dimensions (1536), compare embeddings for different texts, and apply a get_embedding function to a dataframe.

Index normalized embeddings with faiss, ensure correct dimensionality, and retrieve context to feed a qa prompt that answers questions from the most relevant articles.

Stan Lee wins a court battle against Marvel over Spider-Man, earning a windfall with 10% of Spider-Man film profits plus shares from dvd sales and merchandise, and producer credits.

Learn what fine tuning is, when to use it for ChatGPT, and compare its pros and cons with prompt engineering and Rag, including hyperparameters and costs.

Fine-tune OpenAI models with a jsonl data file of chat messages; upload via the file endpoint, start a job, monitor progress, and deploy the fine-tuned model in chat completions.

Learn to fine-tune ChatGPT by preparing a dataset in JSON, loading it from a repo, and converting reviews and responses into the OpenAI API message format with a system prompt.

Upload your training file with the OpenAI client, create a fine-tuning job for GPT-3.5 turbo using the file ID, and monitor status, tokens, and costs.

Use your trained fine-tuning model by locating the model ID, monitoring tokens and epochs, and validating results via the completions API and ChatGPT interactions.

Explore the origins of semantic search by contrasting keyword-based ranking with meaning-based retrieval. See how bag-of-words, dimensionality reduction, and transformers map concepts into vectors for semantic matching.

Learn to use the ChatGPT API with Python to automate tasks and build ChatGPT-based technologies, including applications like chatting with a PDF, guided by building blocks for creative use.

Learn how the appendix, rebranded as faq, offers supplementary information and answers common questions; it's optional, but using the q&a helps you resolve uncertainties.

Adopt installation lectures as scalable guidelines, focusing on Python prerequisites and understanding over syntax. Use pip to install libraries, noting OpenAI Gym in reinforcement learning contexts.

Set up a Windows-based data science environment with Anaconda, installing NumPy, SciPy, Matplotlib, Pandas, NLTK, and scikit-learn, plus TensorFlow, Keras, CNTK, PyTorch, and OpenAI Gym for reinforcement learning.

Set up cross-platform development by installing numpy, scipy, pandas, matplotlib, ipython, theano, and tensorflow; use a virtual machine with Ubuntu on Windows or Mac to run deep learning libraries.

Explore why using Jupyter notebook yields no real advantage over plain Python code, and learn to print and run scripts outside notebooks for real-world consistency.

Troubleshoot code access for this course by avoiding unrelated GitHub folders, ensuring you are logged in when using the code link, and turning off VPNs to receive verification emails.

Learn how to use GitHub with git clone or download, avoid downloading files individually, and understand git basics, extra reading dot txt, and coding exercises in Python across environments.