Master Azure OpenAI and ChatGPT

• Azure Machine Learning is a cloud-based platform that enables you to build, train, deploy, and manage machine learning models at scale

• You can use Azure Machine Learning to fine-tune and customize the AI models available in Azure OpenAI Service, such as GPT-3, Codex, and ChatGPT. Fine-tuning allows you to adapt the models to your specific data and tasks using a simple REST API.

• We will use for the moment the workbook feature of Azure Machine Learning studio to try our code.

In this lesson we will deploy the basic services required to develop the AI solutions for this course. For information even if you can't access Azure Open AI for the moment, the API is compatible with the traditional OpenAI API.

Text completion is a feature of Azure OpenAI Service that allows you to generate natural and fluent text responses based on a given input text or prompt. You can use text completion for various tasks such as content generation, summarization, semantic search, and natural language to code translation.

To use text completion, you need to provide a prompt that specifies what you want the model to do. The prompt can include instructions, examples, or a combination of both. The model will then try to match the context or pattern of your prompt and generate a text completion that is most likely to follow from your input.

You can use text completion with different models available in Azure OpenAI Service, such as GPT-3, Codex, and ChatGPT.

Semantic search is a feature that brings semantic relevance and language understanding to search results. It uses the context or semantic meaning of a query to compute a new relevance score over existing results and promote the most semantically relevant results to the top of the list.

Azure Cognitive Search is a cloud-based platform that enables you to build, manage, and scale search solutions for your applications. You can use Azure Cognitive Search to index, understand, and retrieve data from various sources such as documents, databases, web pages, etc.

Azure OpenAI Service is a cloud service that provides access to advanced AI models such as GPT-3.5, Codex, and ChatGPT. These models can generate natural and fluent text responses based on conversational context or input text.

You can use Azure Cognitive Search and Azure OpenAI Service together to create semantic search applications that leverage the power of large language models. For example, you can use ChatGPT to interact with your enterprise data using natural language queries and get responses based on your own data indexed by Azure Cognitive Search.

Users can use Bing search and Azure OpenAI Service together to create semantic search applications that leverage the power of large language models. For example, users can use ChatGPT to interact with their enterprise data using natural language queries and get responses based on their own data indexed by Azure Cognitive Search. However when the data is not inside the knowledge database we can leverage Azure Bing Search to get more information. Therefore in this lesson you will implement your own BING SEARCH using Azure OpenAI!

Forms Recognizer is an AI service that applies advanced machine learning to extract text, key-value pairs, tables, and structures from documents automatically and accurately. It offers general extraction models such as Read, Layout, and General document model; prebuilt models for common scenarios such as IDs, receipts, invoices; and custom models for specific document types.

You can use Azure OpenAI Service and Forms Recognizer together to create query fields that allow you to specify what fields you want to extract from your documents using natural language. For example, you can ask “What is the total amount due?” or “Who is the sender?” and get the corresponding values from your documents. Form Recognizer utilizes the capabilities of both Azure OpenAI Service and extraction models to analyze and extract the field data and return the values in a structured JSON output.

This project is a simple web application for a OpenAI-enabled document search1. It uses Azure OpenAI Service for creating embeddings vectors from documents and GPT-3 for answering questions based on the most relevant document.

The project consists of three components:

• The WebApp, which is a Streamlit app that allows users to upload documents, ask questions, and get answers using Azure OpenAI Service.

• The Redis Stack, which is a Redis server that stores the embeddings vectors of the documents and provides fast retrieval.

• The Batch Processing Azure Function, which is a serverless function that processes the uploaded documents and creates embeddings vectors using Azure OpenAI Service.

The project can be deployed on Azure or run locally using Docker or Python. It also supports optional features such as text extraction from documents using Form Recognizer and document translation using Translator.

Many thanks to Microsoft Global Black Belt Fabrizio Ruocco for the amazing code and project!