Mastering LLMs Locally using Ollama | Hands-On

Begin with an introduction to large language models and deployment challenges, then explore Olama, a tool that simplifies LLM work locally and compare it with long chain and hugging face.

Explore what large language models are, how transformers work, and deployment challenges—cost, latency, data privacy, and bias—preparing you to deploy LLMs with Ollama.

Get started with Ollama to run LLMs locally on your machine, ensuring data privacy, no cloud dependencies, and offline access with models like llama three and Mistral.

Explore the architecture of ulama, a local client-server system running through cli and http, with a cpu-based inference engine (llama cp), local storage, and model downloads from a remote repository.

Compare Ollama, LangChain, and Hugging Face to reveal their purposes and offline capabilities in local versus hub-based workflows.

Set up your local environment to work with LLMs using Ollama, follow the docs, install on Windows, Linux, and macOS, then run Gemma and explore the CLI and desktop app.

Explore the Ollama documentation to use the quick start, run models locally, and customize them with the CLI, REST API, and model library.

Demonstrates installing Ollama on Windows, Linux, and macOS, and verifying the installation with a version check. Then runs a Gemma model (1 billion parameters) locally and interacts via the terminal.

Explore the Ulama CLI and its HTTP-based interaction with the Ulama server to run models locally. Learn to pull, list, run, and create models using the Ulama base command.

Discover common Ulama commands to manage local models. Use list, ps, run, pull, show, stop, rm, create, push, and cp to interact with a registry and run models.

Demonstrates setting up Ollama Desktop, selecting locally downloaded models like Gemma and the 135 million parameter LM, and using a GUI to chat, generate code, and summarize documents.

Explore working with models in Alama and customize them with a model file. Tune generation parameters like temperature, top-k, and top-p, and learn about the registry and token count.

Learn to configure a local model with a model file, choose a base like llama three, adjust temperature, and define system and message instructions to create a custom model.

Configure how a language model generates text by adjusting parameters in the model file, including temperature, top_k, top_p, stop tokens, seed for reproducibility, output length control, and repetition penalties.

Create a custom ai model with a model file on gemma 3:1, thinkbook, using a 1b llm at temperature 0.3 with a system prompt for devops ci, cd, docker, kubernetes.

Explore the model registry in Ollama to store, version, and share your custom large language models. Push local models to the registry under your namespace for portability and collaboration.

Demonstrates pushing a customized model to the registry, including creating an account, adding a public key, and using Ollama push to publish Think rook to the registry.

Understand token count and context length, how tokens and tokenizers affect cost and performance, and how the prompt and response fit within model limits.

Explore integrating Ollama with Python to build intelligent applications using custom or pre-built models, interact with models via Ollama's REST API, and build a Streamlit front end to visualize results.

Demonstrate using the Ollama rest api to run models and send prompts from a local server. Learn to list models and generate responses with curl or Python requests.

Demonstrate the Ollama Python library for loading, running, and managing local models (llama 3, mistral, gema) from Python code, including chat, streaming responses, custom models, and Flask integration.

Demonstrates integrating code with Streamlit to build a simple user interface using only Python code for an interactive chat app, running locally with a basic chat interface and logo configuration.

Fine-tune an existing model for a specific use case, perform text pre-processing, and iteratively improve outputs, while managing versions for organized tracking.

Explore fine tuning of pre-trained models to tailor performance for specific domains, leveraging transfer learning with targeted datasets, adjusted architectures, and careful learning-rate and regularization to avoid forgetting.

Prepare a dataset for fine tuning by cleaning csv/json data with pandas, tokenizing with Bert tokenizer, enforcing 512 tokens, removing duplicates, and saving refined files for local model training.

Demonstrates fine tuning a model with a prepared dataset in Google Colab, using Hugging Face, Git LFS, and LoRA adapters for efficient training.

Manage versions of fine tuned models in Ulama to ensure reproducibility and safe rollbacks, with clear naming conventions, multiple versions in production, and shadow testing for controlled deployments.

Learn to build your first local rack for retrieval augmented generation using Ulama, and explore the rack pipeline’s key components and its complete local machine implementation.

Explore retrieval augmented generation (rag) and why it helps llms access current information, improve accuracy, enable domain-specific use without retraining, and provide transparent, up-to-date knowledge.

Index documents into chunks, convert text to embeddings in a vector database, and enable fast retrieval, augment the query with retrieved context, and generate a precise response with an llm.

Demonstrates building a Rag application with Ollama, including pulling Gemma three 1B model and Hugging Face embeddings, plus setting up streamlit UI and a retrieval QA pipeline.

Set up Ollama to run large language models locally, master deployment, registry actions, tokens, context length, fine-tuning, REST API integrations, and build apps with Python and Streamlit.