LLM Engineering in Practice with Streamlit and OpenAI

Evaluate hosting an LLM on your own infrastructure versus using an API, considering token usage and costs. Explore options like OpenAI's GPT, Google's Bert, and Gemini for an interview simulator.

Evaluate open source versus closed source models to balance customization, cost, and ease of use. Consider parameters, training data quality, and benchmarks when selecting a model for your project.

Explore tokens as the core elements of LLMs, covering tokenization, context window, token limits, and how prompt and response tokens influence pricing under OpenAI's API.

Compare hosting a model versus pay-by-token API pricing and assess cost, scalability, and a phased strategy using OpenAI's GPT models to validate concepts before self-hosting.

Develop the system prompt to guide a language model using system, user, and AI messages, shaping a realistic interview simulator with personalized prompts and feedback.

Develop a prompt and feedback system with a one-to-ten score and four levels, exploring dual-lm evaluation versus single prompts in an interview simulator and database design and schema development.

Explore the database structure behind the system, including primary keys, foreign keys, and one-to-one, one-to-many, and many-to-many relationships, with a practical schema for positions, industries, companies, and questions.

Visualize your LLM powered application's architecture with an activity diagram, mapping the start point, activities, decisions, forks, joins, and the end point for the database and the user interface.

Develop an activity diagram for an HR interview workflow, covering start, score and feedback, including personal info, job level, company choice, and a six-question interview with retry and restart options.

Design the app architecture with activity diagrams to integrate AI, begin with user information collection, craft a prompt from predefined questions, run a GPT-4 interview, score from chat history, prototype.

Learn to tune temperature and top p for diverse tasks—from brainstormed campaigns and marketing copy to precise code and data analysis—balancing creativity and accuracy.

Explore prompt engineering basics for software development by designing clear prompts, structuring outputs in JSON, and applying few-shot learning to maximize AI model performance in OpenAI playground.

Test prompt templates by replacing placeholders with dummy data to simulate real interviews and ensure an interactive, back-and-forth flow. Iterate in the OpenAI playground and prototype the flow in Streamlit.

Set up your development environment with VS Code and Python. Install the Python extension, install Streamlit, and run a hello world app to verify.

Explore streamlit's ease of use for building interactive data apps and prototypes, with built-in components and simple deployment, including HTML and CSS customization, while noting limited customization and performance considerations.

Explore common Streamlit elements from titles and headers to formatting, emoji shortcodes, colors, and LaTeX equations, with hands-on edits to App.py and live preview.

Explore Streamlit's text, markdown, and write methods for plain text, markdown, and a dictionary. See write method's versatility to render data frames and plots, and note interactive elements like buttons.

Learn to build interactive chat interfaces in streamlit using streamlets elements, including input fields, chat messages, and rich content, with real-time streaming and customizable avatars.

Discover how Streamlit reruns the entire script on each interaction and how session state remembers button states, enabling nested buttons and real-time updates.

Initialize an OpenAI client in a Streamlit project by installing OpenAI package, storing API key in a secrets file, and configuring a session state with the GPT four zero model.

Initialize session state chat history, capture user input, and append messages; send the full messages history to OpenAI API to stream and display assistant responses in a live chat.

Build a setup page to collect personal information such as name, experience, skills, level, position, and company using text inputs, text areas, radio buttons, and select boxes.

Enhance chatbot interaction by using session state to track setup completion, persist user inputs (name, experience, skills, company, position), and reveal the setup form only when incomplete.

Refine post interview feedback by adding session state tracking for user message count, setup completion, and chat completion, and implement show feedback with input limits.

Implement a get feedback button that appears after the interview completes, then generate a score out of ten and detailed feedback from the conversation history using an evaluator model.

Upload your project to GitHub to enable version control and collaboration, commit changes with an initial commit, connect a local repo, push to main, and prepare for Streamlit deployment.

Deploy your Streamlit app by hosting it on Streamlit Community Cloud, creating a requirements.txt with Streamlit, OpenAI, and Streamlit js eval, then deploy from GitHub.

Explore the interview ace tool's high level architecture, LMS integration, prompt design strategies, and key challenges like token management, hallucinations, scalability, accuracy, and security.

Explore how the Vue.js based interview app structures HR and technical tracks with company, custom company, or no company options, delivering questions from a 1500+ database via LLM prompts.

enhance hr interview prompts with a three-llm workflow that generates, humanizes, and evaluates questions in json, improving stability, memory efficiency, and cost effectiveness for production-ready interviews.

Learn how to design and evaluate prompt structures for technical interviews, including written, coding, and database questions, using pre-made databases, dynamic question generation, and targeted feedback.

Enhance JSON generation reliability with validation checks, ensuring formatted JSON, exactly six questions, and valid types; use three retries and a database fallback with clear error messaging.

Explore how large language models hallucinate, causing continuation errors, qa issues, and formatting problems, and learn strategies like enhanced prompt design and strict json output to mitigate them.

Explore direct, visual, and indirect prompt injections in llms, with real-world examples, and learn defenses like limiting sensitive data, guard commands, and strict input validation for resilient applications.

Monitor token usage across the interview process using the tick token and CL 100 base encoding, comparing with the OpenAI dashboard and Lang Smith insights to reduce costs.

Learn practical cost-reduction strategies for LLM apps, including minimizing input tokens, concise prompts, memory management with Lang Chain, smaller models, caching, retrieval augmentation, batch requests, and early stopping.

Scale your app by managing OpenAI API rate limits, TPM, RPM, RPD, and tier usage; apply horizontal scaling for load and vertical scaling for capacity.

Master building an LLM application by exploring hosting options, open source vs closed source models, API cost considerations, prompt templates, database design, and Streamlit deployment to showcase AI driven solutions.