
Build a databricks-based chatbot using vector search and embeddings from hugging face, with a rag architecture and model serving. Create a streamlit app to interact with the data.
Generate synthetic healthcare data and build a Databricks pipeline. Clean data in silver and gold layers, enable embedding and vector search for a hospital chatbot.
This lecture shows how to create a dedicated s3 bucket for source data in a Databricks workflow, configure access, and prepare folders for patient, laboratory, appointment, medical, and insurance data.
Upload daily csv datasets to an S3 bucket, structuring folders like patient data, medical history data, and lab results data, then prepare to pull with Auto Loader into Unity Catalog.
Understand how to create and manage user groups for access control in Databricks, assign roles for developers, admins, and end users, and grant catalog and schema permissions using groups.
Learn to ingest all other data into the bronze layer by parameterizing a notebook and setting up a parallel workflow across six tables.
Register a Hugging Face embedding model into the gold catalog and deploy a serving endpoint using MLflow and SDKs, validating the model before use.
Create a vector search endpoint in Databricks by preparing data, embedding, and a vector search index, then register and use a foundation LM model to answer queries with retrieved context.
Configure and run Databricks workflows with a service principal, ensuring role-based access and serverless compute. Build and commit workflow definitions via git for reliable deployment and execution.
Build and deploy a streamlit chatbot app using Databricks templates, configure the app with a serving endpoint and environment files, and customize prompts along with internal group permissions.
This comprehensive course will teach you how to develop cutting-edge AI applications by combining the power of Databricks and Large Language Models (LLMs). You will explore how to leverage Databricks for large-scale data processing, feature engineering, and model training, while integrating advanced LLMs for natural language processing (NLP) tasks such as text classification, summarization, semantic search, and conversational AI.
Through hands-on labs and real-world projects, you will gain practical experience in building intelligent systems that can understand, process, and generate human language. This course is ideal for data scientists, machine learning engineers, and developers who want to stay ahead in the rapidly evolving world of AI.
By the end of the course, you will have a strong understanding of how to architect end-to-end AI pipelines using Databricks and LLMs, deploy scalable NLP applications, and apply industry best practices for model integration and performance optimization.
Key Highlights:
Scalable data processing and ML using Databricks
NLP-powered applications with state-of-the-art LLMs
Practical, project-based learning approach
Real-world AI use cases and deployment strategies
Use Vector Search indexes to store indexes
Use workflows to refresh the data end - end on schedule basis
Use Serverless compute to refresh the data
Use Databricks Apps to deploy the application