Machine Learning and Data Science with LangChain and LLMs

Explore data and analysis using Python pandas, focusing on its three data structures—series, data frames, and panels—to enable flexible, scalable data analytics.

Discover how to access data in a pandas series using index positions and labels, including single, slice, and multi-element retrieval.

Demonstrate operations on a series, including index checks, elementwise conditions, and maths or function-based computations. Define a function to add two series and visualize results with plotting.

survey data structures with a focus on data frames, two-dimensional tabular data, and creating pandas data frames from lists, dictionaries, arrays, or other frames.

Delete a column in a dataframe using del or pop, and learn how shared references between df1 and df2 cause deletions to affect both dataframes.

Explore statistical analysis on a panel with two groups, using pandas describe to summarize salaries and attributes across groups; create and manipulate pd.Series data to compare groups.

Group data by city and optionally by gender to perform aggregation, transformation, and filtration, then compute counts and display city and gender groups across examples like Cairo, Delhi, Dubai, Paris.

Master prompt templates in the Lang Chain framework by defining a template with curly brace placeholders, specifying input variables, and generating final prompts when running a chain.

Explore how semantic search uses natural language understanding and intent recognition to grasp queries and surface contextually relevant results, using entity recognition and embeddings like word2vec, GloVe, and BERT.

Explore synonym recognition, synonym handling, and contextual relationships in semantic search. Tailor results by user intent, personalization, past interactions, and preferences, foundational to semantic search.

Explore documents and embeddings in the LangChain framework, create sample data, and apply semantic search, using Google Colab and Jupyter notebooks to prototype stable diffusion applications.

Run a retrieval qa chain by formulating a query, executing qa_chain.run, and printing the result, while exploring document indexing, semantic search, and overall integration.

Design a simple calculator program that performs add, subtract, multiply, and divide on two numbers, and prints the result. Handle invalid operations by signaling an error.

Set up and connect prompts and language models using LLMChain to power a calculator that handles complex, natural language queries with an enhanced calculator function.

Discover how to implement text-to-speech output using the Google gTTS library, play audio with playsound, recognize speech, and run a speech-enabled calculator via voice input.

Set up a simple LangChain using OpenAI LLMs to analyze data, generate insights, and prepare for advanced data analysis by integrating prompts, context, and chain usage.

Explore the real estate data set by loading the csv, inspecting head, checking missing values and data types, and computing basic statistics to understand distribution before moving to data cleaning.

Learn how LangChain enables model chaining and API-powered pipelines for tasks like text generation and Q&A, and how Pinecone enables semantic vector search for text data.

Visualize statistics of text data with matplotlib and seaborn by plotting a word-count histogram with 30 bins and kde enabled, using a 10 by 6 figure to show word-count distribution.

Design and deploy robust data pipelines for supervised regression on census data to predict district median housing prices, using batch learning and a clear data flow.

Parses a housing dataset using pandas, detailing ten attributes such as longitude, latitude, housing, total rooms, total bedrooms, population, households, median income, median house value, and ocean proximity.

Plot a histogram for each numerical attribute to visualize distributions and inspect how scaling and capping affect the data. Review value counts, describe statistics, and consider implications for training.

Learn how to create a test set by randomly reserving about 20% of data, and why avoiding data snooping bias helps estimate true generalization error.

Use the row index as the housing dataset id by resetting the index, then apply train_test_split to create train and test sets, and address errors in the code.

Learn to create stable ids for records and use sklearn's train_test_split with a random_state to split the training and test sets consistently across multiple datasets, reducing sampling bias.

Apply stratified sampling with scikit-learn's stratified shuffle split to create a train-test split by income category. Compare income category proportions in the test set and full data.

Compute Pearson's r using cor and build a correlation matrix, then interpret linear relationships such as between median income and median house value, noting non-linear patterns via scatter matrix.

Examine the strong correlation between median income and median house value via a scatter plot, noting a $500k price cap and related lines, and consider removing districts to avoid quirks.

Master data preparation for machine learning by building reusable transformation functions, separating predictors from labels, and applying transformations to new data before feeding it to algorithms.

Build end to end data transformations using scikit-learn pipelines to chain simple imputer, median strategy, standard scaler, and other steps for numerical attributes, with fit and transform methods.

Compare regression models by calculating RMSE from mean squared error on training data, illustrate underfitting with linear regression, then train a decision tree regressor to assess overfitting and test-set considerations.

Use scikit-learn's GridSearchCV to automatically explore hyperparameter combinations for a random forest regressor, using a specified param grid and five-fold cross-validation with negative mean squared error as the metric.

Tune a random forest regressor with grid search cross-validation, exploring 18 hyperparameter combinations across 90 training rounds, identifying best params and estimator, and evaluating RMSE scores.

Explore randomized search versus grid search for hyperparameter tuning, using randomized search CV to explore many parameter values, and analyze feature importances from a random forest regressor to refine models.

Evaluate the final model on the test set with the full pipeline, compare RMSE, and report a 95% confidence interval to gauge generalization against cross-validation.

Shows how to display a digit from the 70,000 MNIST-style images with 784 features by reshaping to 28 by 28 and displaying with matplotlib imshow, using a 60k/10k train-test split.

Evaluate model performance with crossvalscore using cv=3 and accuracy, illustrating a dummy classifier's predictions and explaining why accuracy misleads on skewed datasets.

Explore how to use scikit-learn's decision function to obtain prediction scores, threshold them for predictions with an SGD classifier, and observe how changing thresholds impacts recall.

Compare a random forest classifier with an sgd model by plotting roc curves from cross-validated predictions, using positive class probabilities as scores, and evaluating roc auc, precision, and recall.

Implement one versus one and one versus rest multiclass classification in scikit-learn with an OVO strategy using an SGD classifier; compare to a random forest, and evaluate with cross-validation.

Explore linear regression model predictions using theta hat, x nu, and y predict, and visualize results with matplotlib plots.

Gradient descent updates parameters iteratively in the direction of steepest descent to minimize the cost function, starting from random initialization and using a learning rate to converge to a minimum.

Stochastic gradient descent speeds training by updating gradients from a single random example, enabling scalable learning, while randomness helps escape local minima with a learning rate schedule guiding convergence.

Explore learning curves by comparing a 300-degree polynomial model to linear and quadratic models on training data, illustrating how high degree polynomials fit the data.

Explore how learning curves reveal overfitting and underfitting, using cross-validation and train-test splits to compare models like quadratic versus linear, and visualize with mean squared error plots.

Train a logistic regression model to estimate iris probabilities from petal width, showing 1.6 cm boundary and how predict_proba differs from predict for iris virginica.

Explore nonlinear svm classification via polynomial feature mapping to achieve linear separability, and implement a scikit-learn pipeline with polynomial features, standard scaler, and linear svc on the moons dataset.

Explore how the polynomial kernel enables SVMs to mimic many polynomial features via the kernel trick, balancing degree choices to control overfitting and underfitting in practical models.

Explore the Gaussian RBF kernel with SVC and SVM, showing how gamma shapes the decision boundary and acts as regularization, alongside the C parameter and feature costs.

Explore hard and soft margin linear SVMs, formulating the objective to minimize 1/2 w^T w with margin constraints, and introducing slack variables to handle violations.

Demonstrates the kernel trick for a second-degree polynomial mapping in kernelized SVM, derives transformed dot products, and shows how applying the transformation to all training instances affects the dual problem.

Analyze how Gini impurity quantifies node impurity using class counts from training instances in the iris decision tree, and observe how the cart algorithm builds a binary tree.

Explain how the cart algorithm trains decision trees by greedily splitting data on a feature and threshold to minimize impurity, then recurses until max depth or no improvement.