From 0 to 1: Machine Learning, NLP & Python-Cut to the Chase

There are different approaches to using computers to solve problems. We'll compare and contrast those approaches in this section

Machine learning is quite the buzzword these days. While it's been around for a long time, today its applications are wide and far-reaching - from computer science to social science, quant trading and even genetics. From the outside, it seems like a very abstract science that is heavy on the math and tough to visualize. But it is not at all rocket science. Machine learning is like any other science - if you approach it from first principles and visualize what is happening, you will find that it is not that hard. So, let's get right into it, we will take an example and see what Machine learning is and why it is so useful.

Machine learning usually involves a lot of terms that sound really obscure. We'll see a real life implementation of a machine learning algorithm (Naive Bayes) and by end of it you should be able to speak some of the language of ML with confidence.

We have gotten our feet wet and seen the implementation of one ML solution to spam detection - let's venture a little further and see some other ways to solve the same problem. We'll see how K-Nearest Neighbors and Support Vector machines can be used to solve spam detection.

So far we have been slowly getting comfortable with machine learning - we took one example and saw a few different approaches. That was just the the tip of the iceberg - this class is an aerial maneuver, we will scout ahead and see what are the different classes of problems that Machine Learning can solve and that we will cover in this class.

We've described how to identify classification problems. This section covers some of the most popular classification algorithms such as the Naive Bayes classifier, K-Nearest neighbors, Support Vector machines and Artificial Neural Networks

Let's understand the k-Nearest Neighbors setup with a visual representation of how the algorithm works.

There are few wrinkles in k-Nearest Neighbors. These are just the things to keep in mind if and when you decide to implement it.

We have been talking about different classifier algorithms. We'll learn about Support Vector Machines which are linear classifiers.

Support Vector Machines algorithm can be framed as an optimization problem. The kernel trick can be used along with SVM to perform non-linear classification.

Artificial Neural Networks are much misunderstood because of the name. We will see the Perceptron (a prototypical example of ANNs) and how it is analogous to Support Vector Machine

Clustering helps us understand what are the patterns in a large set of data that we don't know much about. It is a form of unsupervised learning.

K-Means and DBSCAN are 2 very popular clustering algorithms. How do they work and what are the key considerations?

It is all about finding relationships in the data - sometimes there are relationships that you would not intuitively expect to find. It is pretty powerful - so let's take a peek at what it does.

Data that you are working can be noisy or garbled or difficult to make sense of. It can be so complicated that its difficult to process efficiently. Dimensionality reduction to the rescue - it cleans up the noise and shows you a clear picture. Getting rid of unnecessary features makes the computation simpler.

PCA is one of the most famous Dimensionality Reduction techniques. When you have data with a lot of variables and confusing interactions, PCA clears the air and finds the underlying causes.

Regression can be used to predict the value of a variable, given some predictor variables. We'll see an example to understand its use and cover two popular methods : Linear and Logistic regression

In this class, we will talk about some trade-offs which we have to be aware of when we choose our training data and model.

This section will help you put all your hard earned knowledge to practice! Here's a quick overview of what's coming up. 

Anaconda's iPython is a Python IDE. The best part about it is the ease with which one can install packages in iPython - 1 line is virtually always enough. Just say '!pip'

Natural Language Processing is a serious application for all the Machine Learning techniques we have been using. Let's get our feet wet by understanding a few of the common NLP problems and tasks. We'll get familiar with NLTK - an awesome Python toolkit for NLP

We'll continue exploring NLTK and all the cool functionality it brings out of the box - tokenization, Parts-of-Speech tagging, stemming, stopwords removal etc

Web Scraping is an integral part of NLP - its how you prepare the text data that you will actually process. Web Scraping can be a headache - but Beautiful Soup makes it elegant and intuitive.

Auto-summarize newspaper articles from a website (Washington Post). We'll use NLP techniques to remove stopwords, tokenize text and sentences and compute term frequencies. The Python source code (with many comments) is attached as a resource.

Code along with us in Python - we'll use NLTK to compute the frequencies of words in an article.

Code along with us in Python - we'll use NLTK to compute the frequencies of words in an article and the importance of sentences in an article.

Code along with us in Python - we'll use Beautiful Soup to parse an article downloaded from the Washington Post and then summarize it using the class we set up earlier.

Classify newspaper articles into tech and non-tech. We'll see how to scrape websites to build a corpus of articles. Use NLP techniques to do feature extraction and selection. Finally, apply the K-Nearest Neighbours algorithm to classify a test instance as Tech/NonTech. The Python source code (with many comments) is attached as a resource.

Classify newspaper articles into tech and non-tech. We'll see how to scrape websites to build a corpus of articles. Use NLP techniques to do feature extraction and selection. Finally, apply the Naive Bayes Classification algorithm to classify a test instance as Tech/NonTech. The Python source code (with many comments) is attached as a resource.

Code along with us in Python - we'll use BeautifulSoup to build a corpus of news articles

Code along with us in Python - we'll use NLTK to extract features from articles.

Code along with us in Python - we'll use KNN algorithm to classify articles into Tech/NonTech

Code along with us in Python - we'll use a Naive Bayes Classifier to classify articles into Tech/Non-Tech

See how search engines compute the similarity between documents. We'll represent a document as a vector, weight it with TF-IDF and see how cosine similarity or euclidean distance can be used to compute the distance between two documents.

Create clusters of similar articles within a large corpus of articles. We'll scrape a blog to download all the blog posts, use TF-IDF to represent them as vectors. Finally, we'll perform K-Means clustering to identify 5 clusters of articles. The Python source code (with many comments) is attached as a resource.

Code along with us in Python - We'll cluster articles downloaded from a blog using the KMeans algorithm.

Lots of new stuff coming up in the next few classes. Sentiment Analysis (or) Opinion Mining is a field of NLP that deals with extracting subjective information (positive/negative, like/dislike, emotions). Learn why it's useful and how to approach the problem. There are Rule-Based and ML-Based approaches. The details are really important - training data and feature extraction are critical. Sentiment Lexicons provide us with lists of words in different sentiment categories that we can use for building our feature set. All this is in the run up to a serious project to perform Twitter Sentiment Analysis. We'll spend some time on Regular Expressions which are pretty handy to know as we'll see in our code-along.

As people spend more and more time on the internet, and the influence of social media explodes, knowing what your customers are saying about you online, becomes crucial. Sentiment Analysis comes in handy here - This is an NLP problem that can be approached in multiple ways. We examine a couple of rule based approaches, one of which has become standard fare (VADER)

SVM and Naive Bayes are popular ML approaches to Sentiment Analysis. But the devil really is in the details. What do you use for training data? What features should you use? Getting these right is critical.

Sentiment Lexicon's are a great help in solving problems where the subjectivity/emotion expressed by a word are important. SentiWordNet is different even among the popular sentiment lexicons (General Inquirer, LIWC, MPQA etc) all of which are touched upon

Regular expressions are a handy tool to have when you deal with text processing. They are a bit arcane, but pretty useful in the right situation. Understanding the operators from basics help you build up to constructing complex regexps.

re is the module in python to deal with regular expressions. It has functions to find a pattern, substitute a pattern etc within a string.

A serious project - Accept a search term from a user and output the prevailing sentiment on Twitter for that search term. We'll use the Twitter API, Sentiwordnet, SVM, NLTK, Regular Expressions - really work that coding muscle :)

We'll accept a search term from a user and download a 100 tweets with that term. You'll need a corpus to train a classifier which can classifiy these tweets. The corpus has only tweet_ids, so connect to Twitter API and fetch the text for the tweets.

The tweets that we downloaded have a lot of garbage, clean it up using regular expressions and NLTK and get a nice list of words to represent each tweet.

We'll train 2 different classifiers on our training data , Naive Bayes and SVM. The SVM will use Sentiwordnet to assign weights to the elements of the feature vector.

Tree based models are very useful to solve a variety of classification problems. The next few sections will introduce you to decision trees, problems inherent to tree learning such as overfitting and how to use ensemble learning techniques to solve these problems. 

What are Decision Trees and how are they useful? Decision Trees are a visual and intuitive way of predicting what the outcome will be given some inputs. They assign an order of importance to the input variables that helps you see clearly what really influences your outcome.

Recursive Partitioning is the most common strategy for growing Decision Trees from a training set.

Learn what makes one attribute be higher up in a Decision Tree compared to others.

We'll take a small detour into Information Theory to understand the concept of Information Gain. This concept forms the basis of how popular Decision Tree Learning algorithms work.

ID3, C4.5, CART and CHAID are commonly used Decision Tree Learning algorithms. Learn what makes them different from each other. Pruning is a mechanism to avoid one of the risks inherent with Decision Trees ie overfitting.

Build a decision tree to predict the survival of a passenger on the Titanic. This is a challenge posed by Kaggle (a competitive online data science community). We'll start off by exploring the data and transforming the data into feature vectors that can be fed to a Decision Tree Classifier.

We continue with the Kaggle challenge. Let's feed the training set to a Decision Tree Classifier and then parse the results.

We'll use our Decision Tree Classifier to predict the results on Kaggle's test data set. Submit the results to Kaggle and see where you stand!