Sentiment Analysis, Beginner to Expert

In order to perform analysis on textual data, the goals of analysis should be clear in mind. Objective content has factual information and is used for classifying documents based on their similarity i.e., document classification or categorization. While in case of Subjective analysis, the dataset is assumed to have subjectivity or there has to be a filter that would only pass the content that shows subjectivity. In subjective content analysis i.e., sentiment analysis the documents are classified together not based on their similarity but based on their sentiment similarity. For example, the two documents may be about movies and furniture but if both have positive subjectivity, they are going to be classified as one label that is positive.

In this lecture we discuss the difference between the sentiment analysis and opinion mining. The two terms are used as synonyms but there is a slight difference between the two. Opinion mining is the extraction of user opinions from within the reviews. Different patterns matching can be used for it where Parts of speech (POS) tagging is found to be effective. Whereas, sentiment analysis is the process of evaluating all those user opinions and analyzing them to provide summarized information.

Opinion mining is generated represented with the help of quintuple OM(P, A, O, T, H). P is the product or main subject. Complex subjects are not discussed directly but instead through the aspects or features. A represent the aspect of a product, subtopic of a topic or the attribute of an issue. O is the opinion expressed. T is the time at which the opinion was given while H is the person giving the opinion or the opinion source. We may not be picking all these from every review and not every review is going to have them all. But this is generally everything that we may look for in a review.

Sentiment analysis is performed on subjective data. Such data may be acquired from social media platforms, blogs, forums, e-commerce and review websites and we can also scrap other types of webpages.

Sentiment analysis can be performed with 3 different types of techniques. 1) Treating it as a classification problem, given the labeled training data, a classifier can be trained to and it will help to predict the label for an unseen document. 2) Lexicon or dictionary based approach is used with the training data is not available. A general language dictionary is used to find the orientation and strength of each word in a review. 3) A combination of the two approaches can also be used.

The outcome of sentiment analysis can either be binary labels that is 0 for negative and 1 for positive. In case of being trained on multiple labels, it becomes multi-class classification problem and the possible labels may be strong negative, weak negative, neutral, weak positive and strong positive. The outcome can also be in the form of a summary to highlight the key points discussed in the thousands of reviews. Summary can either be extractive or abstractive. Extractive summary picking important sentences from all the reviews and presenting them as a summary. Abstractive summary is the accumulating all the information from all the reviews and generating new sentences to write a summary.

This lecture discusses the subdomains and open challenges in various specific dimensions of sentiment analysis. Sentiment analysis is not attracting much focus as its being around for 20 years now, however different subdomains have emerged from it and are considered for research and building useful applications.

In this lecture, we discuss the text preprocessing steps that would normalize text to its basic form. The idea is to reduce unwanted tokens as features while combining others through case normalization, stemming and lemmatization.

In this lecture, we implement text normalization or preprocessing steps that are converting text to lower case, removing white spaces and removing unwanted punctuations.

In this lecture, we implement stopwords removal from the corpus through a predefined list of stopwords. We compare the corpus words to the stopwords list and if there is a match, that word is dropped as a stopword.

This lecture can be considered a crash course on regular expressions. It is very handy in removing domain specific stopwords. In this lecture, we quickly go through all of the frequently used patterns and characters for writing regular expressions. These sequences, when provided to a function can be dealt with accordingly.

In this lecture we implement regular expressions to get rid of the domain specific stopwords and the source related unwanted characters. In this case, the unwanted tokens is the html code from the webpages.

In this lecture, we continue with the text normalization or preprocessing steps and implement stemming and lemmatization techniques to reduce words to their basic forms, so that the number of features could be reduced.

In this lecture, we apply Parts-of-speech tagging to textual data so that we can find the parts of speech of each word. It may help to look for specific parts of speech only as in sentiment analysis we are not interested in all the words.

In this lecture we use the proper mechanism for text segmentation into sentences and then tokenizing sentences into words. So far, we were splitting on dot (. ) to get sentences and splitting on space ' ' to get words. However, that is not the appropriate way of doing it, because dot and space can be used for other purposes as well. Therefore, NLTK library has functions to deal with it smoothly.

In this lecture, we use the requests package to read content from webpages. It helps with classes and functions to create a new session and then read data from a given URL and present it as text. The content is going to have a lot of unwanted code and has to be cleansed for filtering off unwanted code.

In this lecture we discuss the different text representation schemes that would help to structure the unstructured text. The words are used as features and are represented with columns while the documents are represented in rows. The machine learning models perform different evaluations on numeric data and therefore, we need to convert the textual data into numbers. A value in the matrix is the numerical representation of that feature in that document.

In this lecture we discuss the bag of words approach used with structuring the unstructured data. It compromises the position related information to reduce the number of features. Bag-of-words can be slightly relaxed with using higher-order ngrams. Generally, ngrams upto trigrams are used.

In this lecture we discuss the other represenation schemes. In binary representation, there are 0s and 1s to suggest whether a word occurs in a document or not, irrespective of its frequency. In log frequency, the values are slightly dampened towards the high ends. In case of Tf-idf representation, which is more frequently used, normalized term frequency is multiplied with its inverse document frequency. The words having high frequency in a document has high tf value while a word occuring in most of the documents have a low idf value. Combining the two factors helps tf-idf to focus on significant values only while ignoring the noise words and the stopwords.

In this lecture, we take a corpus of one document to see how it is converted into structured format.

In this lecture, we convert a multi document corpus into a structured format. The objective of taking these baby steps is to manually cross varify the values we are getting, after converting textual data to numbers.

In this lecture, we tune different parameters available with the CountVectorizer. Suitable values of these parameters are to be found for a given dataset.

In this lecture, we use Tfidf scheme for text vectorization. It is generally preferred over frequency or binary representation, however, a better choice can only be made when the data and the task is known. All the parameters that we tried with CountVectorizer can also be used with tfidf vectorizer.

In this lecture, a dummy dataset with few short documents is structured from text to vectors with numbers.

In this lecture use download a dataset from the UCI repository, having raw text. Using vectorization scheme the raw text is converted into structured format.

This lecture introduces the need for machine learning as a kind of inductive learning. It is a kind of learning in which the model learns from patterns in the input data. It is highly desired for automation as it reduces human effort and manual involvement by a great amount.

This lecture overviews the definitions of machine learning and explains them with the help of an example. For those who are new to machine learning, it will serve to lay the foundation based on which the coming concepts of machine learning can be grasped.

Machine learning is a multi-disciplinary area and therefore, needs a basic understanding of multiple subject areas to master it. Probability and statistics are of key importance among others. There is a thin boundary between data mining and machine learning. Generally, it is said that if only patterns are recognized from the data then it is data mining but if those patterns are used for automatic decision making, then it is machine learning. Neurocomputing is important in the sense that many well-performing models come from that domain. That's why most of us are aware of the word deep learning.

There are different types of machine learning techniques. However, the most basic ones are supervised and unsupervised learning. Supervised learning is learning with labels available and has two types i.e., classification and regression. Unsupervised learning is learning without labels and is called clustering. The other types of learning are reinforcement learning and recommender systems. Of course, there are many other types of techniques that arise from these basic forms of learning e.g., semi-supervised learning, online learning, multi-task learning, meta-learning, etc.

In this lecture, we discuss pattern recognition which lies at the heart of machine learning with the help of few examples.

This lecture gives elaborates on the end to end pipeline of any machine learning project. What are the different components and how they collaborate to provide a working model. Since, we are also going to apply machine learning techniques on textual data, therefore, it is very important for us to be aware of the challenges and advantages of using ML approaches. For example, later in this module, we will be using classification algorithms which are supervised machine learning approaches. The pipeline helps us to be aware of issues that may arise at different stages of a  project and how address them.

In this lecture, we briefly overview machine learning to set the stage for sentiment classification using supervised learning or classifiers which is a kind of machine learning techniques.

In this lecture, we discuss supervised learning that is also called Classification. We briefly touch on the different types of classifiers from different perspectives.

In this lecture, we discuss 4 simple classification algorithms that are K-nearest neighbors classifier, naive bayes classifier, decision tree classifier and linear classifier. The background working is briefly elaborated to get an idea of how differently they work to solve the same problem, which is to identify the label for an unseen document given some training data.

In this lecture, we demonstrate the implementation of different classifiers on textual data with the default settings. The main concept of these classifiers is explained in the theoretical part. The lecture demonstrate how to use them in python with the sklearn library.

In this lecture, we demonstrate the implementation of classifiers while tuning some parameters to see how it effects the process of document classification.

In this lecture, we apply the classifiers we studied on a dataset downloaded from the UCI repository. The dataset is about people names as documents and their badge sign as positive or negative being the label.

In this lecture, we have performed Sentiment Analysis with the help of a classifier. The classifier is trained on the labeled training data where we kept only few reviews aside to verify how good the model is doing in predicting the labels for the unseen documents or reviews.

As an opening lecture of a new module, we discuss what we are going to focus on in this module which is validation and evaluation. In the validation we separate the dataset into training and testing part while in evaluation, we are going to use an evaluation measure to check the performance of a model.

In this lecture, we elaborate on the three basic forms of cross validation that are K-Fold cross, leave-p-out and hold-out cross validation. K-Fold is generally preferred, however, with limited data leave-p-out can be used while in case of massive datasets hold-out approaches can be used.

In this lecture, we are separating the dataset into training and test part by calling the function train_test_split, which gives us two sets of documents and labels. We are deciding the size of the two sets with the parameter train_size and suggesting it to shuffle the data before separation through the parameter shuffle = True.

In this lecture, we use K-fold cross validation to separate the dataset into K parts where the K-1 parts are used for traing the model while the K-th part is used for testing the model. The experiements are conducted K times while the evaluation is averaged across the K experiments.

In this lecture we used the Leave one out approach for validation. It is a specific case of Leave-p-out validation where the value of p = 1.

In this lecture, we discuss the evaluation measures that are accuracy, precision, recall and F1-score.

In this lecture, we implement precision, recall and f1 score for a model to evaluate it against these metrics.

In this lecture, we discuss confusion matrix as an evaluation technique which doesn't give us an accumulative score like other parameters but instead helps us to find where the problem is.

In this lecture, we implement confusion matrix to identify the number of correctly and wrongly classified instances for each label. This is particularly helpful to in case the model is biased towards one label against others.

In this lecture, we put everything from this module together. Using two different classifiers we build two models and train them. Using K-fold cross validation, the evaluation of the two models is compared.

In this lecture, we discuss WordNet as a dictionary that can be used for lexicon based sentiment analysis, although that is not its soul purpose. WordNet is a graph with most of the words of the language and they are linked through edges representing similarity. The words closer together are closer in meaning.

In this lecture, we will perform lexicon based sentiment analysis with the help of WordNet dictionary.

In this lecture, we use the sentiWordnet dictionary for performing sentiment analysis on a review.

In this lecture, we discuss the use of TextBlob which is another very useful library for performing lexicon based SA.

In this lecture, we use TextBlox lexicon dictionary for sentiment analysis.

In this lecture, we implement SA with the help of an NLTK libarary called Vader i.e., Valence Aware Dictionary & sEntiment Reasoner.

In this lecture, we introduce topic modeling as an information extraction model. It will be used to identify aspects from the given data. Aspects for commercial product reviews are expected to be features of the product. Topic modeling does this for us in an unsupervised manner. Therefore, it can be directly applied to any domain as there aren't much labeled training data on it.

In this lecture, we discuss the working in the background of topic modeling. It will help you write your own code and modify it according to the new research developments because the library has only the implementation for the basic form of topic modeling while it has hundreds of variations.

In this lecture, we discuss the role of hyperparameters and how they can be used to modify the behavior of a model.

In this lecture, we implement topic model and use it to extract useful topics from the data. Where the topics are represented by a group of contextually correlated words.

In this lecture we use topic modeling for extracting useful concepts from a UCI repository dataset.

In this lecture, we continue with basic code of topic modeling and modify its behavior through its hyperparameters.

Topic modeling works as a batch processing model where all the data has to be available before hand. In this lecture, we tune topic modeling for online processing so that it can accommodate documents that may be added at the later stages.

In this lecture, we first use topic modeling to identify aspects from the data that are represented as topics. Then we perform lexicon based sentiment analysis at the topic or aspect level, which gives us the sentiment scores of each aspect. Therefore, it called aspect-based sentiment analysis.

In this lecture, we introduce the basic concept of neural networks and deep neural networks.

In this lecture, we discuss the propagation function and hyperparameters of neural networks. The propagation function is the function used at each neuron that converts the input of that neuron to an output. The output of each neuron is consumed by the neurons of the next layer unless it is the last layer where the output of the neurons is considered to be the predicted output.

In this lecture, we discuss the process of backpropagation and cost function that helps to adjust the weights associated with the connections of the neural network.

In this lecture, we define a simple 3 layered neural network. Most of the parameters are kept at default settings. We used a dummy dataset of 7 instances to train the model and then two more instances to test the model. The input instances have two dimensions while since it was binary classification one one neuron was used at the output layer.

In this lecture, we had code implementation of Neural networks to be used with a dummy dataset. We particularly had focus on the input shapes of the data and why this is very important for neural networks. Note: Most of the NN/DNN code errors are because of the data structure rather than the actual model.

The lecture discusses the use of training data for training a neural network. We used a randomly generated dataset for this purpose. Then the model is evaluated with accuracy, precison, recall and F1-score measures.

In this lecture, we use the IMDB movie reviews of 50,000 records to perform sentiment analysis using neural network as a binary class classification problem.

The lecture highlights the issue faced in deep neural networks with multiple layers, caused by the small values of the propagation function used. Their derivatives were giving values that were soo small that they would hardly modify the weights of the earlier hidden layers. It was one of the main hurdles towards training good models.

In this lecture we convert our neural network into a deep neural network by increasing the number of hidden layers. We also modified the number of nodes per layer.

In this lecture, we introduced the dropout layers to avoid overfitting as model get complex with more number of layers. Finally, we tried a different type of optimizer i.e., converting sgd to its faster version which is 'rmsprop'.

In this lecture, we discuss the role of convolutional and pooling layers in the deep neural network. CNNs are generally perform better than the basic deep neural networks that are having dense layers.

This lecture introduces a Convolutional neural network architecture. We added A Conv1D layer, a MaxPooling1D layer and a Flatten layer to our model. CNN models are considerably good for text and very good for images, however to take full advantage of its proximity based convolutional layer, we first need to encode our input into sequence of integers. We will do that in the next module.