Sentences as Trees

Lazy Programmer Team
A free video tutorial from Lazy Programmer Team
Artificial Intelligence and Machine Learning Engineer
4.6 instructor rating • 14 courses • 151,892 students

Learn more from the full course

Natural Language Processing with Deep Learning in Python

Complete guide on deriving and implementing word2vec, GloVe, word embeddings, and sentiment analysis with recursive nets

12:06:49 of on-demand video • Updated January 2021

  • Understand and implement word2vec
  • Understand the CBOW method in word2vec
  • Understand the skip-gram method in word2vec
  • Understand the negative sampling optimization in word2vec
  • Understand and implement GloVe using gradient descent and alternating least squares
  • Use recurrent neural networks for parts-of-speech tagging
  • Use recurrent neural networks for named entity recognition
  • Understand and implement recursive neural networks for sentiment analysis
  • Understand and implement recursive neural tensor networks for sentiment analysis
  • Use Gensim to obtain pretrained word vectors and compute similarities and analogies
English [Auto] Everyone and welcome back to this class. Natural Language Processing deep learning in Python Part 6 in this lecture. I'm going to show you a very simple example of how and why we can represent a sentence as a tree sort of as a warm up to the next lecture which is going to be more compact and concise. So the first thing you need to know for this lecture is what a speech tag is. Good thing we covered that earlier in this course. So not too complicated a noun is a person place or a thing a verb is an action word. For example walk run or jump a determiner has a more complicated definition but you can think of it as a word that describes a relationship with noun. Some examples are the A in my. So for example if I say a dog that has a different meaning than my dog saying my dog means the dog belongs to me but saying dog implies that the dog belongs to anyone. OK so why do we need to know parts of speech that works because that's generally how sentences are split up into trees. These are best demonstrated by example. So here's the sentence John hit the ball. So first we can see the parts of speech tag for each individual word. John is a noun. It is a verb. The is a determiner and ball is another noun. We can see that the ball goes together to form a noun phrase. We can see that hit goes with the ball to make up the verb phrase. And finally we can see that John goes with hit the ball to make up the final sentence. So we can see the language is hierarchical. If we look at very long sentences we can start to see patterns form. In particular we can see that a noun phrase is often followed by a verb phrase and a verb phrase is replaceable by a verb. So you can sort of think of them as the same. In computer science we talk about class hierarchies. So for example you might have a Shape class and from that you can have derived classes like circle or square and you can always trust that if you have a circle object it is always also an instance of shape. So it's kind of like that. So here we have this sentence the angry bear chased the frightened little squirrel. But if we replace the verb phrase with a simple verb we would still get a valid sentence. For example the angry bear ran it could also replace the noun phrase with a simple noun. And that still gives us a valid sentence. For example Bob Chase the frightened little squirrel importantly for this section we're doing a task called sentiment analysis. This means that we want to build a model to tell us how positive or negative something is. For example this movie sucks has negative sentiment whereas this movie is great has positive sentiment in this section. Our sentences will not only be parsed as a tree but each node in the tree will also have a label to tell us the sentiment. The reason we want to be able to do this is to handle the problem of negation which has traditionally been very difficult for existing in our models but with a tree you can see that even if an entire phrase is negative or positive combining it with another phrase can reverse its meaning. For example if the review was just there are slow and repetitive parts of course we would just consider that a negative review. But we need to consider the other phrase in the sentence it has just enough space to keep it interesting which we might regard as a positive. So that second the positive phrase sort of takes over the meaning of the whole sentence and that's probably thanks to the word but which we can see here has neutral sentiment. Typically when we use the word but it's in the form of a sentence like here's what I thought momentarily but this is what I think overall. So when you see the word but it's usually second part of the sentence that matters and representing the sentence as a tree allows us to model that relationship. Here's another example we can see that the word humor has very positive sentiment but it's also very far down in the tree. So the overall sentence has the potential to change its meaning and indeed we can see that the sentence starts with this film doesn't which negates the positive part of the sentence. Also notice that when we're doing sentiment analysis we don't actually care whether parts of speech are that only really helped us to parse the tree itself but once that tree has been parsed we don't need that information any longer. So that was your simple one minute introduction to how sentences can be represented as trees instead of sequences in the coming lectures. Well look at how these trees are actually represented in our data files and how we might parse them in code which will bring us to how we can create recursive neural networks.