Deep Learning for NLP - Part 8

Name: Deep Learning for NLP - Part 8
Rating: 4.4 (9 reviews)

Graph Neural Networks

Created byManish Gupta

Last updated 8/2021

English

English [Auto],

What you'll learn

Deep Learning for Natural Language Processing
Graph Neural Networks
Graph convolutions
Graph pooling
Applications of GNNs for NLP
DL for NLP

Course content

1 section • 13 lectures • 2h 33m total length

Introduction2:29
Explore how graph neural networks model graph-structured data with graph convolutions, neighborhood aggregation, and graph pooling, including unsupervised graph learning and applications in multimodal NLP and recommender systems.
Graph Data5:58
Explore graph data with vertices, edges, and node features, learn adjacency and degree matrices, and examine real-world graphs from social networks to syntactic and semantic NLP graphs.
Tasks on Graph-Structured Data6:52
Explore graph tasks by predicting links and classifying nodes, then classify whole graphs using graph neural networks. Learn how filtering refines node embeddings and pooling reduces graphs for scalable representations.
Graph Filtering: Neighborhood aggregation schemes30:42
Explore graph filtering and neighborhood aggregation schemes to revise node embeddings by borrowing neighbor information via graph convolutional networks, attention, and syntactic dependency graphs.
Graph Pooling (downsampling) Introduction5:22
Learn graph pooling and downsampling to obtain compact embeddings for entire graphs or subgraphs, enabling graph and subgraph classification with topology-based, global, and hierarchical pooling methods.
Graph Pooling: Topology based pooling13:40
Explores topology-based graph pooling through multilevel clustering, detailing coarsening, base clustering, and refining, and comparing objective families like average association and normalized cut.
Graph Pooling: Global pooling19:25
Examine global pooling in graph neural networks, including set2set and set-to-set, with attention-based reading and processing blocks, and see how structural rules and W-L coloring guide order.
Hierarchical Graph Pooling: Differentiable Pooling (DiffPool)11:37
Explore hierarchical graph pooling with differentiable pooling (DiffPool) that learns soft cluster assignments and layered embeddings, building multi-layer graphs via Z and S matrices, with entropy-regularized loss.
Hierarchical Graph Pooling: gPool13:09
Explore hierarchical graph pooling with gPool, selecting informative nodes via projection onto a learnable vector to downsample graphs while preserving connectivity, with linear space complexity and good results.
Hierarchical Graph Pooling: SAGPool6:05
Learn SAGPool, a self-attention based graph pooling method that uses graph convolution to build attention masks, select top nodes, and form hierarchical graph representations for classification.
Unsupervised Learning using GNNs14:25
Explore unsupervised learning with graph neural networks: neighbor aggregation methods (mean and pooling), graph autoencoders (variational), and deep graph infomax to learn robust node representations from unlabeled data.
Some applications of Graph Neural Nets19:36
Explore practical NLP applications of graph neural nets, including semantic labeling, relation extraction, multi-event extraction, machine translation, and document time stamping, using syntactic and semantic graphs.
Summary4:21
Explore core graph neural network concepts, from topology and filtering to attention-based message passing, unsupervised learning methods like graph stage and graph autoencoders, and scalable hypergraph approaches for natural language processing tasks.

Requirements

Basics of machine learning
Basic understanding of convolution and pooling operations

Description

More and more evidence has demonstrated that graph representation learning especially graph neural networks (GNNs) has tremendously facilitated computational tasks on graphs including both node-focused and graph-focused tasks. The revolutionary advances brought by GNNs have also immensely contributed to the depth and breadth of the adoption of graph representation learning in real-world applications. For the classical application domains of graph representation learning such as recommender systems and social network analysis, GNNs result in state-of-the-art performance and bring them into new frontiers. Meanwhile, new application domains of GNNs have been continuously emerging such as combinational optimization, physics, and healthcare. These wide applications of GNNs enable diverse contributions and perspectives from disparate disciplines and make this research field truly interdisciplinary.

In this course, I will start by talking about basic graph data representation and concepts like node data, edge types, adjacency matrix and Laplacian matrix etc. Next, we will talk about broad kinds of graph learning tasks and discuss basic operations needed in a GNN: filtering and pooling. Further, we will discuss details of different types of graph filtering (i.e., neighborhood aggregation) methods. These include graph convolutional networks, graph attention networks, confidence GCNs, Syntactic GCNs and the general message passing neural network framework. Next, we will talk about three main types of graph pooling methods: Topology based pooling, Global pooling and Hierarchical pooling. Within each of these three types of graph pooling methods, we will discuss popular methods. For example, in topology pooling we will talk about Normalized Cut and Graclus mainly. In Global pooling, we will talk about Set2Set and SortPool. In Hierarchical pooling, we will talk about diffPool, gPool and SAGPool. Next, we will talk about three unsupervised graph neural network architectures: GraphSAGE, Graph auto-encoders and Deep Graph InfoMax. Lastly, we will talk about some applications of GNNs for NLP including semantic role labeling, event detection, multiple event extraction, neural machine translation, document timestamping and relation extraction.

Who this course is for:

Beginners in deep learning
Python developers interested in data science concepts
Masters or PhD students who wish to learn deep learning concepts quickly
Deep learning engineers and developers