A Tutorial on Speaker Diarization

This course is a tutorial on speaker diarization techniques.

Speaker diarization is an advanced topic in speech processing. It solves the problem "who spoke when", or "who spoke what". It is highly relevant with many other techniques, such as voice activity detection, speaker recognition, automatic speech recognition, speech separation, statistics, and deep learning. It has found various applications in numerous scenarios, such as automatic meeting transcript generation, medical record analysis, media indexing and retrieval, and second pass speech recognition.

In this course, we will first go through the basic concepts and applications of speaker diarization, followed by the scoring and metrics. Then we will introduce the unsupervised methods in speaker diarization, starting with the commonly used modularized framework, followed by an introduction to clustering algorithms, with a focus on spectral clustering and its extensions. Next, we will talk about the problems with clustering algorithms, and introduce the supervised methods in speaker diarization. We will mainly talk about 4 supervised speaker diarization approaches, i.e. UIS-RNN, PIT/EEND, TS-VAD, and DNC. Finally, we will talk about the challenges and future research directions in speaker diarization.

For those who want to dive deep in speaker diarization, we also include video lectures from top speech conferences such as ICASSP and SLT by the instructors as additional learning materials.

Apart from the lecture videos, we have included small quizzes after each lecture to help you better understand the topics we have covered in the lecture.

Also, speaker diarization is a very practical skill. Thus we have carefully prepared various coding practices and projects, to get you familiar with the most popular toolkits which are used by various researchers and scientists, including SCTK, pyannote-metrics, pyannote-audio and uisrnn.

This course would be a great fit for students, researchers, developers, or product managers who work on audio and speech processing.