A Tutorial on Speaker Diarization
What you'll learn
- Basic concepts in speaker diarization
- Commonly used algorithms in speaker diarization
- State-of-the-art academic advances in speaker diarization
- Coding examples of speaker diarization
- Hands-on projects with popular toolkits including SCTK, pyannote-metrics, pyannote-audio, and uisrnn
Requirements
- Basic knowledge in audio and speech processing
- Basic knowledge in machine learning and neural networks
- Basic programming in Python
- Experience with speaker recognition (it's recommended to take the Speaker Recognition course by Dr. Quan Wang first)
Description
This course is a tutorial on speaker diarization techniques.
Speaker diarization is an advanced topic in speech processing. It solves the problem "who spoke when", or "who spoke what". It is highly relevant with many other techniques, such as voice activity detection, speaker recognition, automatic speech recognition, speech separation, statistics, and deep learning. It has found various applications in numerous scenarios, such as automatic meeting transcript generation, medical record analysis, media indexing and retrieval, and second pass speech recognition.
In this course, we will first go through the basic concepts and applications of speaker diarization, followed by the scoring and metrics. Then we will introduce the unsupervised methods in speaker diarization, starting with the commonly used modularized framework, followed by an introduction to clustering algorithms, with a focus on spectral clustering and its extensions. Next, we will talk about the problems with clustering algorithms, and introduce the supervised methods in speaker diarization. We will mainly talk about 4 supervised speaker diarization approaches, i.e. UIS-RNN, PIT/EEND, TS-VAD, and DNC. Finally, we will talk about the challenges and future research directions in speaker diarization.
For those who want to dive deep in speaker diarization, we also include video lectures from top speech conferences such as ICASSP and SLT by the instructors as additional learning materials.
Apart from the lecture videos, we have included small quizzes after each lecture to help you better understand the topics we have covered in the lecture.
Also, speaker diarization is a very practical skill. Thus we have carefully prepared various coding practices and projects, to get you familiar with the most popular toolkits which are used by various researchers and scientists, including SCTK, pyannote-metrics, pyannote-audio and uisrnn.
This course would be a great fit for students, researchers, developers, or product managers who work on audio and speech processing.
Who this course is for:
- College and graduate students interested in audio and speech processing
- Researchers in computer science or signal processing domains
- Developers, system architects, and product managers for intelligent speech systems
- Enthusiasts for cool technology
Instructors
![Quan Wang](data:image/svg+xml,%3Csvg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"%3E%3C/svg%3E){kind=avatar}
- 4.4 Instructor Rating
- 70 Reviews
- 349 Students
- 2 Courses
Dr. Quan Wang is currently a Staff Software Engineer at Google, managing the Speaker, Voice & Language team, and an IEEE Senior Member. He was a former Machine Learning Scientist at Amazon Alexa team. Quan had been leading the efforts to deploy advanced speaker recognition technologies to various products at Google, making Google Home the first smart home speaker to support multiple users in the market.
Quan has authored 50+ impactful patents and papers in speaker recognition, speaker diarization, voice separation, speech detection, language recognition and speech synthesis, with 2900+ citations. Quan's work has received coverage by top tech media including VentureBeat, TechCrunch, Engage and CNET.
Quan is the author of the textbook "Voice Identity Techniques: From core algorithms to engineering practice", which was selected by the bestselling books about AI leaderboard in China, and won the Distinguished Author of Year 2020 Award.
- 4.4 Instructor Rating
- 14 Reviews
- 96 Students
- 1 Course
Dr Chao Zhang received his B.E. and M.S. degrees in Computer Science & Technology from Tsinghua University and received his PhD degree in Information Engineering at the University of Cambridge. Before joining Google, he was a Research Associate at Cambridge University and an Advisor and Speech Team Co-leader at JD AI Research. He has published 60 peer-reviewed papers in speech and language processing and received the best student paper awards from ICASSP 2014, ASRU 2019, and SLT 2021. He is also a Visiting Fellow at Cambridge University and a member of multiple technical committees.