Unofficial implementation of ECAPA-TDNN
Training a ECAPA-TDNN model took 3 days.
The classification loss is bounded by the rescaling factor s, and the number of classes.
When the loss bound is too large, a model fails to discriminate speakers.
Green - s = 1
Gray - s = 16
- Channel independent attention values (Alpha Matrix) are mostly similar along channel axis, indicating channel independent attention is unnecessary.
- Mean attention score (Mean Alpha) tends to be high when the voice is present, indicating the model is "listening" to what is being spoken. The attention score seems pretty similar to VAD(voice activity detection) score.
- Attention score is not correlated to the volume of the speech. The model sometimes focuses on how the speaker ends the speech (the maximum value on the graph). It would be interesting to investigate which portion of the speech characterizes a distinctive speaker.
Cosine similarity of speaker embedding is being close to zero during the training proces, meaning AAM (Additive Angular Margin) Loss is successful in diagonalizing the speaker identity.
Inference result seems to be discriminitive, but there was a mistake. An identity vector of a speaker is calculated as a mean of the vectors from all speechs of that speaker. As seens from below histogram, even for completely random vectors h(x) and h(y), cosine similarity between h(x) and mu(x) becomes biased.
As dimension h increases, distribution of cosine similarity between two random vector becomes a normal distribution lower standard deviation. In other words, two random vectors are more likely to be diagonal as dimension increases.
- ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification
- ArcFace: Additive Angular Margin Loss for Deep Face Recognition
- VoxCeleb
-
Get VoxCeleb statistics (file length and counts, sampling rate)
-
Build speaker tokenizer
-
Apply VAD to segment audio files
-
Run model on VoxCeleb1
-
Implement .m4a loading
-
Run model on VoxCeleb2