Recently, I want to compare our algorithm with your paper in callhome result. Can you kindly provide the rttm hypothesis of callhome for us in the original paper? Thanks a lot.

Hi, for some reson my training process was interupted. I want to resume the train from the lastest ckpt and continue training on the old data. There is a para --spkv_lab: "file path of speaker vector with label and speaker ID conversion table for adaptation" . which file does it exactly mean? I tried the featlab_chunk_indices.txt but failed. I cannot find another file suitable for it... please help. Thanks

Hello, I was wondering have you evaluate with different net architecture? I modified the net according to the transformer's paper (layer numbers, heads numbers and hidden units size). And I found that the result does not get better (even worse on some unseen test wav) with the net become complicated.

Hello, I found there is a paramter named num_speakers in the train.yaml. Dose that mean the number of speaker in audio shoud equal to num_speakers? I

Hello, while I modified the layer and head num of transformer in train.yaml I got a RuntimeError. RuntimeError: shape '[128, -1, 12, 21]' is invalid for input of size 4915200

spk_loss_ratio: 0.03
spkv_dim: 256
max_epochs: 120
input_transform: logmel23_mn
lr: 0.001
optimizer: noam
num_speakers: 3
gradclip: 5
chunk_size: 150
batchsize: 128
num_workers: 8
hidden_size: 256
context_size: 7
subsampling: 10
frame_size: 200
frame_shift: 80
sampling_rate: 8000
noam_scale: 1.0
noam_warmup_steps: 25000
transformer_encoder_n_heads: 12
transformer_encoder_n_layers: 8
transformer_encoder_dropout: 0.1
seed: 777
feature_nj: 100
batchsize_per_gpu: 8
test_run: 0

I didnt go through the model structure code now. So these two parameter cannot random modified? or they are related to other para(context_size)?

Hello there,

Thanks for your efforts in open-sourcing the code, it's vital for us trying to reproduce the result presented in the paper.

Problem

But I've come across a RuntimeError when adapting the model with our private data which shows:

/*/EEND-vector-clustering/eend/pytorch_backend/train.py:186: RuntimeWarning: invalid value encountered in true_divide
  fet_arr[spk] = org / norm
...
Traceback (most recent call last):
...
RuntimeError: The loss (nan) is not finite.

Detail

After some debugging, I found the problem actually happens during the backpropagation step when there exists an entry left with zeros in the embedding layer: https://github.com/nttcslab-sp/EEND-vector-clustering/blob/b3649eed02fe4f0239f2000fb895120d3c549631/eend/pytorch_backend/train.py#L173-L186

Since the embeddings are actually loaded from the dumped speaker embeddings generated by the save_spkv_lab.py script when adapting the model, I suspect there might exist some issue in the save_spkv_lab function.

After some careful step-by-step checking with pdb, I found there is actually some silent speaker label added in the all_labels variable when dumping the speaker embeddings: https://github.com/nttcslab-sp/EEND-vector-clustering/blob/b3649eed02fe4f0239f2000fb895120d3c549631/eend/pytorch_backend/infer.py#L349-L355

Even when if torch.sum(t_chunked_t[sigma[i]]) > 0, lab can still be -1 which is considered as silent speaker acroding to code in: https://github.com/nttcslab-sp/EEND-vector-clustering/blob/b3649eed02fe4f0239f2000fb895120d3c549631/eend/pytorch_backend/diarization_dataset.py#L94-L99. (This is where makes me feels confused since it should not happen as both lab and T/t_chunked produced with info from kaldi_obj.utt2spk)

Since these silent speaker labels are -1 and the python list support negative indexing, this issue is silently ignored when dumping the embedding but will cause Exceptions when training begins.

Question

I could simply fix this issue by adding speaker label to all_labels only if lab < 0 when saving speaker embeddings and the followed training process could continue smoothly resulting in a good performing model.

But before opening any PR, I would like to know if you guys have ever come across such an issue or do you have any idea on why this will happen.

Thanks!

Currently, the following Error might arise when a trained EEND-vector model is used to do inference on an audio record with only a single speaker.

ValueError: Found array with 1 sample(s) (shape=(1, 1)) while a minimum of 2 is required by AgglomerativeClustering.

This PR fixes this issue by adding an extra verification to make sure min_n_samples is always greater than two which avoids doing clustering on one single sample.