After Updating to 1.0.3, Not Able to Output Cantonese (language = 'yue')

[clueple]

Not sure if anyone got the same outcome, when transcribing Cantonese video / mp3, the output language is written Chinese (Traditional Chinese) instead of Cantonese. I've tried transcribing Cantonese videos/audios for almost 2 weeks (still didn't see any Cantonese words generated).

It didn't happen in version 1.0.2. I'm not sure if anythings needs tweaking to make it "perform like version 1.0.2".

Any constructive comments appreciated.

[trungkienbkhn]

@clueple , v1.0.3 almost only different from 1.0.2 in silero vad model (upgrade from v4 to v5). Did you use the vad_filter option ? Could you share your code logic ?

[clueple]

@clueple , v1.0.3 almost only different from 1.0.2 in silero vad model (upgrade from v4 to v5). Did you use the vad_filter option ? Could you share your code logic ?

Thanks for your info. Actually, I did set 'vad_filter' to 'True' but I didn't specify the 'vad_parameters' (so by default it was 2 seconds (or 2000 ms)). Once I add the 'vad_parameters' = dict(min_silence_duration_ms=500) <--- 0.5 seconds, Cantonese words came out AND MIRACULOUSLY, the load time is MUCH FASTER.

I'm not sure if my guess is correct:

Since colloquial contents mostly don't follow grammar rules (e.g. a lot of run-on sentences, some might even be unrelated to the previous chunk of words), the SHORTER VAD_PARAMETERS (meaning, breaking the sentence into smaller chunks, separated with shorter silence time), the MORE ACCURATE, and the FASTER the loading process.

For example, my speech is as follows (where '...' indicates silence time):
chunk = 'What I want is ... em... I'm not sure ... is ... to get a better score."
where chunk [0] = 'what I want is'
chunk[1] = 'em'
chunk[2] = 'I'm not sure'
chunk[3] = 'is'
chunk[4] = 'to get a better score'

The above sentence, if put it in formal context, should be 'What I want is ', 'to get a better score '
(that means chunk[0] and chunk[4] will get retained as the output text).

In case of a longer VAD_PARAMETER time, it MIGHT take longer time to figure out which 'chunk' of text will give the correct output text.

The above situation will get worse in case of Cantonese (where a lot of run-on chunks in between). If you skip all those 'run-on chunks', it's like a robot speaking Cantonese (not-human-like). It might be literally meaningful, but simply doesn't sound like a native Cantonese speaker. Those kind of speech only exists in news report, public speaking, and mostly formal situations.

Listed below is my code snippet that gives a FASTER LOAD time when transcribing Cantonese using Faster Whisper 1.0.3
'''
from faster_whisper import WhisperModel
import pysubs2

model_size = "distil-large-v3"

model_size = "large-v3"

Run on GPU with FP16

model = WhisperModel(model_size, device="cuda", compute_type="float16")

or run on GPU with INT8

model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")

or run on CPU with INT8

model = WhisperModel(model_size, device="cpu", compute_type="int8")

file_name = r'edit1b_j_align_part4'
file_ext = r'mp4'
do_task = 'transcribe' # task can be 'transcribe' or 'translate'
vid_lang = 'yue' # spoken language in the video/audio; 'yue' for Cantonese
srt_lang = 'can'

segments, info = model.transcribe(f"{file_name}.{file_ext}", vad_filter=True ,beam_size=2, vad_parameters=dict(min_silence_duration_ms=500) ,language= vid_lang, word_timestamps=False, task= do_task)

segments, info = model.transcribe(f"{file_name}.{file_ext}", beam_size=5, language='yue', word_timestamps=False, task='translate')

print("Detected language '%s' with probability %f" % (info.language, info.language_probability))

for segment in segments:

print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

to use pysubs2, the argument must be a segment list-of-dicts

results = []

for segment in segments:

for word in segment.words:

segment_dict = {'start': word.start, 'end': word.end, 'text': word.word}

results.append(segment_dict)

for s in segments:
segment_dict = {'start': s.start, 'end': s.end, 'text': s.text }
results.append(segment_dict)

subs = pysubs2.load_from_whisper(results)

end_file_name = f'{file_name}_sub'

#save srt file
subs.save(f'{srt_lang}'+ end_file_name + '' + do_task+'.srt')
'''