Multi-Source Simultaneous Speech Translation

Published: 26. 2. 2024

Dominik Macháček (ÚFAL MFF UK)

We investigate the opportunity to use multiple parallel speech signals — the original and simultaneous interpreting — as sources for translation to achieve higher quality of the simultaneous speech translation. We create an evaluation set ESIC (Europarl Simultaneous Interpreting Corpus). We analyze the challenges of simultaneous interpreting when used as an additional parallel source. Then, we investigate the robustness of multi-sourcing to transcription errors and assess the reliability of machine translation metrics when evaluating simultaneous speech translation. Last but not least, we demonstrate Whisper-Streaming, our tool that enables real-time processing of large offline speech-to-text models.

The talks will start at 2 pm at the Faculty of Mathematics and Physics, Malostranské nám. 25, 4th floor, room S1.
Please use the following link to watch the Monday lectures on-line:
https://cesnet.zoom.us/j/99354954426?pwd=NlBnMERZeW1uUGRCRmRUOEIrUTFhQT09

Date: Monday, 4 March, 2024 - 14:00

Place: MFF UK, Malostranské nám. 25, 4th floor, room S1

More info: https://ufal.mff.cuni.cz/events/multi-source-simultaneous-speech-translation