A Comparison of Language Model Training Techniques in a Continuous Speech Recognition System for Serbian(Conference Paper)
- aDepartment for Power, Electronic and Telecommunication Engineering, Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovića 6, Novi Sad, 21000, Serbia
- bAlfaNum Speech Technologies, Bulevar Vojvode Stepe 40, Novi Sad, 21000, Serbia
- cDepartment for Music Production and Sound Design, Academy of Arts, Alfa BK University, Nemanjina 28, Belgrade, 11000, Serbia
- dIve Andrića 1A, Odžaci, 25250, Serbia
Abstract
In this paper, a number of language model training techniques will be examined and utilized in a large vocabulary continuous speech recognition system for the Serbian language (more than 120000 words), namely Mikolov and Yandex RNNLM, TensorFlow based GPU approaches and CUED-RNNLM approach. The baseline acoustic model is a chain sub-sampled time delayed neural network, trained using cross-entropy training and a sequence-level objective function on a database of about 200 h of speech. The baseline language model is a 3-gram model trained on the training part of the database transcriptions and the Serbian journalistic corpus (about 600000 utterances), using the SRILM toolkit and the Kneser-Ney smoothing method, with a pruning value of 10−7 (previous best). The results are analyzed in terms of word and character error rates and the perplexity of a given language model on training and validation sets. Relative improvement of 22.4% (best word error rate of 7.25%) is obtained in comparison to the baseline language model. © 2018, Springer Nature Switzerland AG.
Indexed keywords
| Engineering controlled terms: | Computational linguisticsContinuous speech recognitionModeling languagesNatural language processing systemsSpeech |
|---|---|
| Engineering uncontrolled terms | Language modelLarge vocabulary continuous speech recognitionLSTMLVCSRObjective functionsRNNLMTime-delayed neural networksWord and characters |
| Engineering main heading: | Long short-term memory |
Funding details
| Funding sponsor | Funding number | Acronym |
|---|---|---|
| Ministarstvo Prosvete, Nauke i Tehnološkog Razvoja | MPNTR | |
| Provincial Secretariat for Higher Education and Scientific Research, Autonomous Province of Vojvodina | 114-451-2570/2016-02 |
-
1
Acknowledgments. The work described in this paper was supported in part by the Ministry of Education, Science and Technological Development of the Republic of Serbia, within the project “Development of Dialogue Systems for Serbian and Other South Slavic Languages”, EUREKA project DANSPLAT, “A Platform for the Applications of Speech Technologies on Smartphones for the Languages of the Danube Region”, id E! 9944, and the Provincial Secretariat for Higher Education and Scientific Research, within the project “Central Audio-Library of the University of Novi Sad”, No. 114-451-2570/2016-02.
- ISSN: 03029743
- ISBN: 978-331999578-6
- Source Type: Book Series
- Original language: English
- DOI: 10.1007/978-3-319-99579-3_54
- Document Type: Conference Paper
- Volume Editors: Potapova R.,Jokisch O.,Karpov A.
- Sponsors:
- Publisher: Springer Verlag
Popović, B.; Department for Power, Electronic and Telecommunication Engineering, Faculty of Technical Sciences, University of Novi Sad, Trg Dositeja Obradovića 6, Novi Sad, Serbia;
© Copyright 2018 Elsevier B.V., All rights reserved.
Cited by 4 documents
(2021) 2021 29th Telecommunications Forum, TELFOR 2021 - Proceedings
(2020) Telfor Journal
(2019) 27th Telecommunications Forum, TELFOR 2019