Аналитический обзор методов решения проблемы малых наборов данных при создании систем автоматического распознавания речи для малоресурсных языков
Ключевые слова:
малоресурсные языки, аугментация речевых данных, перенос знаний, машинное обучение, языковые корпусаАннотация
В статье рассматриваются основные методы решения проблемы малых наборов обучающих данных для создания автоматических систем распознавания речи для так называемых малоресурсных языков. Рассматривается понятие малоресурсных языков и формулируется рабочая дефиниция на основании ряда работ по этой тематике. Определены основные трудности, связанные с применением классических схем автоматического распознавания речи к материалу малоресурсных языков, и очерчен круг основных методов, использующихся для решения обозначенных проблем. В статье подробно рассматриваются методы аугментации данных, переноса знаний и сбора речевого материала. В зависимости от конкретной задачи, выделяются методы аугментации аудиоматериала и текстовых данных, переноса знаний и мультизадачного обучения. Отдельный раздел статьи посвящен существующему информационному обеспечению, базам данных и основным принципам их организации с точки зрения работы с малоресурсными языками. Делаются выводы об оправданности методов аугментации данных и переноса знаний для языков с минимальным информационным обеспечением. В случае полного отсутствия данных для конкретного языка и родительских моделей структурно схожих языков предпочтительным вариантом является сбор новой базы данных, в том числе, при помощи краудсорсинга. Многозадачные модели переноса знаний оказываются эффективными в том случае, если исследователь располагает набольшими наборами данных. Если доступны данные по языку с достаточными ресурсами, предпочтительной является работа с языковой парой. Сделанные в результате данного обзора выводы в дальнейшем предполагается применить при работе с малоресурсным карельским языком, для которого авторы статьи создают систему автоматического распознавания речи.
Литература
1. Magueresse A., Carles V., Heetderks E. Low-resource Languages: A Review of Past Work and Future Challenges // arXiv preprint arXiv:2006.07264. 2020. pp. 1–14.
2. Joshi P., Santy S., Budhiraja A., Bali K., Choudhury M. The State and Fate of Linguistic Diversity and Inclusion in the NLP World // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. pp. 6282–6293.
3. Bender E.M. On achieving and evaluating language-independence in NLP. Linguistic Issues in Language Technology. 2011. vol. 6. no. 3. pp. 1–26.
4. Ponti E.M., O’Horan H., Berzak Y., Vulic I., Reichart R., Poibeau T., Shutova E., Korhonen A. Modeling language variation and universals: A survey on typological linguistics for natural language processing // Computational Linguistics. 2019. vol. 45. no. 3. pp. 559–601.
5. Krauwer S. The basic language resource kit (BLARK) as the first milestone for the language resources roadmap // Proceedings of International workshop on speech and computer (SPECOM-2003). 2003. pp. 8–15.
6. Berment V. Méthodes pour informatiser des langues et des groupes de langues «peu dotées». Doct. Diss. Grenoble, 2004.
7. Cieri Ch., Maxwell M., Strassel S., Tracey J. Selection criteria for low resource language programs // Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). 2016. pp. 4543–4549.
8. Tsvetkov Y. 2017. Opportunities and challenges in working with low-resource languages. Presentation. Carnegie Mellon University, June 22, 2017.
9. Романенко А.Н. Робастное распознавание речи для низко-ресурсных языков. дис. канд. техн. наук: 05.13.11 Ульм. 2020 (на правах рукописи).
10. Мурадова А.Р. Как исчезают языки и как их возрождают // Языковое разнообразие в киберпространстве: российский и зарубежный опыт. Сборник аналитических материалов М.: МЦБС, 2008. С. 70–75.
11. Хусаинов А.Ф., Сулейманов Д.Ш. Система автоматического распознавания речи на татарском языке // Программные продукты и системы. 2013. №4. C. 31–34.
12. Израилова Э.С. О создании фонетико-акустической базы в рамках синтеза чеченской речи // Компьютерная лингвистика и обработка естественного языка. 2017. №2. C. 111–115.
13. Boyko T., Zaitseva N., Krizhanovskaya N., Krizhanovsky A., Novak I., Pellinen N., Rodionova A. The Open corpus of the Veps and Karelian languages: overview and applications // KnE Social Sciences. 2022. vol. 7. no. 3. pp. 29–40.
14. Марковников Н.М., Кипяткова И.C. Аналитический обзор интегральных систем распознавания речи // Труды СПИИРАН. 2018. Вып. 58. C. 77–110.
15. Besacier L., Barnard E., Karpov A., Schultz T. Automatic speech recognition for under-resourced languages: A survey // Speech communication. 2014. vol. 56. pp. 85–100.
16. Карпов A.A., Верходанова В.О. Речевые технологии для малоресурсных языков мира. // Вопросы языкознания. 2015. № 2. С. 117–135.
17. Sabou M., Bontcheva K., Derczynski L., Scharl A. Corpus Annotation through Crowdsourcing: Towards Best Practice Guidelines // Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). 2014. pp. 859–866.
18. Arora S., Arora K.K., Roy M.K., Agrawal S.S., Murthy B.K. Collaborative speech data acquisition for under resourced languages through crowdsourcing // Procedia Computer Science. 2016. vol. 81. pp. 37–44.
19. Schultz T., Schlippe T. GlobalPhone: Pronunciation Dictionaries in 20 Languages // Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). 2014. pp. 337–341.
20. Strassel S., Tracey J. LORELEI Language Packs: Data, Tools, and Resources for Technology Development in Low Resource Languages // Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), 2016. pp. 3273–3280.
21. Simpson H., Cieri Ch., Maeda K., Baker K., Onyshkevych B. Human language technology resources for less commonly taught languages: Lessons learned toward creation of basic language resources // Collaboration: interoperability between people in the creation of language resources for less-resourced languages. 2008. vol. 7. pp. 7–11.
22. Do V.H., Chen N.F., Lim B.P., Hasegawa-Johnson M.A. Acoustic Modeling for Under-resourced Language using Mismatched Transcriptions // International Journal of Asian Language Processing. 2017. vol. 27. no. 2. pp. 141–153.
23. Hasegawa-Johnson M.A., Jyothi P., McCloy D., Mirbagheri M., Liberto, di G.M. Das A., Ekin B., Liu Ch., Manohar V., Tang H., Lalor E., Chen N.A. Hager P., Kekona T., Sloan R., Lee A.K.C. ASR for under-resourced languages from probabilistic transcription // IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2017. vol. 25. no. 1. pp. 50–63.
24. Yu C., Kang M., Chen Y., Wu J., Zhao X. Acoustic modeling based on deep learning for low-resource speech recognition: An overview // IEEE Access. 2020. vol. 8. pp. 163829-163843.
25. Ko T., Peddinti V., Povey D., Khudanpur S. Audio augmentation for speech recognition // Proceedings of the 16th Annual Conference of the International Speech Communication Association. 2015. pp. 3586-3589.
26. Rebai I., BenAyed Y., Mahdi W., Lorré J.P. Improving speech recognition using data augmentation and acoustic model fusion // Procedia Computer Science. 2017. vol. 112. pp. 316–322.
27. Park D.S., Chan W., Zhang Y., Chiu C.C., Zoph B., Cubuk E.D., Le Q.V. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition // Proceedings of Interspeech. 2019. pp. 2613-2617.
28. Hartmann W., Ng T., Hsiao R., Tsakalidis S., Schwartz R. Two-Stage Data Augmentation for Low-Resourced Speech Recognition // Proceedings of Interspeech. 2016. pp. 2378-2382.
29. Jin Z., Finkelstein A., DiVerdi S., Lu J., Mysore G.J. Cute: A concatenative method for voice conversion using exemplar-based unit selection // Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'16). 2016. pp. 5660-5664.
30. Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair Sh., Courville A., Bengio Y. Generative adversarial nets // Advances in Neural Information Processing Systems 27 / Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, K. Q. Weinberger (eds.). pp. 2672–2680 // Curran Associates, Inc., 2014.
31. Hsu Ch.-Ch., Hwang H.-T., Wu Y.-Ch., Tsao Y., Wang H. Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks // arXiv preprint arXiv:1704.00849. 2017. pp. 1–5.
32. Kameoka H., Kaneko T., Tanaka K., Hojo N. StarGAN-VC: non-parallel many-to-many Voice Conversion Using Star Generative Adversarial Networks // Proceedings of 2018 IEEE Spoken Language Technology Workshop (SLT'18). 2018. pp. 266–273.
33. Gokay R., Yalcin H. Improving low resource turkish speech recognition with data augmentation and TTS // Proceedings of 2019 16th International Multi-Conference on Systems, Signals and Devices (SSD). 2019. pp. 357–360.
34. Shen J., Pang R., Weiss R.J., Schuster M., Jaitly N., Yang Z., Chen Z., Zhan Y., Wang Y., Skerrv-Ryan R., Saurous R.A., Agiomvrgiannakis Y., Wu Y. Natural TTS Synthesis by Conditioning WaveNet on Mel SpectrogramPredictions // Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018. pp. 4779–4783.
35. Dua M., Kadyan V., Banthia N., Bansal A., Agarwal T. Spectral warping and data augmentation for low resource language ASR system under mismatched conditions // Applied Acoustics. 2022. vol. 190. 108643.
36. Du C., Yu K. Speaker augmentation for low resource speech recognition // Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2020. pp. 7719-7723.
37. Bagchi D., Wotherspoon Sh., Jiang Zh., Muthukumar P. Speech Synthesis as Augmentation for Low-Resource ASR // arXiv preprint arXiv:2012.13004. 2020. pp. 1–4.
38. Hsu B.J. Generalized linear interpolation of language models // Proceedings of 2007 IEEE workshop on automatic speech recognition & understanding (ASRU). 2007. pp. 136–140.
39. Kurimo M., Enarvi S., Tilk O., Varjokallio M., Mansikkaniemi A., Alumäe T. Modeling under-resourced languages for speech recognition // Language Resources and Evaluation. 2017. vol. 51. no. 4. pp. 961–987.
40. Fadaee M., Bisazza A., Monz Ch. 2017. Data Augmentation for Low-Resource Neural Machine Translation // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017. pp. 567–573.
41. Şahin G.G. To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP // Computational Linguistics. 2022. vol. 48. no. 1. pp. 5–42.
42. Kobayashi S. Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations // Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Vol. 2 (Short Papers). 2018. pp. 452–457.
43. Kumar V., Choudhary A., Cho E. Data Augmentation using Pre-trained Transformer Models // Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems. 2020. pp. 18–26.
44. Ogawa A., Tawara N., Delcroix M. Language Model Data Augmentation Based on Text Domain Transfer // Proceedings of Interspeech. 2020. pp. 4926–4930.
45. Sennrich R., Haddow B., Birch A. Improving Neural Machine Translation Models with Monolingual Data // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016. pp. 86–96.
46. Shahnawazuddin S., Nagaraj A., Kunal K., Aayushi P., Waquar A. Voice Conversion Based Data Augmentation to Improve Children Speech Recognition in Limited Data Scenario // Proceedings of Interspeech 2020. pp. 4382–4386.
47. Tachibana H., Uenoyama K., Aihara S. Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention // Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018. pp. 4784–4788.
48. Edunov S., Ott M., Auli M., Grangier D. Understanding Back-Translation at Scale // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. pp. 489–500.
49. Fadaee M., Monz Ch. Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. pp. 436–446.
50. Karakanta A., Dehdari J., Genabith, van J. Neural machine translation for low-resource languages without parallel corpora // Machine Translation. 2018. vol. 32. no. 1-2. pp. 167–189.
51. Xia M., Kong X., Anastasopoulos A., Neubig G. Generalized Data Augmentation for Low-Resource Translation // Proceedings of the 57th Annual Meeting of the ACL. 2019. pp. 5786–5796.
52. Wang D., Zheng T.F. Transfer learning for speech and language processing // Proceedings of 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). 2015. pp. 1225–1237.
53. Gauvain J.-L., Lee C.-H. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains // IEEE Transactions on Speech and audio processing. 1994. vol. 2. no. 2. pp. 291–298.
54. Wet, de F., Kleynhans N., Compernolle, van D., Sahraeian, R. Speech recognition for under-resourced languages: Data sharing in hidden Markov model systems // South African Journal of Science. 2017. vol. 113. no. 1–2. pp. 1–9.
55. Woldemariam Y. Transfer Learning for Less-Resourced Semitic Languages Speech Recognition: the Case of Amharic // Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL). 2020. pp. 61–69.
56. Thai B., Jimerson R., Arcoraci D., Prud'hommeaux E., Ptucha R. Synthetic data augmentation for improving low-resource ASR // Proceedings of 2019 IEEE Western New York Image and Signal Processing Workshop (WNYISPW). 2019. pp. 1–9.
57. Eberhard O., Zesch T. Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages // Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021). 2021. pp. 208–212.
58. Tachbelie M.Y., Abate S.T., Schultz T. Development of Multilingual ASR Using GlobalPhone for Less-Resourced Languages: The Case of Ethiopian Languages // Proceedings of Interspeech. 2020. pp. 1032–1036.
59. Tachbelie M.Y., Abate S.T., Schultz T. Multilingual speech recognition for GlobalPhone languages // Speech Communication. 2022. vol. 140. pp. 71–86.
60. He D., Lim B.P., Yang X., Hasegawa-Johnson M.A., Chen D. Improved ASR for under-resourced languages through multi-task learning with acoustic landmarks // Proceedings of Interspeech. 2018. pp. 2618-2622.
61. Fantaye T.G., Yu J., Hailu T.T. Investigation of Various Hybrid Acoustic Modeling Units via a Multitask Learning and Deep Neural Network Technique for LVCSR of the Low-Resource Language, Amharic // IEEE Access. 2019. Т. 7. pp. 105593-105608.
62. Açarçiçek H., Çolakoğlu T., Hatipoğlu P., Huang Ch.H., Peng W. Filtering Noisy Parallel Corpus using Transformers with Proxy Task Learning // Proceedings of the Fifth Conference on Machine Translation. 2020. pp. 940–946.
63. Keung Ph., Salazar J., Lu Y., Smith N.A. Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings // Transactions of the Association for Computational Linguistics. 2020. vol. 8. pp. 828–841.
64. Sun Y., Zhu Sh., Yifan F., Mi Ch. Parallel sentences mining with transfer learning in an unsupervised setting // Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. 2021. pp. 136–142.
65. Virtanen A., Kanerva J., Ilo R., Luoma J., Luotolahti J., Salakoski T., Ginter F., Pyysalo S. Multilingual is not enough: BERT for Finnish // arXiv preprint arXiv:1912.07076. 2019. pp. 1–14.
66. Vüren, van J.M.J., Niesler T. Improving N-Best Rescoring in Under-Resourced Code-Switched Speech Recognition Using Pretraining and Data Augmentation // Preprints. 2022. 2022050066.
67. Кипяткова И.C., Марковников Н.М. Исследование методов улучшения интегральных систем распознавания речи при недостатке обучающих данных // Труды III Всероссийской акустической конференции. 2020. C. 361-367.
68. Arivazhagan N., Bapna A., Firat O., Lepikhin D., Johnson M., Krikun M., Chen M.X., Cao Y., Foster G., Cherry C., Macherey W., Chen Zh., Wu Y. Massively multilingual neural machine translation in the wild: Findings and challenges // arXiv preprint arXiv:1907.05019. 2019. pp. 1–27.
69. Chathuranga Sh., Ranathunga S. Classification of Code-Mixed Text Using Capsule Networks // Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). 2021. pp. 256–263.
70. Stickland A.C., Li X., Ghazvininejad M. Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation // Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021. pp. 3440–3453.
2. Joshi P., Santy S., Budhiraja A., Bali K., Choudhury M. The State and Fate of Linguistic Diversity and Inclusion in the NLP World // Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020. pp. 6282–6293.
3. Bender E.M. On achieving and evaluating language-independence in NLP. Linguistic Issues in Language Technology. 2011. vol. 6. no. 3. pp. 1–26.
4. Ponti E.M., O’Horan H., Berzak Y., Vulic I., Reichart R., Poibeau T., Shutova E., Korhonen A. Modeling language variation and universals: A survey on typological linguistics for natural language processing // Computational Linguistics. 2019. vol. 45. no. 3. pp. 559–601.
5. Krauwer S. The basic language resource kit (BLARK) as the first milestone for the language resources roadmap // Proceedings of International workshop on speech and computer (SPECOM-2003). 2003. pp. 8–15.
6. Berment V. Méthodes pour informatiser des langues et des groupes de langues «peu dotées». Doct. Diss. Grenoble, 2004.
7. Cieri Ch., Maxwell M., Strassel S., Tracey J. Selection criteria for low resource language programs // Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16). 2016. pp. 4543–4549.
8. Tsvetkov Y. 2017. Opportunities and challenges in working with low-resource languages. Presentation. Carnegie Mellon University, June 22, 2017.
9. Романенко А.Н. Робастное распознавание речи для низко-ресурсных языков. дис. канд. техн. наук: 05.13.11 Ульм. 2020 (на правах рукописи).
10. Мурадова А.Р. Как исчезают языки и как их возрождают // Языковое разнообразие в киберпространстве: российский и зарубежный опыт. Сборник аналитических материалов М.: МЦБС, 2008. С. 70–75.
11. Хусаинов А.Ф., Сулейманов Д.Ш. Система автоматического распознавания речи на татарском языке // Программные продукты и системы. 2013. №4. C. 31–34.
12. Израилова Э.С. О создании фонетико-акустической базы в рамках синтеза чеченской речи // Компьютерная лингвистика и обработка естественного языка. 2017. №2. C. 111–115.
13. Boyko T., Zaitseva N., Krizhanovskaya N., Krizhanovsky A., Novak I., Pellinen N., Rodionova A. The Open corpus of the Veps and Karelian languages: overview and applications // KnE Social Sciences. 2022. vol. 7. no. 3. pp. 29–40.
14. Марковников Н.М., Кипяткова И.C. Аналитический обзор интегральных систем распознавания речи // Труды СПИИРАН. 2018. Вып. 58. C. 77–110.
15. Besacier L., Barnard E., Karpov A., Schultz T. Automatic speech recognition for under-resourced languages: A survey // Speech communication. 2014. vol. 56. pp. 85–100.
16. Карпов A.A., Верходанова В.О. Речевые технологии для малоресурсных языков мира. // Вопросы языкознания. 2015. № 2. С. 117–135.
17. Sabou M., Bontcheva K., Derczynski L., Scharl A. Corpus Annotation through Crowdsourcing: Towards Best Practice Guidelines // Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). 2014. pp. 859–866.
18. Arora S., Arora K.K., Roy M.K., Agrawal S.S., Murthy B.K. Collaborative speech data acquisition for under resourced languages through crowdsourcing // Procedia Computer Science. 2016. vol. 81. pp. 37–44.
19. Schultz T., Schlippe T. GlobalPhone: Pronunciation Dictionaries in 20 Languages // Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14). 2014. pp. 337–341.
20. Strassel S., Tracey J. LORELEI Language Packs: Data, Tools, and Resources for Technology Development in Low Resource Languages // Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), 2016. pp. 3273–3280.
21. Simpson H., Cieri Ch., Maeda K., Baker K., Onyshkevych B. Human language technology resources for less commonly taught languages: Lessons learned toward creation of basic language resources // Collaboration: interoperability between people in the creation of language resources for less-resourced languages. 2008. vol. 7. pp. 7–11.
22. Do V.H., Chen N.F., Lim B.P., Hasegawa-Johnson M.A. Acoustic Modeling for Under-resourced Language using Mismatched Transcriptions // International Journal of Asian Language Processing. 2017. vol. 27. no. 2. pp. 141–153.
23. Hasegawa-Johnson M.A., Jyothi P., McCloy D., Mirbagheri M., Liberto, di G.M. Das A., Ekin B., Liu Ch., Manohar V., Tang H., Lalor E., Chen N.A. Hager P., Kekona T., Sloan R., Lee A.K.C. ASR for under-resourced languages from probabilistic transcription // IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2017. vol. 25. no. 1. pp. 50–63.
24. Yu C., Kang M., Chen Y., Wu J., Zhao X. Acoustic modeling based on deep learning for low-resource speech recognition: An overview // IEEE Access. 2020. vol. 8. pp. 163829-163843.
25. Ko T., Peddinti V., Povey D., Khudanpur S. Audio augmentation for speech recognition // Proceedings of the 16th Annual Conference of the International Speech Communication Association. 2015. pp. 3586-3589.
26. Rebai I., BenAyed Y., Mahdi W., Lorré J.P. Improving speech recognition using data augmentation and acoustic model fusion // Procedia Computer Science. 2017. vol. 112. pp. 316–322.
27. Park D.S., Chan W., Zhang Y., Chiu C.C., Zoph B., Cubuk E.D., Le Q.V. SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition // Proceedings of Interspeech. 2019. pp. 2613-2617.
28. Hartmann W., Ng T., Hsiao R., Tsakalidis S., Schwartz R. Two-Stage Data Augmentation for Low-Resourced Speech Recognition // Proceedings of Interspeech. 2016. pp. 2378-2382.
29. Jin Z., Finkelstein A., DiVerdi S., Lu J., Mysore G.J. Cute: A concatenative method for voice conversion using exemplar-based unit selection // Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'16). 2016. pp. 5660-5664.
30. Goodfellow I.J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair Sh., Courville A., Bengio Y. Generative adversarial nets // Advances in Neural Information Processing Systems 27 / Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, K. Q. Weinberger (eds.). pp. 2672–2680 // Curran Associates, Inc., 2014.
31. Hsu Ch.-Ch., Hwang H.-T., Wu Y.-Ch., Tsao Y., Wang H. Voice Conversion from Unaligned Corpora Using Variational Autoencoding Wasserstein Generative Adversarial Networks // arXiv preprint arXiv:1704.00849. 2017. pp. 1–5.
32. Kameoka H., Kaneko T., Tanaka K., Hojo N. StarGAN-VC: non-parallel many-to-many Voice Conversion Using Star Generative Adversarial Networks // Proceedings of 2018 IEEE Spoken Language Technology Workshop (SLT'18). 2018. pp. 266–273.
33. Gokay R., Yalcin H. Improving low resource turkish speech recognition with data augmentation and TTS // Proceedings of 2019 16th International Multi-Conference on Systems, Signals and Devices (SSD). 2019. pp. 357–360.
34. Shen J., Pang R., Weiss R.J., Schuster M., Jaitly N., Yang Z., Chen Z., Zhan Y., Wang Y., Skerrv-Ryan R., Saurous R.A., Agiomvrgiannakis Y., Wu Y. Natural TTS Synthesis by Conditioning WaveNet on Mel SpectrogramPredictions // Proceedings of 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018. pp. 4779–4783.
35. Dua M., Kadyan V., Banthia N., Bansal A., Agarwal T. Spectral warping and data augmentation for low resource language ASR system under mismatched conditions // Applied Acoustics. 2022. vol. 190. 108643.
36. Du C., Yu K. Speaker augmentation for low resource speech recognition // Proceedings of 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2020. pp. 7719-7723.
37. Bagchi D., Wotherspoon Sh., Jiang Zh., Muthukumar P. Speech Synthesis as Augmentation for Low-Resource ASR // arXiv preprint arXiv:2012.13004. 2020. pp. 1–4.
38. Hsu B.J. Generalized linear interpolation of language models // Proceedings of 2007 IEEE workshop on automatic speech recognition & understanding (ASRU). 2007. pp. 136–140.
39. Kurimo M., Enarvi S., Tilk O., Varjokallio M., Mansikkaniemi A., Alumäe T. Modeling under-resourced languages for speech recognition // Language Resources and Evaluation. 2017. vol. 51. no. 4. pp. 961–987.
40. Fadaee M., Bisazza A., Monz Ch. 2017. Data Augmentation for Low-Resource Neural Machine Translation // Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 2017. pp. 567–573.
41. Şahin G.G. To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP // Computational Linguistics. 2022. vol. 48. no. 1. pp. 5–42.
42. Kobayashi S. Contextual Augmentation: Data Augmentation by Words with Paradigmatic Relations // Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Vol. 2 (Short Papers). 2018. pp. 452–457.
43. Kumar V., Choudhary A., Cho E. Data Augmentation using Pre-trained Transformer Models // Proceedings of the 2nd Workshop on Life-long Learning for Spoken Language Systems. 2020. pp. 18–26.
44. Ogawa A., Tawara N., Delcroix M. Language Model Data Augmentation Based on Text Domain Transfer // Proceedings of Interspeech. 2020. pp. 4926–4930.
45. Sennrich R., Haddow B., Birch A. Improving Neural Machine Translation Models with Monolingual Data // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. 2016. pp. 86–96.
46. Shahnawazuddin S., Nagaraj A., Kunal K., Aayushi P., Waquar A. Voice Conversion Based Data Augmentation to Improve Children Speech Recognition in Limited Data Scenario // Proceedings of Interspeech 2020. pp. 4382–4386.
47. Tachibana H., Uenoyama K., Aihara S. Efficiently trainable text-to-speech system based on deep convolutional networks with guided attention // Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2018. pp. 4784–4788.
48. Edunov S., Ott M., Auli M., Grangier D. Understanding Back-Translation at Scale // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. pp. 489–500.
49. Fadaee M., Monz Ch. Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 2018. pp. 436–446.
50. Karakanta A., Dehdari J., Genabith, van J. Neural machine translation for low-resource languages without parallel corpora // Machine Translation. 2018. vol. 32. no. 1-2. pp. 167–189.
51. Xia M., Kong X., Anastasopoulos A., Neubig G. Generalized Data Augmentation for Low-Resource Translation // Proceedings of the 57th Annual Meeting of the ACL. 2019. pp. 5786–5796.
52. Wang D., Zheng T.F. Transfer learning for speech and language processing // Proceedings of 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). 2015. pp. 1225–1237.
53. Gauvain J.-L., Lee C.-H. Maximum a posteriori estimation for multivariate Gaussian mixture observations of Markov chains // IEEE Transactions on Speech and audio processing. 1994. vol. 2. no. 2. pp. 291–298.
54. Wet, de F., Kleynhans N., Compernolle, van D., Sahraeian, R. Speech recognition for under-resourced languages: Data sharing in hidden Markov model systems // South African Journal of Science. 2017. vol. 113. no. 1–2. pp. 1–9.
55. Woldemariam Y. Transfer Learning for Less-Resourced Semitic Languages Speech Recognition: the Case of Amharic // Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL). 2020. pp. 61–69.
56. Thai B., Jimerson R., Arcoraci D., Prud'hommeaux E., Ptucha R. Synthetic data augmentation for improving low-resource ASR // Proceedings of 2019 IEEE Western New York Image and Signal Processing Workshop (WNYISPW). 2019. pp. 1–9.
57. Eberhard O., Zesch T. Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages // Proceedings of the 17th Conference on Natural Language Processing (KONVENS 2021). 2021. pp. 208–212.
58. Tachbelie M.Y., Abate S.T., Schultz T. Development of Multilingual ASR Using GlobalPhone for Less-Resourced Languages: The Case of Ethiopian Languages // Proceedings of Interspeech. 2020. pp. 1032–1036.
59. Tachbelie M.Y., Abate S.T., Schultz T. Multilingual speech recognition for GlobalPhone languages // Speech Communication. 2022. vol. 140. pp. 71–86.
60. He D., Lim B.P., Yang X., Hasegawa-Johnson M.A., Chen D. Improved ASR for under-resourced languages through multi-task learning with acoustic landmarks // Proceedings of Interspeech. 2018. pp. 2618-2622.
61. Fantaye T.G., Yu J., Hailu T.T. Investigation of Various Hybrid Acoustic Modeling Units via a Multitask Learning and Deep Neural Network Technique for LVCSR of the Low-Resource Language, Amharic // IEEE Access. 2019. Т. 7. pp. 105593-105608.
62. Açarçiçek H., Çolakoğlu T., Hatipoğlu P., Huang Ch.H., Peng W. Filtering Noisy Parallel Corpus using Transformers with Proxy Task Learning // Proceedings of the Fifth Conference on Machine Translation. 2020. pp. 940–946.
63. Keung Ph., Salazar J., Lu Y., Smith N.A. Unsupervised Bitext Mining and Translation via Self-Trained Contextual Embeddings // Transactions of the Association for Computational Linguistics. 2020. vol. 8. pp. 828–841.
64. Sun Y., Zhu Sh., Yifan F., Mi Ch. Parallel sentences mining with transfer learning in an unsupervised setting // Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. 2021. pp. 136–142.
65. Virtanen A., Kanerva J., Ilo R., Luoma J., Luotolahti J., Salakoski T., Ginter F., Pyysalo S. Multilingual is not enough: BERT for Finnish // arXiv preprint arXiv:1912.07076. 2019. pp. 1–14.
66. Vüren, van J.M.J., Niesler T. Improving N-Best Rescoring in Under-Resourced Code-Switched Speech Recognition Using Pretraining and Data Augmentation // Preprints. 2022. 2022050066.
67. Кипяткова И.C., Марковников Н.М. Исследование методов улучшения интегральных систем распознавания речи при недостатке обучающих данных // Труды III Всероссийской акустической конференции. 2020. C. 361-367.
68. Arivazhagan N., Bapna A., Firat O., Lepikhin D., Johnson M., Krikun M., Chen M.X., Cao Y., Foster G., Cherry C., Macherey W., Chen Zh., Wu Y. Massively multilingual neural machine translation in the wild: Findings and challenges // arXiv preprint arXiv:1907.05019. 2019. pp. 1–27.
69. Chathuranga Sh., Ranathunga S. Classification of Code-Mixed Text Using Capsule Networks // Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021). 2021. pp. 256–263.
70. Stickland A.C., Li X., Ghazvininejad M. Recipes for Adapting Pre-trained Monolingual and Multilingual Models to Machine Translation // Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. 2021. pp. 3440–3453.
Опубликован
2022-07-14
Как цитировать
Кипяткова, И. С., & Кагиров, И. А. (2022). Аналитический обзор методов решения проблемы малых наборов данных при создании систем автоматического распознавания речи для малоресурсных языков. Информатика и автоматизация, 21(4), 678-709. https://doi.org/10.15622/ia.21.4.2
Раздел
Искусственный интеллект, инженерия данных и знаний
Copyright (c) Ирина Сергеевна Кипяткова, Ильдар Амирович Кагиров
Это произведение доступно по лицензии Creative Commons «Attribution» («Атрибуция») 4.0 Всемирная.
Авторы, которые публикуются в данном журнале, соглашаются со следующими условиями: Авторы сохраняют за собой авторские права на работу и передают журналу право первой публикации вместе с работой, одновременно лицензируя ее на условиях Creative Commons Attribution License, которая позволяет другим распространять данную работу с обязательным указанием авторства данной работы и ссылкой на оригинальную публикацию в этом журнале. Авторы сохраняют право заключать отдельные, дополнительные контрактные соглашения на неэксклюзивное распространение версии работы, опубликованной этим журналом (например, разместить ее в университетском хранилище или опубликовать ее в книге), со ссылкой на оригинальную публикацию в этом журнале. Авторам разрешается размещать их работу в сети Интернет (например, в университетском хранилище или на их персональном веб-сайте) до и во время процесса рассмотрения ее данным журналом, так как это может привести к продуктивному обсуждению, а также к большему количеству ссылок на данную опубликованную работу (Смотри The Effect of Open Access).