Building a mixed Vietnamese-English speech recognition solution

Abstract

In this study, we propose a Vietnamese–English mixed speech recognition solution that introduces a novel approach: converting English phonetic structures into their approximate Vietnamese phonetic representations. This strategy aims to unify the pronunciation system, enabling deep learning models to better capture and decode speech in a code-switching context. Experimental results demonstrate that the proposed method significantly improves recognition accuracy on mixed-language speech and simplifies the handling of phonetic variability across languages. This solution shows strong potential for practical applications in multilingual virtual assistants, language learning, and real-time translation systems in Vietnamese–English bilingual environments.