Development of multilingual and code-mixed translation app for Malaysian

Oi, Jia Zhun (2026) Development of multilingual and code-mixed translation app for Malaysian. Final Year Project, UTAR.

PDF Download (3788Kb)

Abstract

This project focuses on developing a multilingual and code-mixed translation mobile application to solve the communication challenges faced in Malaysia’s multilingual society. Most existing translation apps are not able to recognize or translate mixed-language speech well, especially when users switch between Malay, Chinese and English within the same sentence. To address this problem, a fine-tuned SeamlessM4T ASR model is used to improve the accuracy of recognizing code-mixed audio. The recognized text is then processed by Google Gemini to generate translations. Google Cloud Text-to-Speech is used to produce natural-sounding voice results. The whole system is implemented as a mobile application using React Native with Firebase for backend management. The aim of this project is to provide a more accurate, user-friendly and practical translation tool that supports real-time code-mixed communication in daily life.

Actions (login required)