Real-time money counting app for visually impaired

Lim, Boon Chong (2025) Real-time money counting app for visually impaired. Final Year Project, UTAR.

Preview

PDF Download (5Mb) | Preview

Abstract

Visually impaired individuals usually need assistance from others to perform daily activities such as grocery shopping, reading documents, and recognizing banknotes due to their limited vision. Low vision or blindness causes their daily life activities to become time-consuming, especially dealing with financial transactions because they are not able to determine the amount of money that they are handling especially if the banknote is old, worn, or even torn. Even though efforts are made to print Braille on the banknotes, the tactile loses its ability quickly after circulation. Moreover, the existing mobile application specifically designed to help the visually impaired is usually unable to recognize Malaysian currency, especially coins. Additionally, most mobile applications require user subscriptions to enable full functionality, which limits the visually impaired users' ability to use them in daily life. Therefore, this project aims to develop a mobile application using Flutter for both Android and iOS platforms, with the transfer learning on a pre-trained YOLOv8 model to recognize and count new Malaysian banknotes and coins one by one in real-time. The app also utilized TensorFlow Lite to convert the model into a mobile-compatible format to run on mobile devices. Furthermore, the app designed with a user-friendly interface and accessibility features such as high-contrast text, audio feedback, and vibration notification. The mobile application can help visually impaired users recognize and count the banknotes and coins they are handling in a more accessible with mAP50 (mean average precision calculated at an intersection over union threshold of 50) up to 98.7%. A custom counting technique is also implemented, utilizing Euclidean distance calculations and a timeout mechanism for accurate object tracking.

Actions (login required)