Mobile tour guide application with real-time attraction spot recognition using VLM and AR for Kampar

Pang, Jing Thean (2026) Mobile tour guide application with real-time attraction spot recognition using VLM and AR for Kampar. Final Year Project, UTAR.

PDF Download (5Mb)

Abstract

Kampar is a town in Perak, Malaysia, that has many historical places often overlooked by tourists. Most tourists do not know the history and information about the places they visit. To learn more, they often rely on internet search engines such as Google and Google Lens. However, this can be a time-consuming process and yields inaccurate results. To address these issues, this project proposes a mobile tour guide application with recognition features for Kampar. By utilising modern technologies, this application aims to enhance cultural and historical tourism by making the experience more interactive and educational. Developing a mobile application offer seamless accessibility, as people frequently carry their devices. The main feature of this mobile application is to function as a virtual tour guide when users visit these places. It integrates pre-trained models such as Vision-Language Models (VLM) for real-time recognition. The application is designed to identify specific points of interest or distinct structural features rather than the general vicinity. When an attraction spot is identified, it provides Augmented Reality (AR) overlays with the name of attraction spot. Users can tap the labels to view more detailed information. By offering an accessible and immersive tour guide experience, the application promotes Kampar’s cultural places to both tourists and locals. The recognition feature focuses on two places: UTAR Grand Hall (Dewan Tun Dr. Ling Liong Sik) and Kampar Seng Fatt Temple. Overall, this project involves pre-trained models, including vision-language model for object detection and image classification, along with augmented reality (AR) for content delivery.

Actions (login required)