Mobile application for real-time food image segmentation and nutritional guidance at grocery store

Tan, Chee Lin (2024) Mobile application for real-time food image segmentation and nutritional guidance at grocery store. Final Year Project, UTAR.

Preview

PDF Download (5Mb) | Preview

Abstract

This project is a development-based initiative that aims to help consumers make informed choices about their food purchases by improving existing health and nutrition applications. There are existing similar applications such as MyFitnessPal, Open Food Facts, Food Visor and Food Check that offers some common features that could be found in most health and nutrition applications. However, these applications contain limitations such as inefficiency in searching for food products, insufficient and inaccurate nutritional information, and complexity caused by the application and human errors and other limitations that will soon be discussed. The proposed Mobile Application for Real-Time Food Image Segmentation and Nutritional Guidance at Grocery Store is to solve their limitations by combining both real-time image segmentation and nutritional information provision, allowing users to capture and analyse food product images through their mobile phone's camera. The application identifies and segments food items within these images, providing users with detailed nutritional information, including macronutrients, micronutrients, and caloric contents. Furthermore, the application will include a personalized system that allows users to login with authentication and update their profile’s health status and a robust recommendation system that generates recommendations based on a user's health status by calling the Gemini API. In this work, DeepLabV3+ architecture will be utilized to train the model specifically for real-time image segmentation. Once trained, it will be converted into a TensorFlow Lite model, enabling its integration into a mobile application for performing image segmentation tasks. The images from the Freiburg Dataset are used and manually labelled, to provide 14 different groceries category with a total of 50 images each with their respective ground truth images, totalling only 700 images. The evaluation measurement to determine the performance for the DeeplabV3+ model is the training and validation loss and accuracy. An average accuracy of 87.55% for training set and 66.84% for validation set. In order to evaluate the overall performance of the model, the trained model is applied on the testing set and achieved an accuracy of 82.70%.

Actions (login required)