Large language model with facial recognition

Foo, Wai Hong (2026) Large language model with facial recognition. Final Year Project, UTAR.

PDF Download (2815Kb)

Abstract

Business intelligence is often about understanding complicated relationships between financial data, business risks, companies, market factors, and operations. Financial information is often embedded in long documents, structured tables, and across multiple business entities, which makes it difficult for a single AI agent to analyse these relationships. In this project we propose a secure Multi-Agent RAG (Retrieval Augmented Generation) system for financial report analysis. Users can upload 10-K reports, register and log in with facial recognition and query financial questions through a chatbot interface. The backend stores the extracted text chunks, entities, relationships, and structured table rows from the uploaded reports into MongoDB, Milvus, and Neo4j. This project proposed a multi-agent retrieval pipeline that analyses each user query, routes it to appropriate retrieval agents, retrieves relevant text and table evidence, traverses graph relationships, supports financial calculations, rerank evidence and generates a final answer with citations. The system integrates vector retrieval, graph-based reasoning, table retrieval and agent-based coordination to help answer factual, comparative and analytical financial questions. The evaluation was done in two stages, retrieval evaluation, and Retrieval Augmented Generation Assesssment (RAGAS) evaluation. We evaluated the system on 150 examples of financial question answering, consisting of 60 factual, 48 comparative and 42 analytical questions. In the retrieval evaluation, the system achieved 100.00% pass rate, 99.33% Hit@5, and 100.00% Recall@10. For the standard RAGAS evaluation, it scored 97.73% faithfulness, 100.00% context recall, 87.43% answer relevancy, 70.09% answer correctness. The results reveal that the proposed system is capable of retrieving relevant financial evidence and generating responses that are mostly faithful and grounded in context. The results indicate that the proposed facial recognition authentication and Multi-Agent Graph RAG can enable secure, efficient, and explainable analysis of financial reports. However, we recommend further improvements to improve the correctness of answers, strengthen analytical reasoning, and expand the system to support more financial reports and user scenarios. This project highlights the potential of secure multi-agent Graph RAG systems for business intelligence and financial decision-making. Keywords: Large Language Model; Facial Recognition; Multi-Agent System; Graph Retrieval-Augmented Generation; Financial Report Analysis; Question Answering; Natural Language Processing; Vector Database; Knowledge Graph; Artificial Intelligence Subject Area: QA75.5-76.95 Electronic computers. Computer science

Actions (login required)