Self-hosted multi-agent RAG system for contextual document processing
The increasing use of Artificial Intelligence (AI) in document processing faces persistent challenges such as hallucination, privacy risks, and limited adaptability. This study presents a self-hosted multi-agent Retrieval-Augmented Generation (RAG) system designed to address these limitations by enh...
Saved in:
| Main Author: | |
|---|---|
| Format: | Final Year Project / Dissertation / Thesis |
| Published: |
2025
|
| Subjects: | |
| Online Access: | http://eprints.utar.edu.my/7287/1/SE_2104132_FYP_Report%2DEngZiJun_ENG_ZI_JUN.pdf http://eprints.utar.edu.my/7287/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1855616550686425088 |
|---|---|
| author | Eng, Zi Jun |
| author_facet | Eng, Zi Jun |
| author_sort | Eng, Zi Jun |
| building | UTAR Library |
| collection | Institutional Repository |
| content_provider | Universiti Tunku Abdul Rahman |
| content_source | UTAR Institutional Repository |
| continent | Asia |
| country | Malaysia |
| description | The increasing use of Artificial Intelligence (AI) in document processing faces persistent challenges such as hallucination, privacy risks, and limited adaptability. This study presents a self-hosted multi-agent Retrieval-Augmented Generation (RAG) system designed to address these limitations by enhancing accuracy and preserving data privacy through a fully local and modular architecture. Built using Marker, Ollama, LangGraph, and Weaviate, the system enables flexible deployment and coordination between agents. Evaluation using the SQuAD dataset measured retrieval and generation performance through metrics such as Recall@3, Mean Reciprocal Rank (MRR), Context Recall, Faithfulness, and Answer Correctness. Two evaluation methods were employed: a calculation-based approach on 100 samples for quantitative assessment, and an LLM-as-Judge approach using GPT-4o on 20 samples for qualitative, human-like evaluation. Results show strong retrieval performance with a Recall@3 of 90%, MRR of 75%, and Context Recall of 100%, demonstrating accurate and consistent grounding. The generation results indicate improved faithfulness and contextual relevance, though challenges remain in scalability and factual precision. Overall, the findings show that the proposed multi-agent RAG system effectively mitigates hallucination and privacy concerns while maintaining adaptability, making it a promising approach for secure and accurate AI-driven document processing.
Keywords: Artificial Intelligence (AI), Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), Self-Hosted AI
Subject Area: Q300-390 Cybernetics |
| format | Final Year Project / Dissertation / Thesis |
| id | my-utar-eprints.7287 |
| institution | Universiti Tunku Abdul Rahman |
| publishDate | 2025 |
| record_format | eprints |
| spelling | my-utar-eprints.72872026-01-13T10:08:31Z Self-hosted multi-agent RAG system for contextual document processing Eng, Zi Jun QA75 Electronic computers. Computer science QA76 Computer software The increasing use of Artificial Intelligence (AI) in document processing faces persistent challenges such as hallucination, privacy risks, and limited adaptability. This study presents a self-hosted multi-agent Retrieval-Augmented Generation (RAG) system designed to address these limitations by enhancing accuracy and preserving data privacy through a fully local and modular architecture. Built using Marker, Ollama, LangGraph, and Weaviate, the system enables flexible deployment and coordination between agents. Evaluation using the SQuAD dataset measured retrieval and generation performance through metrics such as Recall@3, Mean Reciprocal Rank (MRR), Context Recall, Faithfulness, and Answer Correctness. Two evaluation methods were employed: a calculation-based approach on 100 samples for quantitative assessment, and an LLM-as-Judge approach using GPT-4o on 20 samples for qualitative, human-like evaluation. Results show strong retrieval performance with a Recall@3 of 90%, MRR of 75%, and Context Recall of 100%, demonstrating accurate and consistent grounding. The generation results indicate improved faithfulness and contextual relevance, though challenges remain in scalability and factual precision. Overall, the findings show that the proposed multi-agent RAG system effectively mitigates hallucination and privacy concerns while maintaining adaptability, making it a promising approach for secure and accurate AI-driven document processing. Keywords: Artificial Intelligence (AI), Retrieval-Augmented Generation (RAG), Large Language Models (LLMs), Self-Hosted AI Subject Area: Q300-390 Cybernetics 2025 Final Year Project / Dissertation / Thesis NonPeerReviewed application/pdf http://eprints.utar.edu.my/7287/1/SE_2104132_FYP_Report%2DEngZiJun_ENG_ZI_JUN.pdf Eng, Zi Jun (2025) Self-hosted multi-agent RAG system for contextual document processing. Final Year Project, UTAR. http://eprints.utar.edu.my/7287/ |
| spellingShingle | QA75 Electronic computers. Computer science QA76 Computer software Eng, Zi Jun Self-hosted multi-agent RAG system for contextual document processing |
| title | Self-hosted multi-agent RAG system for contextual document processing |
| title_full | Self-hosted multi-agent RAG system for contextual document processing |
| title_fullStr | Self-hosted multi-agent RAG system for contextual document processing |
| title_full_unstemmed | Self-hosted multi-agent RAG system for contextual document processing |
| title_short | Self-hosted multi-agent RAG system for contextual document processing |
| title_sort | self-hosted multi-agent rag system for contextual document processing |
| topic | QA75 Electronic computers. Computer science QA76 Computer software |
| url | http://eprints.utar.edu.my/7287/1/SE_2104132_FYP_Report%2DEngZiJun_ENG_ZI_JUN.pdf http://eprints.utar.edu.my/7287/ |
| url_provider | http://eprints.utar.edu.my |