Enhancing reliability and mitigating hallucinations in GPT-based tutors : a comparative study of RAG and document-augmented methods | |
| Author | Kakati, Richa |
| Call Number | AIT Thesis no.IM-25-02 |
| Subject(s) | Generative artificial intelligence Generative artificial intelligence in higher education ChatGPT |
| Note | A thesis submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Information Management |
| Publisher | Asian Institute of Technology |
| Abstract | Generative AI tools such as GPT are becoming increasingly common in programming education due to their potential to enhance student learning. However, these tools fre quently generate incorrect or misleading information, a problem known as ’hallucina tion’, which can negatively affect learning experiences. This research aims to address this issue by developing and testing GPT-based tutors designed to be more reliable and accurate. Specifically, the study proposes comparing two augmentation techniques: Retrieval-Augmented Generation (RAG) and Document-Augmented Generation. While retrieval-based AI support has been studied in general educational settings, its applica tion within GPT-powered tutoring for programming, particularly on difficult-level pro gramming tasks, remains underexplored. Experiments involve programming assign ment problems sourced from the Machine Learning course taught at the Asian Institute of Technology. Three GPT-based tutors are developed: two Document-Augmented GPT tutor utilizing provided course materials (one using Custom ChatGPT from OpenAI and one using LangChain), and a RAG-Augmented GPT tutor. The three developed tutors are then compared with two non-augmented models: baseline ChatGPT and baseline ChatGPT in Study Mode. The study contributes by testing RAG in a domain where hal lucination risk is high and factual accuracy grounding is critical. The findings of this study will provide valuable insights and practical recommendations for educators look ing to integrate generative AI tools safely and effectively into programming education. |
| Year | 2025 |
| Type | Thesis |
| School | School of Engineering and Technology |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Information Management (IM) |
| Chairperson(s) | Chutiporn Anutariya; |
| Examination Committee(s) | Chantri Polprasert;Aekavute Sujarae; |
| Scholarship Donor(s) | AIT Scholarship; |
| Degree | Thesis (M. Eng.) - Asian Institute of Technology, 2025 |