561: An Evidence Grading Algorithm to Enhance LLMs� Performance in Addressing Patient Medical Queries

Topic: Education, Societal Impact, and Others
Subtopic: Artificial Intelligence/Machine Learning

Background and Aims: Patients increasingly turn to Large Language Models (LLMs) for medical advice, yet these models often struggle with providing evidence-based answers and can generate "hallucinations." This study aims to develop and validate evidence grading algorithm designed to integrate the GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework into LLMs, enhancing their ability to address complex patient medical queries with optimal evidence supporting.

Methods: We propose a novel algorithmic framework that combines Retrieval-Augmented Generation (RAG) with an automated evidence-quality assessment module. The methodology involves: 1) Training the algorithm to identify and extract key clinical evidence from biomedical databases; 2) Implementing a logic-based grading system to automatically rate the certainty of evidence; and 3) Optimizing the LLM’s response generation by prioritizing high-certainty data. The performance of this enhanced LLM is evaluated against expert-verified clinical guidelines and traditional RAG models based on accuracy, reliability, and adherence to established medical standards.

Results: The development of the evidence grading algorithm is currently in the optimization phase. Preliminary testing indicates that the algorithm improves the model's ability to distinguish between strong and weak recommendations. Comprehensive results regarding the algorithm's impact on reducing misinformation and its consistency with human expert assessments will be presented at the conference.

Conclusions: By bridging the gap between raw medical data and clinical evidence grading, this algorithm provides a scalable solution to enhance the trustworthiness of AI-generated medical advice. This framework holds the potential to support patient-centered care by ensuring that AI-driven responses are grounded in the most reliable clinical evidence available.

References: no

Presenting Author: MingMing Wang
Submitting Author: mingming wang

Senior Author: Liang Yao

IASP 2026 Global Year theme of Neuropathic Pain: Yes

• Weize Kong - lanzhou university
• Ming Liu - Nanyang Technological University
• Dandan AI - Nanyang Technological University
• Meixuan LI - lanzhou university
• Liang Yao - Nanyang Technological University