0461: Right Diagnoses For the Wrong Reasons: Limitations of Current Large-Language Model based Agentic Frameworks for Screening of Rheumatoid Arthritis

Background/Purpose: In India, Rheumatoid arthritis (RA) is often diagnosed late. Artificial Intelligence (AI) driven Large Language models (LLMs) can be deployed via widely available mobile phone devices, offering a potential screening solution. Here, we attempted to explore various AI agentic frameworks to identify the most effective model SARA (Screening Agent for RA), and also to validate how these models arrive at a diagnosis of RA.

Methods: We had developed the PreRAID (Pre-Screening Rheumatoid Arthritis Information Database) from consenting patients with joint pain, classifying patients into RA or not RA as per treating physician diagnosis. The dataset was split into 280 cases for the knowledge base (KB) and 70 for testing. A Neo4j vector database enabled embedding-based retrieval.
Six LLMs were evaluated—four closed-source (OpenAI o1, OpenAI o3 mini, Gemini 2.5 Pro, Gemini 2.0 Flash) and two open-source (QwQ, Deepseek R1 70B). Each model was tested under three agentic configurations: (1) a single agent without knowledge base (KB) access; (2) a single agent with retrieval-augmented generation (RAG); and (3) a dual-agent setup, where the first agent generated a diagnosis and reasoning, validated by a second agent. [Figure 1]
Each model-configuration pair was evaluated on 50 new cases. The models were also asked to explain the reasons behind each diagnosis, which was independently assessed by two fellows and one consultant rheumatologist using a four-point Likert scale.

Results: The PreRAID dataset included 84% RA-confirmed cases and 16% controls. Deepseek R1 showed the highest accuracy (82%) in the single-agent with KB setting, followed by o1 and o3 mini (80% each). Accuracy dropped in the two-agent setup, most notably in Gemini 2.5 Pro (37%) and Gemini 2.0 Flash (40%) [Figure 2].
Despite moderate to high accuracy, Gemini models underperformed in reasoning when agentic complexity increased. Reasoning quality was suboptimal across all models. Gemini 2.0 Flash (36/50) and Deepseek R1 (28/50) had the most correct justifications, while QwQ and Gemini 2.5 Pro scored the lowest (6 and 10, respectively). Many outputs showed minor or major reasoning flaws, indicating a disconnect between diagnostic accuracy and reasoning integrity.

Conclusion: Deepseek R1 achieved the highest diagnostic accuracy (82%) in the single-agent with KB configuration. However, reasoning quality across all models was suboptimal, with many correct diagnoses attributed to flawed reasons. The dual-agent setup, intended to enhance reasoning, led to decreased diagnostic performance. Though we built a robust system (SARA) for screening, the lack of proper reasoning by the LLMs preclude the use as a diagnostic tool. Future directions should prioritize explainable AI approaches so that clinicians can trust the tool.