Clinical Reasoning and Self-confidence among Preclinical Medical Students, Internal Medicine Specialists and Artificial Intelligence: A Cross-sectional Study

Abstract

Aims: This study evaluated diagnostic skills by comparing clinical reasoning accuracy and self-confidence among preclinical medical students, internal medicine specialists, and large language models using the Clinical Reasoning and Self-confidence Assessment Tool. Study Design: Cross-sectional study employing a previously validated assessment tool called CRESCAT. Place and Duration of Study: Conducted at the Middle West State University of Paraná and the Londrina State University in Brazil from March to November 2023. Methodology: We assessed accuracy and self-confidence in seven clinical cases across 133 preclinical students, 16 specialists, and 2 large language models, utilizing statistical tests such as the Student’s T-test and the Kruskal-Wallis’s test. Spearman’s test conducted correlation analysis. Results: Average accuracy improved from beginners (31.7±11.2%) to second-year students (60.0±10.9%; P < .001). Specialists (75.7±10.0%) and large language models (80.0%) outperformed students (P < .001). Self-confidence was lowest in beginners (2.07 [1.71-2.89]) compared to others (3.14 [2.71-3.43]; P < .001), and a moderate and positive correlation between accuracy and self-confidence was observed (Rho = .623; P < .001) in the overall sample. Conclusion: The findings highlight the value of the CRESCAT dedicated assessment tool and artificial intelligence in evaluating clinical reasoning.