Objective: Providing patients with accurate and reliable information significantly improves their quality of life and reduces the burden on healthcare services. Recent advancements in artificial intelligence (AI), particularly Large Language Models (LLMs), offer opportunities for enhanced patient education. This study aimed to compare an asthma patient information text generated by an LLM with one provided by the Turkish National Society of Allergy and Clinical Immunology (TNSACI), as evaluated by specialized physicians.

Materials and Methods: Physicians with a minimum of five years of experience in Allergy and Immunology were recruited to assess blinded versions of two asthma educational texts: one generated by Chat Generative Pre-trained Transformer (ChatGPT) and the other sourced from TNSACI`s website. Participants evaluated the texts using a Likert scale, assessing accuracy, comprehensiveness, level of detail, comprehensibility, consistency, reliability, and overall satisfaction. Additionally, readability scores were determined using the Flesch-Kincaid formulas.

Results: A total of 21 physicians participated (mean age: 38.4 ± 4.9 years; mean professional experience: 6.6 ± 2.7 years). The ChatGPT text contained 973 words with a readability score of 56.3 (10th-12th grade level), while the TNSACI text contained 1,603 words with a readability score of 48.5 (college level). Likert scale evaluations showed no significant difference in accuracy, comprehensiveness, consistency, or reliability between the two texts. However, the ChatGPT text was rated significantly higher for comprehensibility (p=0.003) and was considered less overly detailed (p=0.001). Regarding overall preference, 57.1% of physicians favored the ChatGPT text, 4.8% preferred the TNSACI text, and 38.1% rated them equally.

Conclusion: Specialist physicians found the ChatGPT-generated asthma information text to be more comprehensible and preferable. These results suggest that AI-based educational content could enhance patient information materials and contribute to more effective patient education.