Eyal Klang

2 months ago | nature.com | Ofira Zloto |Avner Hostovsky |Ido Didi Fabian |Oded Sagiv |Benjamin S Glicksberg |Eyal Klang | +7 more

To examine the abilities of ChatGPT in writing scientific ophthalmology introductions and to compare those abilities to experienced ophthalmologists. OpenAI web interface was utilized to interact with and prompt ChatGPT 4 for generating the introductions for the selected papers. Consequently, each paper had two introductions—one drafted by ChatGPT and the other by the original author. Ten ophthalmology specialists with a minimal experience of more than 15 years, each representing distinct subspecialties—retina, neuro-ophthalmology, oculoplastic, glaucoma, and ocular oncology were provided with the two sets of introductions without revealing the origin (ChatGPT or human author) and were tasked to evaluate the introductions. For each type of introduction, out of 45 instances, specialists correctly identified the source 26 times (57.7%) and erred 19 times (42.2%). The misclassification rates for introductions were 25% for experts evaluating introductions from their own subspecialty while to 44.4% for experts assessed introductions outside their subspecialty domain. In the comparative evaluation of introductions written by ChatGPT and human authors, no significant difference was identified across the assessed metrics (language, data arrangement, factual accuracy, originality, data Currency). The misclassification rate (the frequency at which reviewers incorrectly identified the authorship) was highest in Oculoplastic (66.7%) and lowest in Retina (11.1%). ChatGPT represents a significant advancement in facilitating the creation of original scientific papers in ophthalmology. The introductions generated by ChatGPT showed no statistically significant difference compared to those written by experts in terms of language, data organization, factual accuracy, originality, and the currency of information. In addition, nearly half of them being indistinguishable from the originals. Future research endeavours should explore ChatGPT-4’s utility in composing other sections of research papers and delve into the associated ethical considerations.

Feb 10, 2025 | nature.com | Vera Sorin |Idan Hecht |Ofira Zloto |Benjamin S Glicksberg |Hila Bufman |Yiftach Barash | +4 more

Recent advancements in generative artificial intelligence have enabled analysis of text with visual data, which could have important implications in healthcare. Diagnosis in ophthalmology is often based on a combination of ocular examination, and clinical context. The aim of this study was to evaluate the performance of multimodal GPT-4 (GPT-4 V) in an integrated analysis of ocular images and clinical text. This retrospective study included 40 patients seen in our institution with images of their ocular examinations. Cases were selected by a board-certified ophthalmologist, to represent various pathologies. We provided the model with each patient image, without and then with the clinical context. We also asked two non-ophthalmology physicians to write diagnoses for each image, without and then with the clinical context. Answers for both GPT-4 V and the non-ophthalmologists were evaluated by two board-certified ophthalmologists. Performance accuracies were calculated and compared. GPT-4 V provided the correct diagnosis in 19/40 (47.5%) cases based on images without clinical context, and in 27/40 (67.5%) cases when clinical context was provided. Non-ophthalmologist physicians provided the correct diagnoses in 24/40 (60.0%), and 23/40 (57.5%) of cases without clinical context, and in 29/40 (72.5%) and 27/40 (67.5%) with clinical context. For all study participants adding context improved accuracy (p = 0.033). GPT-4 V is currently able to simultaneously analyze and integrate visual and textual data, and arrive at accurate clinical diagnoses in the majority of cases. Multimodal large language models like GPT-4 V have significant potential to advance both patient care and research in ophthalmology.

Nov 26, 2024 | nature.com | Daniel David |Ofira Zloto |Avner Hostovsky |Eyal Klang |Vicktoria Vishnevskia-Dai |Gabriel Katz | +5 more

To evaluate AI-based chat bots ability to accurately answer common patient’s questions in the field of ophthalmology. An experienced ophthalmologist curated a set of 20 representative questions and responses were sought from two AI generative models: OpenAI’s ChatGPT and Google’s Bard (Gemini Pro). Eight expert ophthalmologists from different sub-specialties assessed each response, blinded to the source, and ranked them by three metrics—accuracy, comprehensiveness, and clarity, on a 1–5 scale. For accuracy, ChatGPT scored a median of 4.0, whereas Bard scored a median of 3.0. In terms of comprehensiveness, ChatGPT achieved a median score of 4.5, compared to Bard which scored a median of 3.0. Regarding clarity, ChatGPT maintained a higher score with a median of 5.0, compared to Bard’s median score of 4.0. All comparisons were statistically significant (p < 0.001). AI-based chat bots can provide relatively accurate and clear responses for addressing common ophthalmological inquiries. ChatGPT surpassed Bard in all measured metrics. While these AI models exhibit promise, further research is indicated to improve their performance and allow them to be used as a reliable medical tool.