Accuracy of a popular online symptom checker for ophthalmic diagnoses

Abstract Body:

Purpose: To evaluate the diagnostic accuracy, triage urgency, and interuser variability of a popular online symptom checker for common ophthalmic presentations

Study Design: Cross-sectional descriptive study

Methods: Forty-two validated clinical vignettes of ophthalmic complaints were generated and distilled to their core presenting symptoms. Cases were entered into WebMD® symptom checker by both medically trained and non-medically trained personnel. Output from the symptom checker including number of symptoms, ranking and list of diagnoses, and triage urgency was recorded.

Results: The mean number of symptoms entered was 3.6±1.6 (range 1-8), of which a mean of 0.5±0.8 (range 0-3) were extra-ocular. The median number of diagnoses generated by the symptom checker was 26.8±21.8 (range 1-99). The primary diagnosis by the symptom checker was correct in 10/42 (24%) of cases. The correct diagnosis was included in the symptom checker's top 3 diagnoses in 16/42 (38%) of cases. The correct diagnosis was not included in the symptom checker's list at all in 19/42 (45%) of cases. The average position on the differential list generated by the symptom checker when the correct diagnosis was appropriately listed was 4.8±8.2 (range 1-39). The most common primary diagnosis made by the symptom checker was, "nearsightedness". Fourteen of 17 cases where the primary diagnosis' triage urgency was incorrect would have led to an urgent case being triaged as non-urgent. The symptom checker performed better in diagnostic accuracy of non-urgent conditions. Interuser variability for the correct diagnosis being in the top 3 listed was strong (cohen's kappa = 0.84).

Conclusions: As more patients present with self-guided research of their medical symptoms, it is important for ophthalmologists to be familiar with the features and limitations of online symptom checkers as popular tools. While online symptom checkers can arrive at the correct clinical diagnosis, a significant proportion of diagnoses are not captured. As a particularly visual specialty, with similar common symptomatic presentations of distinct diseases, ophthalmology may represent a particularly challenging field for internet based symptom checkers to excel in. More research to reflect the real-life application of internet diagnostic resources is required.