The First Look at Code-Free Custom GPTs in Medicine: An Ophthalmology Perspective (Preprint)

Abstract

BACKGROUND OpenAI recently introduced the ability to create custom GPTs in its advanced model (GPT-4). They do not need any coding knowledge; therefore, clinicians can easily create them without any programming experience. Since they have plain text customization functionality and information expansion capabilities with file upload, they can overcome some drawbacks of the standard GPT-4. OBJECTIVE Our aim was to use ophthalmologic GPTs as a base and examine their general properties, advantages and disadvantages, and potential practical uses in detail. METHODS Data collection took place on January 20 and 21, 2024, and custom GPTs were found by entering ophthalmology keywords into the “Explore GPTS” section of the website. General and specific features of custom GPTs were recorded, such as knowledge other than GPT-4 training data. The instruction and description sections, where users can get the most information about a custom GPT, were analyzed for compatibility using the Likert scale. We analyzed two custom GPTs with the highest Likert score in detail. We attempted to create a malicious GPT to test security features. RESULTS We analyzed 22 ophthalmic custom GPTs, of which 55% were for general use and the most common subspecialty was glaucoma (18%). Over half (55%) contained knowledge other than GPT-4 training data. The representation of the instructions through the description was between “Moderately representative” and “Very representative” with a median Likert score of 3.5 (IQR 3.0 – 4.0). The instruction word count was significantly associated with Likert scores (P = 0.03, 95% CI, 0.050-0.739). Instruction length for high Likert score description was generally between 241 and 338 words. Tested custom GPTs demonstrated potential for specific conversational tone, information, retrieval and combining knowledge from an uploaded source. With these security settings creating a malicious GPT was possible. CONCLUSIONS This is the first study to examine the GPT store for a medical field. Publicly available custom GPTs for ophthalmology are available in the GPT store. Studies are needed to see their use in other medical areas. Custom GPTs can be put into practice immediately. Reliable GPTs can be more useful for a specific aim than classical GPT-4. However, more detailed studies are needed to test their capabilities. The security features currently appear to be rather limited. It may be helpful for the user to review the instruction section before using a custom GPT. CLINICALTRIAL this is not a trial study