1 Introduction

Artificial intelligence (AI) has become an integral part of medicine, offering tools to analyze and synthesize large volumes of information to support clinical decision-making, research, and education.¹ In orthopaedics, AI is transforming how clinicians learn and work by simplifying tasks and improving efficiency in medical processes. Studies have demonstrated that generative AI models can perform at a level comparable to human experts on board-style orthopaedic surgery questions, underscoring their potential in medical education and assessment.^2,3

AI has also shown significant promise in clinical practice and orthopaedic research. Currently, it is used to assist in diagnostic imaging, surgical planning, and automating complex measurements such as of joint alignment on radiographs.⁴ Recent advancements, particularly in Deep Learning Networks (DLNs), have enabled AI systems to solve complex problems and generate human-like responses in real time. Large Language Models (LLMs), such as ChatGPT by OpenAI, are a subset of DLNs designed to process vast amounts of data, generate contextually appropriate text, and engage in dynamic interactions.⁵ ChatGPT has been utilized in orthopaedics to create educational materials for patients, streamline manuscript drafting, offer suggestions for scientific writing, and format citations, significantly reducing the workload for researchers.⁶

Despite these benefits, concerns remain regarding the accuracy and reliability of research generated by ChatGPT. Articles produced by generative AI are prone to bias, inaccuracies, and misleading information.^6,7 ChatGPT, like other generative AI models, are known to “hallucinate” – producing nonsensical or inaccurate responses that are plausible at face value.⁸ This study aims to establish the hallucination rate of ChatGPT 3.5 and 4.0 in generating primary peer-reviewed citations within orthopaedic arthroplasty literature and to identify the prompt factors most influential in producing accurate responses.

2 Materials and methods

Generative Citations: In August 2024, we tasked ChatGPT 3.5 and 4.0 to generate presentation outlines with corresponding reference lists of 10 citations following a simple, medium, and complex prompt. We studied 4 common lower limb arthroplasty surgeries: primary total knee arthroplasty (TKA), primary total hip arthroplasty (THA), revision TKA, and revision THA. The simple prompt used, adjusted for each surgical indication, was: “Write an outline for an orthopedic presentation about primary total knee arthroplasty (TKA), include 10 sources”. We then refined the prompt to specify source type and citation style, using the medium prompt: “Write an outline for an orthopedic presentation about primary total knee arthroplasty (TKA) in American Medical Association (AMA) citation format, include 10 primary peer reviewed sources”. Finally, we employed a complex prompt that included additional context about the intended speaker, location, time, and intended audience: “I am a PGY4 orthopedic resident presenting a research paper on primary total knee arthroplasty (TKA) at the 2024 annual American Academy of Orthopaedic Surgeons conference. Create a topic outline for my paper on primary TKA in American Medical Association (AMA) citation format, include 10 primary peer reviewed sources”.

ChatGPT use:www.chatgpt.com 4.0 a monthly subscription-based AI system.

IRB status: This study was determined to be IRB-exempt by our Institutional Review Board due to its use of open-source data and the absence of patient-identifiable information.

Reference categorization: Papers were searched in PubMed, google, and google scholar. Found references were recorded for: publishing journal, journal impact factor, author list, publication year, and title. These accurately sourced citations were compared with ChatGPT's generated citation. Generated references were categorized as “Nonexistent,” “Secondary Work,” “Improperly Cited,” or “Properly Cited”. References categorized as “Nonexistent” were not found in all three search engine databases. “Secondary Work” articles were classified as non-primary sourced material (e.g., textbooks or websites). “Improperly Cited” references were primary works identified in at least one of the search engines, but the generated citation had inaccuracies in the title, author list, publishing year, publishing journal, or citation format when compared to the accurate source material. “Properly Cited” articles were primary error-free works and verified through citation manager systems (e.g., EndNote and PubMed). Common errors for generated citations pertaining to publishing journal, author list, publication year, and title were noted for each identified reference.

Data Analyses: Data were analyzed using IBM SPSS 29 (International Business Machines Corporation, Armonk, NY) to generate descriptive and comparative statistics including frequency, median, Chi-square, and binomial logistic regression for odds ratios. Statistical significance was defined as P-values <0.05 with 95 % Confidence Intervals (CI). Data tables and graphs were generated in Microsoft Excel version 16.87.

3 Results

A total of 240 generated citations were collected between both chatbots. Each ChatGPT version generated 120 new citations consisting of 30 citations per arthroplasty procedure evenly distributed based on prompt specificity.

Of the 240 generated citations between both GPT versions, 56 were classified as Nonexistent (23.3 %), 76 as Secondary Works (31.7 %), 33 as Improperly Cited (13.8 %), and 75 as Properly Cited (31.3 %), which did not statistically differ between groups (p = 0.174) (Fig. 1). GPT-3.5 generated more Secondary Work citations than GPT-4 (35.8 % vs. 27.5 %, p = 0.174) which was offset by a comparatively lower rate of Nonexistent citations (17.5 % vs. 29.2 %, p = 0.174). References categorized as Improperly Cited and Properl1 Cited were similar between GPT versions.

Close

Fig. 1 ChatGPT's Citation Accuracy: Source Categorization of Generated Citations by ChatGPT Version 3.5 vs. 4.0. Pearson Chi-Square value = 4.966, p = 0.174.

Export to PPT

Among common arthroplasty procedures, primary and revision TKA had more citations characterized as Nonexistent (26.7 % and 35 %, p = 0.184) while primary and revision THA which had more Properly Cited works (35 % and 40 %, p = 0.184) (Fig. 2).

Close

Fig. 2 Citation Accuracy for Common Arthroplasty Procedures: Categorization of ChatGPT Citations for Primary THA, Primary TKA, Revision THA, and Revision TKA. Pearson Chi-Square value = 12.543, p = 0.184.

Export to PPT

GPTs demonstrated significant responsiveness to graded prompt specificity. Generated citations from simple prompts were categorized exclusively as Nonexistent (10 %, p < 0.001) or Secondary Work (90 %, p < 0.001) (Fig. 3). Medium prompts with added specifiers for citation style and source material returned more legitimate primary works (66.3 %, p < 0.001) but with a high rate of citation errors (39.6 %, p < 0.001). Complex prompts that included additional factors for speaker identity, time, location, and prospective audience generated the greatest number of legitimate primary works (68.8 %, p < 0.001) with significantly less errors (21.8 %, p < 0.001). However, even with these added specifiers, the complex prompts generated Nonexistent references at a rate of 30 % (ƒ = 24, p < 0.001).

Close

Fig. 3 Teachability of ChatGPT: Citation Accuracy for Simple, Medium, and Complex prompts. Pearson Chi-Square value = 198.271, p < 0.001.

Export to PPT

The median impact factor and publication year for accurate articles were 4.2 (range, 0.7–97.7) and 2009 (range, 1987–2023), respectively. Generative citations commonly exhibited errors, with the highest frequency in publication year (32.4 %), followed by title (28.2), publishing journal (14.6 %), and author list (13.9 %). To identify factors influencing these errors, we first conducted a univariate logistic regression analysis (Table 1). The analysis revealed that errors in publication year were less likely with complex prompts (OR = 0.219, 95 % CI: 0.066–0.722, p = 0.013) but more common in recent publications (OR = 1.181, 95 % CI: 1.099–1.099, p < 0.001). Similarly, errors in the publishing journal were more likely in recent publications (OR = 1.155, 95 % CI: 1.038–1.285, p = 0.008) but were reduced with higher impact factors (OR = 0.489, 95 % CI: 0.307–0.777, p = 0.002). Errors in the author list were also more likely in recent publications (OR = 1.203, 95 % CI: 1.066–1.357, p = 0.003). No significant factors were found to influence title errors. Following this univariate analysis, we conducted a multivariate logistic regression to control for inter-factor relationships. The multivariate model, which included publication year, impact factor, and prompt complexity, yielded similar odds ratios and statistically significant values as those observed in the univariate analysis (Table 1).

4 Discussion

OpenAI's generative chatbot models, such as those of ChatGPT, are capable of analyzing large datasets, and these tools may play a role in streamlining literature searches as a research aid. In this study, the ChatGPT demonstrated strong adaptability to tailored prompts for literature searches on lower limb arthroplasty. Specificity was poor with simple prompts, but optimized prompts significantly improved the yield of accurate and primary source articles, although inaccuracies remained at all levels. Publication year, publication journal, and impact factor were all important factors in determining citation accuracy. ChatGPT 3.5 and 4.0 produced similar outcomes when tasked with generating citations for common lower limb arthroplasty procedures. Initially, GPT-3.5 predominantly generated secondary sources, unless specific input data, such as restricting results to peer reviewed sources, were applied. In contrast, GPT-4 tended to produce more primary sources but also exhibited a higher incidence of hallucinations.

We categorized inaccuracies in the generated reference lists as either total or partial hallucinations. Total hallucinations, labeled as “Nonexistent”, were citations without any identifiable source. Partial hallucinations, labeled as “Improperly Cited”, involved identifiable sources but with minor errors. The overall rate of total hallucinations in our study was 23.3 %, which aligns with previous literature.⁹ As expected, prompt engineering significantly improved the accuracy and relevance of the results.^10,11 According to Giray, an effective prompt consists of four components: instruction, context, input data, and output indicator.¹¹ In our study, adding an output indicator (e.g., “10 primary peer-reviewed sources”) to the prompt increased accuracy to 26.3 %. Further enhancements, such as providing more detailed context (e.g., presenter identity, location, time, and prospective audience), improved primary source legitimacy to 68.8 % of which 78.2 % were citated accurately. These results demonstrate the remarkable adaptability of GPT models in generating more accurate responses with well-engineered prompts. However, even with optimal prompt engineering, the reliability of obtaining accurate primary scientific citations remained close to a coin flip.

We observed minor variations in the citations generated for various arthroplasty procedures. Arthroplasty procedures pertaining to total hip arthroplasty (THA), or revision surgeries had fewer publication-related errors. This may be due to the lower degree of word-sense disambiguation (WSD) required by the language model, as these terms have fewer alternative meanings in the English language compared to primary total knee arthroplasty (TKA), which can be associated with multiple other contexts (e.g., primary colors, or TKA in boxing). The variability in WSD between key phrases may be reduced by capitalizing important words, which can improve named entity recognition (NER).^12,13

Common citation errors were influenced by several factors. Errors in publication year and title were twice as likely as those in the publishing journal and author list. Multivariate regression analysis revealed that more recent publications were significantly more likely to have errors in the publishing journal, author list, and publication year. This is likely due to the pre-trained nature of the chatbots, which rely on a fixed dataset with training cut-offs in September 2021 for GPT-3.5 and April 2023 for GPT-4. Conversely, publications in journals with higher impact factors showed a significantly reduced likelihood of journal-related errors, suggesting that these more prominent journals are more accurately recognized by the chatbots. In summary, primary scientific literature citations generated by GPT-3.5 or GPT-4 should be carefully scrutinized for accuracy, particularly with regard to the title and publication year, especially for works published after 2021.

This work suggests that while the current ChatGPT-3.5 and 4 models are teachable, they still require improvement in reliability and source accuracy. Their ease of use and wide range of applications highlight their potential utility in orthopaedic research. However, researchers must remain vigilant, as the ultimate responsibility for accuracy—and the prevention of false claims or plagiarism—rests with the human authors.¹⁴

This study is limited by its cross-sectional design and narrow scope. While ChatGPT is not the only generative AI tool available, we focused on OpenAI's GPT version due to its broader use and accessibility in the current scientific landscape.¹⁵ Future research could compare the accuracy and reliability of these alternative AI-powered tools for orthopaedic research.

5 Conclusions

Both ChatGPT 3.5 and 4.0 demonstrate significant adaptability when responding to engineered prompts for lower limb arthroplasty. With properly designed prompts, they produce legitimate primary sources more than twice that of hallucinated ones. To optimize the output of these models, scientific writers should provide precise input data and ample relevant context. Due to training cut-offs on fixed datasets, information generated about studies published after 2021 may be inaccurate and should be critically evaluated. Despite their limitations, these models can greatly enhance the efficiency of scientific writing, offering valuable support in the drafting process.

Guardian/parent consent

Not applicable. Patient consent waived.

Credit author statement

None.

Ethical approval and patient consent

This study has been conducted in accordance with the ethical principles mentioned in the Declaration of Helsinski (2013). IRB approval was not required as it does not involve human or animal subjects. Patient informed consent was waived.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work the authors used ChatGPT by OpenAI in order to generate novel arthroplasty citations for study and edit manuscript writing styles. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial or not-for-profit sectors.