Evaluation of the quality of information provided by ChatGPT on pelvic and acetabular surgery

Conor J. Kilkenny; Martin S. Davey; David O'Sullivan; Conor Medlar; Conor O’ Driscoll; Brendan O'Daly

doi:10.1016/j.jorep.2025.100561

View/Download PDF

Buy Reprints

PDF

Translate this page into:

Research Article

2025

:4;

100561

doi:

10.1016/j.jorep.2025.100561

Evaluation of the quality of information provided by ChatGPT on pelvic and acetabular surgery

Conor J. Kilkenny^a,b,, Martin S. Davey^a, David O'Sullivan^a, Conor Medlar^a, Conor O’ Driscoll^a, Brendan O'Daly^a

a Department of Orthopaedics, Tallaght University Hospital, Dublin 24, D24 NR0A, Tallaght, Ireland

b Royal College of Surgeons in Ireland, 123 St Stephen's Green, Dublin 2, Ireland

⁎Corresponding author: Conor J. Kilkenny. conor.kilkenny@tuh.ie

Received: 2024-11-21, Accepted: 2025-1-19, Published: 2025-1-26

The Authors

Disclaimer:
This article was originally published by Reed Elsevier India Pvt. Ltd. and was migrated to Scientific Scholar after the change of Publisher.

Abstract

Pelvic and Acetabular fractures are complex injuries commonly caused by both high-energy mechanisms in young patients and low velocity trauma in older patients, which can result in significant morbidity and mortality for both. This study aimed to assess the information provided by ChatGPT on pelvic and acetabular surgery using standard scoring systems to assess the quality, reliability, and readability of the content. We hypothesized that while ChatGPT would generate information of high quality, its readability would be low.

An open AI model (ChatGPT) was used to answer 20 commonly asked questions from patients about Pelvic and Acetabular surgery. These answers were evaluated for medical accuracy, quality, and readability using the JAMA Benchmark criteria, DISCERN score, Flesch-Kincaid Reading Ease Score (FRES), and Grade Level (FKGL).

The JAMA Benchmark criteria score was 0, the lowest score, indicating no reliable resources cited. The DISCERN score was 42.5, which scores information of fair quality. The area the open AI model scored lowest was the reliability portion of the DISCERN score due to a lack of resources. The FRES was 52.9, and the FKGL was at a 10th-12th grade reading level.

A 10th to 12th grade reading level was required to comprehend the information provided by ChatGPT with regards to Pelvic and acetabular surgery, and the evidence supplied was of fair quality. With no citations provided, it remains unclear where these answers originate and how scientifically valid they are with respect to the latest literature. In spite of this, ChatGPT did safety-net patients by encouraging the importance of further discussion with a surgeon.

Keywords

Artificial intelligence

Pelvis

Acetabulum

Trauma

Readability

Show Related Articles from PubMed

1 Introduction

Pelvic and Acetabular fractures are complex injuries commonly caused by both high-energy mechanisms in young patients and low velocity trauma in older patients, which can result in significant morbidity and mortality for both.¹ The incidence of acetabular fractures has been estimated at 0.003 %,² with pelvic ring fractures representing 3 % of blunt skeletal injuries.³ Road Traffic accidents (RTA) are the most common cause of major pelvic ring injuries.^4–6 Pelvic fractures are the third most common cause of death post RTA, behind injuries to the central nervous system and the chest.⁶ Although pelvic and acetabular fractures may often be managed conservatively, these injuries can have devastating long-term physical, social and financial implications for patients and their families.⁷ Surgically restoring the bony pelvic and acetabular anatomy can help facilitate early mobilisation, and prevent immobilisation-associated complications.⁸

Keulers et al. has shown that surgeons under-appreciate the level of information patients desire prior to surgery.⁹ Given the impact these injuries can have, many patients and family members turn to the internet for further information about their condition or surgery. With the advances in technology patients are now better informed regarding their medical care than previously.¹⁰^,¹¹ In this current technological era we have recently seen an emergence in the use of Artificial Intelligence (AI)-driven ChatBots like ChatGPT. However, no current standards exist to verify the accuracy or reliability of information provided by ChatGPT, with research suggesting further governance is required to ensure the quality and trustworthiness of AI-generated responses.¹²

While prior studies have examined the quality and readability of the responses generated by Chat GPT to questions regarding hip, knee and upper limb pathologies,^13–17 to our knowledge, this is the first study assessing the quality of information generated by ChatGPT on pelvic and acetabular surgery. This study aimed to assess the information provided by ChatGPT on pelvic and acetabular surgery using standard scoring systems to assess the quality, reliability, and readability of the content. We hypothesized that while ChatGPT would generate information of high quality, its readability would be low.

2 Methods

2.1

2.1 Artificial Intelligence

On September 21st^, 2024, an AI software (ChatGPT) answered commonly asked questions by patients in relation to pelvic and acetabular surgery. A number of validated scoring systems assessed the readability and quality of the answers provided. The questions were based on previous studies examining patient expectations following pelvis and acetabular surgery and the Hospital of special surgery website.¹⁸ The answers provided by ChatGPT were assessed independently by three named authors CJK, MD and DOS.

2.2

2.2 Quality analysis

In order to assess the readability and quality of the information provided by the AI software two validated tools were. This included the DISCERN¹⁹ and the Journal of the American Medical Association (JAMA)²⁰ benchmark criteria. The DISCERN score evaluates the quality of written health information and ranges from 16 to 80 points. Each question is scored from 1 to 5 points. The DISCREN tool is divided into three sections. Eight questions assess reliability, seven focus on treatment information, and a final question provides an overall quality rating. Weil et al. excluded the final question and categorised the Discern score into the following categories.²¹ Scores of 63–75 are considered “excellent”, scores of 51–62 are considered “good”, scores of 39–50 are considered “fair”, scores of 27–38 are considered poor, and scores of 15–26 are considered “very poor”.

The JAMA criteria consists of 4 accepted standards that evaluate the quality of information. The four criteria are authorship, attribution, disclosure and currency. Each criterion scores 1 point. Authorship requires clear identification of the author's name, affiliations, and credentials. Attribution involves appropriately citing sources and documenting relevant copyright information. Disclosure highlight any potential conflicts of interest, while currency refers to the date of the information provided, ensuring it is up to date and relevant.

2.3

2.3 Readability analysis

The readability was evaluated using the Flesch-Kincaid Reading Ease Score (FRES) and Grade Level (FKGL), with scores calculated through an online tool: https://goodcalculators.com/flesch-kincaid-calculator/. The FKGL is an updated version of the FRES. The FRES score ranges from 0 to 100, with 0 representing that the text is unreadable and a score of 100 would indicate it is very easy to read. The FKGL represents the level of education required to understand the material. The higher the FKGL the more complex it is to understand the content.²²

2.4

2.4 Statistical analysis

Qualitative statistical analysis was conducted using Microsoft Excel. Inter-rater reliability (IRR) was calculated for the DISCERN score, with the with IRR interpreted as between 0.01 and 0.20 indicates little to no agreement, 0.21 to 0.40 indicates fair agreement, 0.41 to 0.60 indicates moderate agreement, 0.61 to 0.80 indicates substantial agreement, and 0.81 to 1.00 indicates almost perfect agreement.²³

3 Results

3.1

3.1 Quality and readability analysis

The JAMA Benchmark criteria were scored as 0 due to the lack of references provided by ChatGPT to source material used in compiling the answers. The DISCERN score was 42.5, with an IRR of 0.6. The areas where it lost marks were also due to the lack of available source material and information not being supported by background literature. The FRES was 52.9, indicating the information provided is “fairly difficult to read”, and the FKGL was 8.4, indicating a reading level of 10th to 12th grade. A complete list of answers generated by ChatGPT is further demonstrated in Appendix 1.

4 Discussion

The most important finding of this study is that the quality of answers provided by AI was considered to be of “fair” quality, but was unable to reliably identify information sources and required a high reading level to be interpreted. ChatGPT can assist patient education in certain patient groups with regards to Pelvic and Acetabular surgery, however the source and validity of the AI platforms certainly warrants questionning. The AI software also routinely recommends that questions relating to their condition should be clarified further with a surgeon or doctor.

There is often a considerable difference between patients' reading abilities and the readability of educational materials provided.²⁴ It has been previously shown that clinicians underestimate the level of information and detail that patients require in relation to their condition and proposed treatment plan.⁹ This often can lead patients to access information online or more recently to AI platforms, such as ChatGPT, in hope of better understand clinical information provided by healthcare professionals. ChatGPT has surged in popularity not only amongst the general population but also within the medial field, with physicians and patients alike often accessing this prior to the internet itself.²⁵

Although ChatGPT aims to provide its users with accurate and reliable information based on the data which it has been trained on, however a lack of citations has called the validity of its responses into account.²⁶ In spite of this, ChatGPT was found to be capable of passing the United States Medical Licensing Examination(USMLE)²⁷ while also scoring 67.5 % in an Orthopaedic FRCS exam (an exit exam prior to becoming an Orthopaedic consultant in Ireland and the United Kingdom).²⁸ AI has shown potential as an adjunct in medical academic writing, however care must be taken with respect to potential plagiarism.²⁹ Additionally, it is deemed highly likely that AI may play a role in assisting radiographic diagnoses in future, with ever-growing radiomic studies being published.³⁰^,³¹

As the apparent use of ChatGPT and AI is only expected to grow and expand, it is crucial that the readability and quality of its answers are assessed for various medical conditions. In 2024, 54 % of adults within the US are thought to have a literacy below a 6th grade level.³² A Survey conducted in Ireland by the Central statistics office in 2012 found that nearly one-fifth of the population were at or below level one on a literacy scale, which suggests a potential issue with understanding basic written language.³³ This suggest that the reading level required to interpret AI responses with regards to pelvic and acetabular surgery could be too high for the average person. Likewise, similar studies in other Orthopaedic subspecialties have found that higher reading levels were required than the recommended 5th grade as advised by Weiss et al.¹³^,¹⁴^,³⁴ Previous work by Wally et al. encourages patients when searching medical information online to only trust websites with the HON rating.³⁵ The HON certification process ensures certain standards are met with regards to the dissemination of medical information, which has not yet been applied to ChatGPT.³⁶ Therefore, continued exploration and validation of these AI platform must be considered a crucial area of study prior to their use as standard in informing patients and physicians alike in providing world class care to prospective patients.

A previous readability study by Murphy et al. looked at the readability and quality of online information in relation to pelvic and acetabular fractures.³⁷ Similar to our study, Murphy et al. found that a high reading level was required to comprehend the information. The mean FRES score for acetabular internet information was 43.83 while the mean FRES score for pelvic internet information was 41.38. This is quite similar to the FRES in this study of 52.9. However, this may suggest that the information provided by ChatGPT in relation to pelvic and acetabular fractures may be more complex than the information provided by the more “traditional” internet. This is concerning and suggests that information on this area accessed through the internet is too complex for many patients.

5 Limitations

There are number of limitations in this study, many of them inherent to the study design. Firstly, this is an analysis of an AI software which has its own innate constraints. The software provides no source information, so it is unclear where this information originates from and what data this software is trained on. The software could also develop overtime and thus improve the quality and readability of answers provided to the same question over time. Bias may also be present in the generation of questions provided to ChatGPT and it is possible these biases could be reflected in the answers provided by ChatGPT. While the quality of evidence can be considered to be of fair quality, the AI software lacks empathy, emotional intelligence and the ability to contextualise the information to a given patient.

6 Conclusion

Ethical statement

Not applicable.

Funding

The authors have no relevant financial or non-financial interests to disclose.

CRediT authorship contribution statement

Conor J. Kilkenny: Investigation, Writing – original draft, Visualization. Martin S. Davey: Investigation, Methodology, Writing – review & editing. David O'Sullivan: Conceptualization, Writing – review & editing. Conor Medlar: Methodology, Formal analysis, Investigation, Writing – review & editing. Conor O’ Driscoll: Investigation, Formal analysis, Writing – review & editing. Brendan O'Daly: Conceptualization, Writing – review & editing, Supervision.

References

Davarinos N., Ellanti P., Morris S., Mc Elwain J.P., . Epidemiology of pelvic and acetabular trauma in a Dublin tertiary hospital: a 10-year experience. Ir J Med Sci. 2012 Jun;181(2):243-246.
[Google Scholar]
Muhammad Atif OH., Baloch Naveed, Umer Masood, . A comprehensive basic understanding of pelvis and acetabular fractures after high-energy trauma with associated injuries: Narrative review of targeted literature. JPMA (J Pak Med Assoc). 2020;70
[Google Scholar]
Papakostidis C., Giannoudis P.V., . Pelvic ring injuries with haemodynamic instability: efficacy of pelvic packing, a systematic review. Injury. 2009 Nov;40(Suppl 4):S53-S61.
[Google Scholar]
Laird A., Keating J.F., . Acetabular fractures: a 16-year prospective epidemiological study. J Bone Joint Surg Br. 2005 Jul;87(7):969-973.
[Google Scholar]
Gansslen Tp A., Paul Ch, Lobenhoffer Ph, Tscheme H., . Epidemiology of pelvic ring injuries. Injury 1996
[Google Scholar]
Giannoudis P.V., Grotz M.R., Tzioupis C., et al . Prevalence of pelvic fractures, associated injuries, and mortality: the United Kingdom perspective. J Trauma. 2007 Oct;63(4):875-883.
[Google Scholar]
Geoghegan J.M., Longdon E.J., Hassan K., Calthorpe D., . Acetabular fractures in the UK. What are the numbers? Injury. 2007 Mar;38(3):329-333.
[Google Scholar]
Devaney G.L., Bulman J., King K.L., Balogh Z.J., . Time to definitive fixation of pelvic and acetabular fractures. J Trauma Acute Care Surg. 2020 Oct;89(4):730-735.
[Google Scholar]
Keulers B.J., Scheltinga M.R., Houterman S., Van Der Wilt G.J., Spauwen P.H., . Surgeons underestimate their patients' desire for preoperative information. World J Surg. 2008 Jun;32(6):964-970.
[Google Scholar]
Shah N.V., Gold R., Dar Q.A., Diebo B.G., Paulino C.B., Naziri Q., . Smart technology and orthopaedic surgery: current Concepts regarding the impact of Smartphones and Wearable technology on our patients and Practice. Curr Rev Musculoskelet Med. 2021 Dec;14(6):378-391.
[Google Scholar]
Davey M.S., Coveney E., Rowan F., Cassidy J.T., Cleary M.S., . Virtual fracture Clinics in orthopaedic surgery - a systematic review of current evidence. Injury. 2020 Dec;51(12):2757-2762.
[Google Scholar]
Kim J.K., Chua M., Rickard M., Lorenzo A., . ChatGPT and large language model (LLM) chatbots: the current state of acceptability and a proposal for guidelines on utilization in academic medicine. J Pediatr Urol. 2023 Oct;19(5):598-604.
[Google Scholar]
Crook B.S., Park C.N., Hurley E.T., Richard M.J., Pidgeon T.S., . Evaluation of online artificial intelligence-generated information on common Hand procedures. J Hand Surg Am. 2023 Nov;48(11):1122-1127.
[Google Scholar]
Warren E., Hurley E.T., Park C.N., et al . Evaluation of information from artificial intelligence on rotator cuff repair surgery. JSES Int. 2024 Jan;8(1):53-57.
[Google Scholar]
Fahy S., Oehme S., Milinkovic D., Jung T., Bartek B., . Assessment of quality and readability of information provided by ChatGPT in relation to Anterior cruciate Ligament injury. J Pers Med. 2024 Jan 18;14(1)
[Google Scholar]
AlShehri Y., McConkey M., Lodhia P., . ChatGPT provides Satisfactory but Occasionally Inaccurate answers to common patient hip Arthroscopy questions. Arthroscopy 2024
[Google Scholar]
Wright B.M., Bodnar M.S., Moore A.D., et al . Is ChatGPT a trusted source of information for total hip and knee arthroplasty patients? Bone Jt Open. 2024 Feb 15;5(2):139-146.
[Google Scholar]
Helfet D.L., . Pelvic fractures/acetabular fractures. Hospital for Special Surgery 2019
[Google Scholar]
Deborah Charnock SS., Gill Needham, Gann Robert, . DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105-111.
[Google Scholar]
Silberg William M., George D., Lundberg M.R.A.M., . PhD. Assessing, Controlling, and Assuring the Quality of Medical Information on the Internet Caveant Lector et Viewor—Let the Reader and Viewer Beware. JAMA 1997
[Google Scholar]
Weil A.G., Bojanowski M.W., Jamart J., Gustin T., Leveque M., . Evaluation of the quality of information on the Internet available to patients undergoing cervical spine surgery. World Neurosurg. 2014 Jul-Aug;82(1-2):e31-e39.
[Google Scholar]
Friedman D.B., Hoffman-Goetz L., . A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006 Jun;33(3):352-373.
[Google Scholar]
McHugh M.L., . Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276-282.
[Google Scholar]
Badarudeen S., Sabharwal S., . Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010 Oct;468(10):2572-2580.
[Google Scholar]
Tan S., Xin X., Wu D., . ChatGPT in medicine: prospects and challenges: a review article. Int J Surg. 2024 Jun 1;110(6):3701-3706.
[Google Scholar]
Dave T., Athaluri S.A., Singh S., . ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6
[Google Scholar]
Kung T.H., Cheatham M., Medenilla A., et al . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023 Feb;2(2)
[Google Scholar]
Saad A., Iyengar K.P., Kurisunkal V., Botchu R., . Assessing ChatGPT's ability to pass the FRCS orthopaedic part A exam: a critical analysis. Surgeon. 2023 Oct;21(5):263-266.
[Google Scholar]
Sanchez-Sotelo J., Jed Kuhn J.E., Mallon W.J., . Artificial intelligence and the creation of scientific papers. J Shoulder Elbow Surg. 2023 Apr;32(4):685-686.
[Google Scholar]
Davey M.S., Davey M.G., Ryan E.J., Hogan A.M., Kerin M.J., Joyce M., . The use of radiomic analysis of magnetic resonance imaging in predicting distant metastases of rectal carcinoma following surgical resection: a systematic review and meta-analysis. Colorectal Dis. 2021 Dec;23(12):3065-3072.
[Google Scholar]
Davey M.S., Davey M.G., Kenny P., Gheiti A.J.C., . The use of radiomic analysis of magnetic resonance imaging findings in predicting features of early osteoarthritis of the knee-a systematic review and meta-analysis. Ir J Med Sci. 2024 Oct;193(5):2525-2530.
[Google Scholar]
[Google Scholar]
[Google Scholar]
Barry D., Weiss C.C., . Communicating WITH PATIENTS WHO CANNOT READ. N Engl J Med. 1997;337(4)
[Google Scholar]
Wally M.K., Bemenderfer T., McKnight R.R., et al . Quality and content of internet-based information for Osteoporosis and Fragility fracture diagnoses. J Am Acad Orthop Surg Glob Res Rev. 2021 Feb 12;5(2)
[Google Scholar]
Sabine Laversin VB., Gaudinat Arnaud, Simonet Maria-Ana, Boyer Célia, . Improving the Transparency of health information found on the internet through the Honcode: a Comparative study. 2011
[Google Scholar]
Murphy E.P., Fenelon C., Murphy F., et al . Does Google have the answers? The internet-based information on pelvic and acetabular fractures. Cureus. 2019 Oct 21;11(10)
[Google Scholar]

Introduction

Methods

Artificial Intelligence
Quality analysis
Readability analysis
Statistical analysis

Results

Quality and readability analysis

Discussion

Limitations

Conclusion

Ethical statement

Informed consent

Funding

CRediT authorship contribution statement

Fulltext Views
93

PDF downloads
4

View/Download PDF
Download Citations

BibTeX
RIS

Show Sections

[1] Davarinos N., Ellanti P., Morris S., Mc Elwain J.P., . Epidemiology of pelvic and acetabular trauma in a Dublin tertiary hospital: a 10-year experience. Ir J Med Sci. 2012 Jun;181(2):243-246.
[Google Scholar]

[2] Muhammad Atif OH., Baloch Naveed, Umer Masood, . A comprehensive basic understanding of pelvis and acetabular fractures after high-energy trauma with associated injuries: Narrative review of targeted literature. JPMA (J Pak Med Assoc). 2020;70
[Google Scholar]

[3] Papakostidis C., Giannoudis P.V., . Pelvic ring injuries with haemodynamic instability: efficacy of pelvic packing, a systematic review. Injury. 2009 Nov;40(Suppl 4):S53-S61.
[Google Scholar]

[4] Laird A., Keating J.F., . Acetabular fractures: a 16-year prospective epidemiological study. J Bone Joint Surg Br. 2005 Jul;87(7):969-973.
[Google Scholar]

[5] Gansslen Tp A., Paul Ch, Lobenhoffer Ph, Tscheme H., . Epidemiology of pelvic ring injuries. Injury 1996
[Google Scholar]

[6] Giannoudis P.V., Grotz M.R., Tzioupis C., et al . Prevalence of pelvic fractures, associated injuries, and mortality: the United Kingdom perspective. J Trauma. 2007 Oct;63(4):875-883.
[Google Scholar]

[7] Geoghegan J.M., Longdon E.J., Hassan K., Calthorpe D., . Acetabular fractures in the UK. What are the numbers? Injury. 2007 Mar;38(3):329-333.
[Google Scholar]

[8] Devaney G.L., Bulman J., King K.L., Balogh Z.J., . Time to definitive fixation of pelvic and acetabular fractures. J Trauma Acute Care Surg. 2020 Oct;89(4):730-735.
[Google Scholar]

[9] Keulers B.J., Scheltinga M.R., Houterman S., Van Der Wilt G.J., Spauwen P.H., . Surgeons underestimate their patients' desire for preoperative information. World J Surg. 2008 Jun;32(6):964-970.
[Google Scholar]

[10] Shah N.V., Gold R., Dar Q.A., Diebo B.G., Paulino C.B., Naziri Q., . Smart technology and orthopaedic surgery: current Concepts regarding the impact of Smartphones and Wearable technology on our patients and Practice. Curr Rev Musculoskelet Med. 2021 Dec;14(6):378-391.
[Google Scholar]

[11] Davey M.S., Coveney E., Rowan F., Cassidy J.T., Cleary M.S., . Virtual fracture Clinics in orthopaedic surgery - a systematic review of current evidence. Injury. 2020 Dec;51(12):2757-2762.
[Google Scholar]

[12] Kim J.K., Chua M., Rickard M., Lorenzo A., . ChatGPT and large language model (LLM) chatbots: the current state of acceptability and a proposal for guidelines on utilization in academic medicine. J Pediatr Urol. 2023 Oct;19(5):598-604.
[Google Scholar]

[13] Crook B.S., Park C.N., Hurley E.T., Richard M.J., Pidgeon T.S., . Evaluation of online artificial intelligence-generated information on common Hand procedures. J Hand Surg Am. 2023 Nov;48(11):1122-1127.
[Google Scholar]

[14] Warren E., Hurley E.T., Park C.N., et al . Evaluation of information from artificial intelligence on rotator cuff repair surgery. JSES Int. 2024 Jan;8(1):53-57.
[Google Scholar]

[15] Fahy S., Oehme S., Milinkovic D., Jung T., Bartek B., . Assessment of quality and readability of information provided by ChatGPT in relation to Anterior cruciate Ligament injury. J Pers Med. 2024 Jan 18;14(1)
[Google Scholar]

[16] AlShehri Y., McConkey M., Lodhia P., . ChatGPT provides Satisfactory but Occasionally Inaccurate answers to common patient hip Arthroscopy questions. Arthroscopy 2024
[Google Scholar]

[17] Wright B.M., Bodnar M.S., Moore A.D., et al . Is ChatGPT a trusted source of information for total hip and knee arthroplasty patients? Bone Jt Open. 2024 Feb 15;5(2):139-146.
[Google Scholar]

[18] Helfet D.L., . Pelvic fractures/acetabular fractures. Hospital for Special Surgery 2019
[Google Scholar]

[19] Deborah Charnock SS., Gill Needham, Gann Robert, . DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53:105-111.
[Google Scholar]

[20] Silberg William M., George D., Lundberg M.R.A.M., . PhD. Assessing, Controlling, and Assuring the Quality of Medical Information on the Internet Caveant Lector et Viewor—Let the Reader and Viewer Beware. JAMA 1997
[Google Scholar]

[21] Weil A.G., Bojanowski M.W., Jamart J., Gustin T., Leveque M., . Evaluation of the quality of information on the Internet available to patients undergoing cervical spine surgery. World Neurosurg. 2014 Jul-Aug;82(1-2):e31-e39.
[Google Scholar]

[22] Friedman D.B., Hoffman-Goetz L., . A systematic review of readability and comprehension instruments used for print and web-based cancer information. Health Educ Behav. 2006 Jun;33(3):352-373.
[Google Scholar]

[23] McHugh M.L., . Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276-282.
[Google Scholar]

[24] Badarudeen S., Sabharwal S., . Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010 Oct;468(10):2572-2580.
[Google Scholar]

[25] Tan S., Xin X., Wu D., . ChatGPT in medicine: prospects and challenges: a review article. Int J Surg. 2024 Jun 1;110(6):3701-3706.
[Google Scholar]

[26] Dave T., Athaluri S.A., Singh S., . ChatGPT in medicine: an overview of its applications, advantages, limitations, future prospects, and ethical considerations. Front Artif Intell. 2023;6
[Google Scholar]

[27] Kung T.H., Cheatham M., Medenilla A., et al . Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023 Feb;2(2)
[Google Scholar]

[28] Saad A., Iyengar K.P., Kurisunkal V., Botchu R., . Assessing ChatGPT's ability to pass the FRCS orthopaedic part A exam: a critical analysis. Surgeon. 2023 Oct;21(5):263-266.
[Google Scholar]

[29] Sanchez-Sotelo J., Jed Kuhn J.E., Mallon W.J., . Artificial intelligence and the creation of scientific papers. J Shoulder Elbow Surg. 2023 Apr;32(4):685-686.
[Google Scholar]

[30] Davey M.S., Davey M.G., Ryan E.J., Hogan A.M., Kerin M.J., Joyce M., . The use of radiomic analysis of magnetic resonance imaging in predicting distant metastases of rectal carcinoma following surgical resection: a systematic review and meta-analysis. Colorectal Dis. 2021 Dec;23(12):3065-3072.
[Google Scholar]

[31] Davey M.S., Davey M.G., Kenny P., Gheiti A.J.C., . The use of radiomic analysis of magnetic resonance imaging findings in predicting features of early osteoarthritis of the knee-a systematic review and meta-analysis. Ir J Med Sci. 2024 Oct;193(5):2525-2530.
[Google Scholar]

[32] [Google Scholar]

[33] [Google Scholar]

[34] Barry D., Weiss C.C., . Communicating WITH PATIENTS WHO CANNOT READ. N Engl J Med. 1997;337(4)
[Google Scholar]

[35] Wally M.K., Bemenderfer T., McKnight R.R., et al . Quality and content of internet-based information for Osteoporosis and Fragility fracture diagnoses. J Am Acad Orthop Surg Glob Res Rev. 2021 Feb 12;5(2)
[Google Scholar]

[36] Sabine Laversin VB., Gaudinat Arnaud, Simonet Maria-Ana, Boyer Célia, . Improving the Transparency of health information found on the internet through the Honcode: a Comparative study. 2011
[Google Scholar]

[37] Murphy E.P., Fenelon C., Murphy F., et al . Does Google have the answers? The internet-based information on pelvic and acetabular fractures. Cureus. 2019 Oct 21;11(10)
[Google Scholar]

Evaluation of the quality of information provided by ChatGPT on pelvic and acetabular surgery

Abstract