1 Introduction

Tibial diaphysis fractures are commonly encountered in orthopaedic practice. Fixation with an intramedullary (IM) nail is the most preferred technique for treating this type of fracture. Delayed union and non-union remain serious issues despite advancements in treatment modalities. The incidence of these complications in fractures fixed with IM nail is reported to be up to 16.7% in the literature.¹ This is a significant cause of morbidity to the patient along with high socioeconomic costs.

Additionally, implant removal may be desired by the patient or indicated in conditions such as severe anterior knee pain, infection, implant failure, and hardware prominence. In such instances, fracture union must be confirmed before implant removal. The absence of reliable and standardised criteria for assessing fracture union poses a challenge when planning implant removal.

Clinical methods to evaluate fracture union are well established. The most commonly used clinical parameters are no tenderness or pain at the site of the fracture and being able to bear weight without pain/tenderness.²

Orthopaedic surgeons are often unable to reach a consensus on fracture healing when using conventional radiographic criteria. An international survey of 400 orthopaedic surgeons conducted by Djikman et al. revealed that 39.7% to 45.8% of surgeons used radiological criteria like size of the callus, continuity of the cortex, and gradual disappearance of the fracture line.³

The Radiological Union Score of Tibia (RUST) Score was proposed by Whelan et al., in 2010 to address the problem of the lack of a gold standard to radiologically assess fracture union.⁴ It evaluates whether a fracture line and bridging callus are present on the anterior, posterior, medial, and lateral cortices. Many studies have reported the interobserver and intraobserver variability of the RUST score when it was applied by orthopaedic surgeons.^5–7 But very few studies have assessed the variability when the radiographs were assessed by radiologists and non-specialist doctors.

The primary objective of this study was to assess the interobserver and intraobserver variation of the RUST score when applied by orthopaedic surgeons, radiologists, and interns for tibial diaphysis fractures fixed with IM interlocking nailing.

2 Materials and methods

A retrospective study was conducted using 115 radiographs of tibial diaphysis fractures treated between January 2021 and July 2022 with intramedullary interlocking nails. A radiology resident who was not an observer in the study compiled the radiographs from the hospital archives. Eight observers (two orthopaedic surgeons, two radiologists, and four interns) independently assessed each radiograph with the RUST score. The interns who were a part of this study had completed a 1-month rotation in orthopaedics. The images were randomised using a script written in Python programming language.⁸ Each image was allotted a unique three-letter code. The software randomised the images for every round of observation. The radiographs were converted from DICOM to PNG format and viewed on a high-definition (1080p) screen. Clinical data related to the radiographs such as duration since surgery and medical history were not disclosed to the observers. After observing the AP and lateral view simultaneously, the observers entered the cortex-wise score in a worksheet. The interobserver and intraobserver reliability of the RUST scores was measured using intraclass correlation coefficients (ICC) with 95% confidence intervals (CI). The scores of each observer in the first assessment were used to calculate interobserver reliability. To calculate intraobserver reliability, each reviewer was instructed to reassess the radiographs after a period of 4 weeks following the original evaluation.

Sample Size Determination: To accurately evaluate the agreement between observers, a minimum of 50 samples is needed with a level of confidence of 0.05 and anticipated ICC in the range of 0.6 to 0.7.⁹ By this standard, a sample size of 115 is adequate for this study.

Inclusion Criteria: Radiographs of male and female patients above the age of 18 years with tibial diaphysis fractures that were managed by IM nails were included. Both open and closed fractures were included. Radiographs of patients treated by both closed and open reduction with or without reaming were a part of the study.

Exclusion Criteria: Immediate post-operative radiographs with skin staples were excluded to prevent bias. Tibial diaphysis fractures treated by plating or by a combination of plating and interlocking nails were excluded from the study. Additionally, radiographs of skeletally immature patients, pathological fractures, and patients who presented with infection were also not taken into consideration.

RUST Score: The Radiological Union Score for Tibia⁴ provides a score for each cortex based on the extent of fracture union. The AP view shows the lateral cortex and medial cortex while the lateral view shows the anterior cortex and posterior cortex. The lowest score for each set of AP and L radiographs is 4, whereas the highest possible score is 12. A score of 1 is given to a cortex if there is no callus and a fracture line is visible; 2 if there is a bridging callus and fracture line; and 3 if there is a callus and no visible fracture line (Fig. 1).⁴ The criteria have been summarised in Table 1.

Close

Fig. 1 Model radiographs to demonstrate the application of the Radiological Union Score for Tibia (RUST). A) No callus is noted and a fracture line is visible in each cortex. Hence each cortex gets a score of 1. B) Callus formation is noted but the fracture line is still visible. Hence each cortex gets a score of 2. C) Callus formation is noted and a fracture line is not visible. Hence each cortex gets a score of 3. (Note: It is possible for every cortex to have a different score depending on the presence of callus and visibility of fracture line.)

Export to PPT

Statistical analysis: The scores of each observer were entered into a Microsoft Excel spreadsheet. ICC values and 95% confidence intervals were calculated using IBM SPSS Statistics for Windows, version 25 (IBM Corp., Armonk, NY, USA). A mean-rating, absolute-agreement, 2-way mixed-effects model was used.¹⁰ The interpretation¹⁰ of ICC values has been summarised in Table 2.

The research was conducted in adherence to the principles outlined in the Declaration of Helsinki. The Institutional Ethics Committee approved the study protocol.

3 Results

The RUST score of the 115 radiographs varied from 4 to 12. The mean score in the first assessment was 8.47 ± 2.75 (median score 8). The mean score was 8.68 ± 2.65 (median 8.5) in the second assessment. The reliability between all the observers was excellent (ICC, 0.95. 95% CI, 0.93–0.96). The ICC among various groups of observers was as follows: orthopaedic surgeons (0.87), radiologists (0.89), and interns (0.91). The ICC of orthopaedic surgeons and radiologists is comparable, while the ICC of interns is slightly higher. The ICC values of all the observers have been summarised in Table 2.

The intraobserver variability among various groups was as follows: orthopaedic surgeons (0.94), radiologists (0.91), interns (0.87), and overall (0.81). Intraobserver variability among individual observers (Table 3) of the same group (i.e. orthopaedic surgeons or radiologists) was minimal. However, there was a significant amount of variation among interns, with the intraobserver ICC ranging from 0.71 to 0.96.

4 Discussion

This study assessed the interobserver and intraobserver variation of the RUST score when applied by orthopaedic surgeons, radiologists, and interns for tibial diaphysis fractures treated by IM nailing.

Fracture union has been defined using a variety of scales, classifications, and radiological criteria.^3,11 Panjabi et al.¹² correlated the radiographic appearance of transversely osteotomised rabbit tibiae with the mechanical strength of fractures that were in the process of union. They reported that continuity in the cortex was the most accurate indicator of the strength of a fracture that was healing. The least significant indicator was callus area. Another study compared fracture stiffness measurements with radiographic fracture healing assessment.¹³ It suggested that cortical callus bridging assessment was a much better method to evaluate the union of a fracture when compared to the "general appearance of healing."¹³ The Hammer's index scale¹⁴ is based on whether a fracture line is present or absent and on the maturity of the bone callus. Even though there are several subjective and objective ways to evaluate fracture healing in the literature, a trustworthy and gold-standard method has not yet been devised.

The interobserver and intraobserver reliability of the Hammer score was evaluated by Wheelan et al. (2002).¹⁵ Their observations led to the development of a more accurate, simpler, and dependable radiological grading system, the RUST score.⁴

Whelan et al. proposed that the RUST score will have improved consistency compared to various methods of assessing fracture union.⁴ The advantage of the RUST score is that each cortex contributes to the final score therefore making it more reliable. This scale has strong interobserver and intraobserver agreement and is simple to use.^4–7,16

A prospective study conducted by Kooistra et al. comprising 549 pairs of tibia x-rays reported that the interobserver reliability of the RUST score was good (ICC 0.84, 95% CI 0.80–0.87).¹⁶ In the present study, interobserver reliability was excellent (ICC 0.95, 95% CI, 0.93–0.96).

Ali et al.¹⁷ compared the reproducibility between orthopaedic surgeons and radiologists. According to their findings, the RUST score yields consistent results whether assessed by either an orthopaedic surgeon or a radiologist. A similar observation was noted in the present study (ICC 0.87 vs 0.89). This study also evaluated the validity of the RUST score when interns implemented it. The ICC of interns (0.91) was comparable to that of orthopaedic surgeons (0.87) and radiologists (0.89). However, there was a wide disparity among the interns in the values of intraobserver ICC, which varied from 0.71 to 0.96.

Azevedo Filho et al. reported an ICC of 0.87 (95% CI: 0.81–0.91). They reported that as the evaluator's experience grew, so did the reliability: greater reliability among traumatologists compared to 1st, 2nd, and 3rd-year residents (ICC 0.94, 0.80, 0.92, and 0.90 respectively).⁶ Other studies in the literature have not tested the reliability of the RUST score when it is applied by non-specialist doctors such as interns and general practitioners. In the present study, the overall interobserver reliability of interns was marginally higher when compared to orthopaedic surgeons and radiologists. However, the wide disparity in intraobserver reliability among the interns suggests that less experienced observers are more likely to misinterpret cortical bridging and the presence of a fracture line when applying the RUST score. The reliability can be improved by comparison with radiographs taken after surgery.¹⁸

The RUST score was originally devised for tibial fractures. However, it may also be applied to other long bone fractures that are treated by intramedullary nails, such as the femur, humerus, radius, and ulna. However, the mechanics of fracture union are markedly different in these bones. The RUST score may also be applied in fractures treated by plating or an external fixator, as the cortices are discernible on radiographs. More studies are warranted in this domain to standardise the application of the RUST score for other long bones and treatment modalities.

The score may vary depending on the stage of fracture union. A one-time application of this score might not make any major difference while assessing patient outcomes. Application of the RUST score at regular intervals during follow-up will provide a thorough progression of fracture healing over a period.

A notable limitation of the present study is the inclusion of only a small pool of observers. A study involving a larger and more diverse set of observers with varying levels of experience will provide more substantial information about the consistency of the score.

Conclusion: The RUST score is a dependable criterion for evaluating tibial fracture healing with excellent interobserver and intraobserver reliability. It also supports the idea that there is almost perfect agreement among orthopaedic surgeons and radiologists.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Consent form

Not applicable.

CRediT authorship contribution statement

Sudhir Shankar Mane: Conceptualization, Methodology, Writing – original draft, Writing – review & editing, Data curation, Formal analysis. Sindhura Yamajala: Conceptualization, Writing – original draft, Writing – review & editing, Data curation. Madhavi Latha Vinjamuri N S: Conceptualization, Writing – review & editing. Sree Ramya Prathyusha Mitnala: Methodology, Writing – review & editing.

Declaration of generative AI and AI-assisted technologies in the writing process

During the preparation of this work, the authors did not use any generative AI and AI-assisted technologies in the writing process.