- Research
- Open access
- Published:
Crosswalk between the PROMIS physical function CAT and PROMIS upper extremity CAT v1.2 in a hand surgery population
Journal of Patient-Reported Outcomes volume 8, Article number: 53 (2024)
Abstract
Background
There is no gold standard patient-reported outcome measure (PROM) in hand surgery. As a result, a diverse array of PROM instruments have been utilized across centers over time. Lack of score interchangeability limits the ability to compare or conglomerate scores when new instruments are introduced. Our aim was to develop a linkage for the PROMIS UE CAT v1.2 and PROMIS PF CAT scores and develop crosswalk tables for interconversion between these PROMs.
Methods
Retrospective review was conducted to identify adult (≥ 18y) patients seen by orthopaedic hand surgeons at a single academic tertiary care hospital who had completed PROMIS UE CAT v1.2 and PROMIS PF CAT score at the same visit. For those with multiple visits, only one randomly selected visit was included in the analyses. Pearson’s correlation was calculated to determine the linear relationship between the scores. Linkage from PF to UE was performed utilizing several commonly utilized equating models (identity, mean, linear, equipercentile and circle-arc methods). The performance of the models was assessed using intraclass correlation (ICC) between observed PROMIS UE CAT v1.2 and estimated PROMIS UE CAT v1.2 scores generated using the model as well as Root Mean Square Error (RMSE). The model chosen as the ‘best’ was further assessed for population invariance using root expected mean squared difference (REMSD) where < 0.08 were considered good.
Results
Of 10,081 included patients, mean age was 48.3 (SD = 17.0), and 54% were female (5,477/10,081). Mean UE CAT v1.2 and PF CAT scores were 37 (SD = 9.8) and 46 (SD = 10.0), respectively. There was a strong correlation between the scores (Pearson correlation r = 0.70). All methods performed acceptably (ICC ≥ 0.66 and RMSE < = 7.52 for all). The equipercentile method had the highest ICC (ICC = 0.70 (95% CI 0.69–0.71)) while the mean and circle arc methods had the lowest RMSE. The circle arc method is the most reliable with the smallest standard error and has satisfactory population invariance across age group (REMSD 0.065) and sex (REMSD 0.036).
Conclusions
Crosswalk tables to be used for bidirectional conversion between scores were created.
Level of evidence
: III.
Background
Patient-reported outcome measures (PROMs) are pre- and post-operative measurements of patient-reported functionality, pain, and quality of life that are utilized in research and clinical practice [1].
In the era of value-based care there is growing emphasis on the collection and reporting of PROMs to facilitate the evaluation of patient conditions and treatment efficacy [2,3,4,5]. PROMs provide surgeons with validated measures through which to assess clinical improvement and to compare outcomes with other patient populations or between surgeons. The Patient-Reported Outcomes Measurement Information System (PROMIS) was created to standardize the process of administering and interpreting PROMs [6] and utilizes Computer Adaptive Tests (CAT), which minimizes responder burden [7]. PROMIS metrics are designed with a mean score of 50 and standard deviation (SD) of 10 for the reference population, allowing for easy interpretation [6].
A number of established PROMs are utilized in the field of hand and upper extremity surgery, including PROMIS Physical Function (PROMIS PF) CAT and the PROMIS Upper Extremity (PROMIS UE) CAT, of which the latter has multiple versions—mostly recently version 1.2 and 2.0 [6]. Both PROMIS PF CAT and PROMIS UE CAT have been utilized in the hand and upper extremity population [8,9,10,11,12,13,14,15]. PROMIS UE CAT v1.2 and v2.0 are not interchangeable, but are for PROMIS PF CAT [6, 10]. PROMIS PF CAT provides an overview of overall patient function level without specifically targeting the upper extremity, however, this score has demonstrated validity and responsiveness within hand and upper extremity populations and is ubiquitous throughout literature [16,17,18,19]. Much like other PROM instruments, PROMIS PF CAT and PROMIS UE CAT have limitations worth mentioning, including the fact that younger individuals report higher normative PROMIS PF and PROMIS UE scores—indicating greater function—than older individuals [20]. Additionally, there are known concerns regarding floor and ceiling effect for both instruments [6]. A clear consensus on which PROM application and clinical assessment with these measures has not emerged [4, 9, 21,22,23,24]. The variety of existing PROMs, lack of universal gold standard, and institutional differences in PROM utilization has led to challenges in directly comparing different groups and research.
Linking represents a mathematical method used to connect two correlated PROM scores, allowing the creation of a common metric. For scores that can be linked, crosswalk tables can be generated to facilitate interconversion. The linking of different PROMs allows for comparison between groups and increases statistical power. It also allows for the inclusion of studies using different PROMs in meta-analyses. Prior studies have shown correlations between the PROMIS PF CAT and PROMIS UE CAT [8, 9], although we are unaware of prior literature that has established a linkage between these two measures. Current literature in our field has emphasized using CAT instruments and it is unclear if short form (SF) CAT scores are interchangeable or not. The study objective was to develop a linkage model between the PROMIS PF CAT score and the PROMIS UE CAT v1.2 score, enabling crosswalks between these frequently utilized PROMs.
Materials and methods
A retrospective review study approved by the Institutional Review Board was performed including patients from February 2014 to August 2017 who were treated by one of five fellowship-trained orthopaedic hand surgeons at a single academic tertiary care hospital. Patients were included if they had concurrently completed PROMIS UE CAT v1.2 and PROMIS PF CAT instruments at the same clinical encounter. Patients were excluded if they were < 18-year-old or if they were seen for a lower extremity or shoulder primary complaint. PROM instruments were completed by patients in clinic or in the preoperative holding area electronically on a tablet computer. One randomly selected visit was included in the analysis for patients who had multiple visits with concurrent PROMIS PF and PROMIS UE scores.
Total score distribution for both the instruments was summarized descriptively. The linear relationship between the scores was assessed using Pearson correlation. A score between r = 0.60–0.79 was considered a “strong relationship” [25, 26] and a score r > 0.80 was considered a “very strong relationship.” Linkage between the PROMIS PF CAT and PROMIS UE CAT v1.2 was accomplished using the R package equate [27]. Multiple methods were used to develop linkages: identity, mean, linear, equipercentile equating (EE) and circle-arc methods [28, 29]. Standard error and root-mean square errors (RMSE) for the linking functions were assessed using bootstrapping methods, with the total cohort serving as the reference for defining the mean observed difference between observed and estimated scores in these groups. Intraclass correlation coefficients (ICC) was used to assess linkage model performance. The performance and population invariance of the linkage models was further assessed in a subgroup analysis evaluating sex and age (< 60 or ≥ 60-year-old).
Results
A total of 11,508 patients were identified who were seen for non-shoulder UE complaint with concurrent PROMIS PF CAT and PROMIS UE CAT v1.2 scores. Minor patients (n = 1,041), those with PROMIS PF score = 0 (n = 12), those with PROMIS UE score = 0 (n = 4), and patients not seen by hand surgeons (n = 370) were excluded. Therefore, the final analysis included 10,081 patients (Fig. 1). The mean age was 48 years (SD = 17), 54% were female (5,477/10,081), and 85% were White (8,564/10,081) (Table 1). The mean PROMIS PF CAT score was 46 (SD = 10), and the mean PROMIS UE CAT v1.2 score was 37 (SD = 9.8) (Table 2). A strong positive linear relationship between PROMIS PF CAT and PROMIS UE CAT v1.2 was observed (Pearson’s correlation coefficient r = 0.70) (Fig. 2).
The various equations for the PROMIS PF CAT and PROMIS UE CAT v1.2 linkage models are displayed in Table 3. All methods performed acceptably (ICC ≥ 0.66 and RMSE < = 7.52 for all) (Fig. 3). The equipercentile method had the highest ICC (ICC = 0.70 (95% CI 0.69–0.71)) while the mean and circle arc methods had the lowest RMSE. The circle arc method is the most reliable with the smallest standard error and has satisfactory population invariance across age group (REMSD 0.065) and sex (REMSD 0.036).
A crosswalk table linking PROMIS PF CAT to PROMIS UE CAT v1.2 for all linkage models allowing for bidirectional conversion between scores is included in Appendix 1. Table 4 details the application of equipercentile equating model to age and sex subpopulations. ICC, SE, and RMSE were all slightly higher in the subgroups, particularly in the age ≥ 60 population (EE model, RMSE 7.46), (Table 4). However, subgroup analysis by age or sex did not demonstrate significant alteration in equating model performance or introduction of population invariance that would require the production of crosswalk tables dichotomized by age or sex.
Discussion
PROMs are a tool for assessing the results of hand and upper extremity surgery [30] and remain relevant in the clinical setting by capturing the patient’s perception of their own health status, including physical function, social function, quality of life, and other health domains [2, 31]. Given that the proposed use of these instruments includes shared decision-making, quality assurance, post-operative care, the development and interpretation of clinically meaningful research, and value-based care delivery [32], improving our understanding and ability to translate between PROMs is critical for continued research. Our main study finding is that the linking of PROMIS PF CAT and PROMIS UE CAT v1.2 through equating models had acceptable performance. We recommend the use of the equipercentile equating model, due to acceptable performance of the model and frequent utilization of this method in the crosswalk literature [33,34,35,36,37]. This allows for the derivation of bidirectional crosswalk tables that allow for interconversion between PROMIS PF CAT and PROMIS UE CAT v1.2 for future research in the hand and upper extremity population. This improves our ability to combine, synthesize, and understand PROMs in the literature, as well as in meta-analyses including multiple studies with differing methodologies.
The PROMIS UE CAT v1.2 has been administered in many important settings for a range of upper extremity issues, including chronic disease, long-term disability, and acute conditions [11]. Hung et al. documented high correlation between PROMIS UE CAT v1.2 and PROMIS PF CAT, Anxiety CAT, and Pain Interference CAT [11]. They also confirmed that the PROMIS UE CAT v1.2 question bank items represent an independent domain from overall PF. The PROMIS UE CAT will continue to be used in clinical outcomes research and has a role in specifically assessing upper extremity function, noting that the most current version is v2.0 [6]. However, as discussed by Hung et al., the PROMIS UE CAT v1.2 has a ceiling effect that limits its ability to assess patients with higher levels of upper extremity function—seen as a limited number of questions targeted toward those with higher function—that results in an impaired ability to discriminate high-end functioning of the upper extremity [11]. The presence of a ceiling effect is not limited to the PROMIS UE CAT v1.2 and has been documented in the Disabilities of the Arm, Shoulder and Hand questionnaire and Short Musculoskeletal Functional Assessment [17, 38]. Each PROM has its limitations that need to be understood for appropriate instrument application.
It is important to note that the impact of subpopulations (sex and age) on the linkage models was found to be minimal. Across all evaluated groups (male versus female; age < 60 versus age ≥ 60), there were no substantial increased levels of population invariance. These findings suggest a lack of age and sex-specific differences in patient response to these instruments and, as a result, additional crosswalk tables accounting for these subgroup differences were not indicated. This is important, as prior research has documented that younger individuals can have higher normative PROMIS PF and PROMIS UE scores—indicating greater function—than older individuals [20]. As a result, the universal reference of a score of 50 may not apply to certain normative subpopulations [20]. However, given our extensive sample size (> 10,000 patients/>19,000 observations) and broad inclusion (both trauma and elective hand and upper extremity conditions), we contend that our findings hold relevance to populations responsive to the PROMIS PF CAT and PROMIS UE CAT v1.2 measures. Given our findings, the crosswalk tables presented are sufficient for all adult groups (excluding minors) and both sexes.
This study has limitations that warrant mention. The data was collected prospectively but reviewed in a retrospective nature, and not purposely collected for the linking the PROMIS PF CAT to the PROMIS UE CAT v1.2. Retrospective data analysis can introduce confounders, and we did not assess the difference between PROM responders and non-responders. However, the robust PROMs completion rate that our collection workflow yields (> 90%) may reduce this concern [39]. The cohort studied originates from a single institution and comprises predominantly White individuals (85%), which exceeds the general U.S. population by 9.4% [40]. This limits the generalizability of the findings to other populations based on both race/ethnicity and the types of hand and upper extremity pathology seen at the institution. Finally, these results may not apply to shoulder conditions.
Conclusions
In conclusion, all models equating the PROMIS PF CAT and PROMIS UE CAT v1.2 performed acceptably, although we recommend the use of the equipercentile equating model moving forward. These results allow for crosswalking between PROMIS PF CAT and PROMIS UE CAT v1.2 instruments for adult non-shoulder hand and upper extremity patients with both elective and traumatic diagnoses. Additional crosswalk tables accounting for sex and age were not needed given lack of appreciable population invariance based on these demographic factors. These crosswalk tables have potential applications for development of multicenter clinical databases and broader inclusion in meta-analysis studies, where contributing centers or researchers employ varying PROMs.
Data availability
The crosswalk tables are available in Appendices I & II submitted with this manuscript. The datasets generated and/or analyzed during the current study are not publicly available due to it being a collected component of patient clinical care but are available from the corresponding author on reasonable request.
References
Brook EM, Glerum KM, Higgins LD, Matzkin EG (2017) Implementing patient-reported outcome measures in your practice: pearls and pitfalls. Am J Orthop (Belle Mead NJ) 46(6):273–278
Chung KC, Burns PB, Davis Sears E (2006) Outcomes Research in hand surgery: where have we been and where should we go? J Hand Surg 31(8):1373–1379
Relman AS (1988) Assessment and accountability. N Engl J Med 319(18):1220–1222
Shapiro LM, Ring D, Akelman E et al (2021) How should we use patient-reported outcome measures at the Point of Care in hand surgery? J Hand Surg 46(12):1049–1056
Makhni EC, Baumhauer JF, Ayers D, Bozic KJ (2019) Patient-reported outcome measures: how and why they are collected. Instr Course Lect 68:675–680
Tyser AR, Hung M, Bounsanga J, Voss MW, Kazmers NH (2019) Evaluation of Version 2.0 of the PROMIS Upper Extremity Computer Adaptive Test in Nonshoulder Upper Extremity patients. J Hand Surg Am 44(4):267–273
Cella D, Yount S, Rothrock N et al (2007) The patient-reported outcomes Measurement Information System (PROMIS). Med Care 45(5):S3–S11
St John MJ, Mitten D, Hammert WC (2017) Efficacy of PROMIS Pain Interference and Likert Pain scores to assess physical function. J Hand Surg Am 42(9):705–710
Beleckas CM, Padovano A, Guattery J, Chamberlain AM, Keener JD, Calfee RP (2017) Performance of patient-reported outcomes Measurement Information System (PROMIS) Upper Extremity (UE) Versus physical function (PF) computer adaptive tests (CATs) in Upper Extremity Clinics. J Hand Surg Am 42(11):867–874
Kazmers NH, Hung M, Rane AA, Bounsanga J, Weng C, Tyser AR (2017) Association of physical function, anxiety, and Pain Interference in Nonshoulder Upper Extremity patients using the PROMIS platform. J Hand Surg Am 42(10):781–787
Hung M, Voss MW, Bounsanga J, Crum AB, Tyser AR (2017) Examination of the PROMIS upper extremity item bank. J Hand Ther 30(4):485–490
Anthony CA, Glass NA, Hancock K, Bollier M, Wolf BR, Hettrich CM (2017) Performance of PROMIS instruments in patients with shoulder instability. Am J Sports Med 45(2):449–453
Beckmann JT, Hung M, Bounsanga J, Wylie JD, Granger EK, Tashjian RZ (2015) Psychometric evaluation of the PROMIS physical function computerized adaptive test in comparison to the American Shoulder and Elbow surgeons score and simple shoulder test in patients with rotator cuff disease. J Shoulder Elb Surg 24(12):1961–1967
Overbeek CL, Nota SPFT, Jayakumar P, Hageman MG, Ring D (2015) The PROMIS physical function correlates with the QuickDASH in patients with Upper Extremity illness. Clin Orthop Relat Res 473(1):311–317
Doring AC, Nota SP, Hageman MG, Ring DC (2014) Measurement of upper extremity disability using the patient-reported outcomes Measurement Information System. J Hand Surg Am 39(6):1160–1165
Hung M, Saltzman CL, Greene T et al (2017) The responsiveness of the PROMIS instruments and the qDASH in an upper extremity population. J Patient-Reported Outcomes 1(1)
Tyser AR, Beckmann J, Franklin JD et al (2014) Evaluation of the PROMIS Physical Function Computer Adaptive Test in the Upper Extremity. J Hand Surg 39(10):2047–2051.e2044
Beleckas CM, Gerull W, Wright M, Guattery J, Calfee RP (2019) Variability of PROMIS scores across Hand conditions. J Hand Surg Am 44(3):186–191.e181
Fries JF, Krishnan E, Rose M, Lingala B, Bruce B (2011) Improved responsiveness and reduced sample size requirements of PROMIS physical function scales with item response theory. Arthritis Res Ther 13(5):R147
Yedulla NR, Wilmouth CT, Franovic S, Hazime AA, Hudson JT, Day CS (2021) Establishing age-calibrated normative PROMIS scores for Hand and Upper Extremity Clinic. Plast Reconstr Surg Glob Open 9(8):e3768
Brodke DJ, Zhang C, Shaw JD, Cizik AM, Saltzman CL, Brodke DS (2022) How do PROMIS scores correspond to common physical abilities? Clin Orthop Relat Res 480(5):996–1007
Bernstein DN, Houck JR, Hammert WC (2019) A comparison of PROMIS UE Versus PF: correlation to PROMIS PI and Depression, Ceiling and Floor effects, and Time to Completion. J Hand Surg 44(10):901.e901–901.e907
Lloyd-Hughes H, Geoghegan L, Rodrigues J et al (2019) Systematic review of the Use of Patient reported outcome measures in studies of electively-managed hand conditions. J Hand Surg (Asian-Pacific Volume) 24(03):329–341
Kaat AJ, Buckenmaier CT, Cook KF et al (2019) The expansion and validation of a new upper extremity item bank for the patient-reported outcomes Measurement Information System® (PROMIS). J Patient-Reported Outcomes 3(1):69
Koo TK, Li MY (2016) A Guideline of selecting and reporting Intraclass correlation coefficients for Reliability Research. J Chiropr Med 15(2):155–163
Portney L (2009) MW. Foundations of Clinical Research: applications to practice, 3rd edn. Prentice, Upper Saddle River, NJ
Albano ADequate (2016) Appl Psychol Meas 40(5):361–362
Schalet BD, Lim S, Cella D, Choi SW (2021) Linking scores with patient-reported Health Outcome instruments:a VALIDATION STUDY AND COMPARISON OF THREE LINKING METHODS. Psychometrika 86(3):717–746
Cook LLED (1991) IRT equating methods. Instructional Top Educational Meas 37–45
Baumhauer JF (2017) Patient-reported outcomes - are they living up to their potential? N Engl J Med 377(1):6–9
Alderman AK, Chung KC (2008) Measuring outcomes in hand surgery. Clin Plast Surg 35(2):239–250
Makhni EC (2021) Meaningful clinical applications of patient-reported outcome measures in Orthopaedics. J Bone Joint Surg Am 103(1):84–91
Polascik BA, Hidaka C, Thompson MC et al (2020) Crosswalks between knee and hip arthroplasty short forms: HOOS/KOOS JR and Oxford. J Bone Joint Surg Am 102(11):983–990
Soh SE, Harris IA, Cashman K, Graves SE, Ackerman IN (2022) Crosswalks between the Oxford hip and knee scores and the HOOS-12 and KOOS-12 instruments. Osteoarthritis Cartilage 30(4):570–577
Johnson JL, Boulton AJ, Spindler KP et al (2022) Creating crosswalks for knee outcomes after ACL Reconstruction between the KOOS and the IKDC-SKF. J Bone Joint Surg Am 104(8):723–731
Heng M, Tang X, Schalet BD et al (2021) Can the knee outcome and osteoarthritis score (KOOS) function subscale be linked to the PROMIS physical function to Crosswalk Equivalent scores? Clin Orthop Relat Res 479(12):2653–2664
O’Connor DP (2017) CORR insights((R)): a crosswalk between UCLA and Lower Extremity Activity scales. Clin Orthop Relat Res 475(2):549–551
Morgan JH, Kallen MA, Okike K, Lee OC, Vrahas MS (2015) PROMIS physical function computer adaptive test compared with other Upper Extremity Outcome measures in the evaluation of proximal Humerus fractures in patients older than 60 years. J Orthop Trauma 29(6):257–263
Kazmers NH, Stephens AR, Tyser AR (2019) Effects of Baseline Opioid Medication Use on patient-reported functional and psychological impairment among Hand Clinic patients. J Hand Surg Am 44(10):829–839
U.S. Census Bureau Quick Facts (2022) https://www.census.gov/quickfacts/fact/table/US/PST045221. Published 2021. Updated July 1, 2021. Accessed June 17
Acknowledgements
Not applicable.
Funding
This investigation was supported by the University of Utah Population Health Research (PHR) Foundation, with funding in part from the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1TR002538 (formerly 5UL1TR001067-05, 8UL1TR000105 and UL1RR025764).
Author information
Authors and Affiliations
Contributions
MJR and DRJ performed the study coordination, results interpretation, and writing of the manuscript. CZ and APP participated in reviewing the data and the statistical work. They also contributing to the manuscript writing. AC contributed to study design and manuscript writing. NHK contributed to study concept and design, organization, project management, and manuscript writing. All authors read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Rogers, M.J., Daryoush, J.R., Zhang, C. et al. Crosswalk between the PROMIS physical function CAT and PROMIS upper extremity CAT v1.2 in a hand surgery population. J Patient Rep Outcomes 8, 53 (2024). https://doi.org/10.1186/s41687-024-00736-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1186/s41687-024-00736-6