- Open Access
The use of patient-reported outcome measures in hip and knee arthroplasty in Alberta
Journal of Patient-Reported Outcomes volume 5, Article number: 87 (2021)
PROMs are part of routine measurement for hip and knee replacement in Alberta, Canada. We provide an overview of how PROMs are implemented in routine care, and how we use PROMs data for decision-making at different levels within the health system. The Alberta Bone and Joint Health Institute (ABJHI) ran a randomized controlled trial to determine the effectiveness and cost-effectiveness of an evidence-based care pathway for hip and knee arthroplasty in 2004. The study included several PROMs questionnaires: Western Ontario and McMaster Universities Osteoarthritis Index, Health Utility Index, Short Form 36 and the EQ-5D-3L. Subsequently, the focus shifted to spread and scale of the care pathway provincially. WOMAC and EQ-5D-3L and a patient experience survey were selected for provincial adoption – captured before surgery, three-months post-surgery, and 12-months post-surgery. These PROMs data were integrated into research and routine clinical practice at the micro, meso and macro levels. At the micro level, PROMs data are used at the individual patient and provider level for patients to provide input on their care and as a tool to communicate with their healthcare providers. We examined the relationship of appropriateness and patient reported outcomes in a prospective cohort study. We evaluated whether routinely collected PROMs could be integrated into a patient decision aid to better inform shared decision making. At the meso level, continuous quality improvement reports are provided routinely to individual health care providers, hospitals and clinics on their performance against the measurement framework and standard key performance indicators. At the macro level, PROMs data are used to evaluate system performance by comparing outcomes across different jurisdictions or over time and support health policy decision making. Combined with administrative databases, we have used simulation models to reflect transition through the continuum of care from disease onset through end-stage care regarding the burden of disease, healthcare resource requirements and associated healthcare costs. The addition of PROMs data in clinical repositories and analyses enables the system to identify and address issues of continuous quality improvement against a measurement framework of performance indicators and to explicitly recognize the trade-offs that are inherent in any resource-constrained system.
In the last two decades, the number of joint replacements grew rapidly around the world . Patient-reported outcomes following joint replacement have been considered an important indicator of evaluating the outcome of these procedures, with the increasing orientation towards patient-centred care. Patient-reported outcome measures (PROMs) have been routinely applied in hip and knee replacement programs in the United Kingdom and Sweden [2, 3]. The Organization for Economic Co-operation and Development (OECD) is also piloting its Patient-Reported Indicator Surveys initiative in the field of hip and knee replacement to promote a consistent approach across OECD countries to collect PROMs data .
Likewise, implementation of PROMs has also become part of the routine care for hip and knee replacement in Alberta, Canada. This paper first introduces briefly how PROMs are implemented in routine care, how we use PROMs data to make decisions at different levels within the health system, and the associated challenges with implementing PROMs with examples of what have we learned from implementation efforts.
PROMs implementation (Table 1)
The application of PROMs in hip and knee arthroplasty in Alberta began in 2004, when the Alberta Bone and Joint Health Institute (ABJHI) ran a randomized controlled trial sponsored by Alberta Health and Wellness, the Alberta Orthopedic Society and three regional health authorities (Calgary Health Region, Capital Health, and David Thompson Health Region) . The two-year study determined the effectiveness and cost-effectiveness of a novel, evidence-based care pathway for hip and knee replacement . Central to that effort was the measurement of several PROMs to capture health status and health utility. The study included several PROMs questionnaires including the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC) , Health Utility Index Mark 3 (HUI3) , Short Form 36 (SF-36)  and the EQ-5D-3L .
Following the successful conclusion of the study, the focus shifted to spread the pathway across the rest of the province and to scale up data capture to all patients seeking care. With the transition to routine care, the implementation team selected the WOMAC and EQ-5D-3L along with a non-validated patient experience survey for provincial adoption. Among the set of instruments captured during the study, these PROMs were deemed the most clinically and operationally meaningful, the least redundant, and the easiest to implement at a large scale. The WOMAC and EQ-5D were captured on paper before surgery, three months post-surgery, and 12-months post-surgery, and scanned using optical character recognition (OCR) technology into a database for statistical analysis and reporting.
Summary of the up-to-date PROMs data repository
Following completion of the spread and scale phase in 2011, ABJHI introduced automated reporting and feedback systems, electronic capture of PROMs, and transitioned from the EQ-5D-3L to the EQ-5D-5L . The paper-based capture of PROMs proved to be time-consuming, expensive and prone to errors. As the volume of patients and participating clinics increased, the corresponding volume of paper soon exceeded 30,000 pages per year, costing about $20,000 in labour. OCR errors on the patient labels confounded patient identification, which required significant effort to correct. Furthermore, the process of collecting, collating, transporting, scanning, validating, and integrating PROMs data from paper into the database often resulted in delays of up to six months between data capture and the ability to analyze and report results.
Electronic capture was implemented using two data capture platforms that were integrated into the clinic workflows. Some clinics had licensed and implemented the proprietary Ortech software  to record orthopedic implant use and to submit data to the Canadian Joint Replacement Registry. These clinics opted to implement PROMs capture using Ortech. For the other clinics, ABJHI provided an open-source solution using REDCap software .
The EQ-5D-5L was introduced because it was considered superior to the 3L version due to 1) the increase in possible health states from 243 to 3,125 which offered better detection in changes in health-related quality of life, 2) the publication of Canadian value sets in the 5L provided more regionally relevant scoring than was available in the 3L, and 3) the 5L score was believed to be less sensitive to a ceiling effect when measuring the impact of arthritis on quality of life. We used PROMs data collected in this repository to compare the cross-sectional and longitudinal validity of the EQ-5D-3L and EQ-5D-5L. The results demonstrated that the EQ-5D-5L has better construct validity and responsiveness (tested against the WOMAC) than the 3L version. These findings also supported our decision to replace the EQ-5D-3L with the EQ-5D-5L as a PROM for routine data collection [14, 15].
Application of PROMS in routine clinical practice and health care services
In this section, we highlight some novel applications of these PROMs data in research programmes and integrated into routine clinical practice settings to deliver healthcare services and to assess patient outcomes and performance of the system at the micro, meso and macro levels in the healthcare system in Alberta .
At the micro level, PROMs data can be used at the individual patient and provider level for patients to provide input on their care and as a tool to communicate with their healthcare providers. An example is the development of a mobile health app for patients with knee osteoarthritis to guide self-management, provide evidence-based information to patients and physicians and facilitate communication that address patient needs and challenges in disease management. The co-design process with patients, family physicians and researchers, identified a visual symptom graph using PROMs to be amongst the highest priority for functional requirements of the app .
In another application of PROMs measures, we have examined the relationship of appropriateness and patient reported outcomes at one-year post-surgery in a prospective cohort study implemented at the two highest volume centres in Alberta which use a centralised intake process for surgical referrals. With the continued increase in rates of arthroplasty surgery, particularly in patients at younger ages and with higher levels of functioning and lower levels of pain, there are concerns about the appropriate selection of candidates for surgery [18, 19]. A key factor for appropriateness suggests that patients should have realistic expectations for total knee arthroplasty (TKA) outcomes. We found that the pre-operative expectations of patients regarding kneeling and psychological wellbeing were significantly associated with the patient-reported level of TKA satisfaction at one year post-surgery .
Building on this appropriateness research, another related application of PROMs at the micro level is the role of PROMs to inform setting realistic expectations for patients, and promoting shared decision-making with their care provider . We evaluated whether routinely collected pre- and post-TKA PROMs could be integrated into a patient decision aid to better inform these appropriateness criteria in a randomized controlled trial with a primary outcome measure of decision quality . We found that decision quality improved for patients who completed the decision aid. [Under review].
At the meso level, healthcare providers can use aggregate PROMs data to identify certain outcomes or symptoms in groups of patients or for a clinic site, and compare these data across providers, clinics, or hospital sites. Using the ABJHI Health Data Repository, in the context of the clinical care pathway and a measurement framework with key performance indicators (waiting time, length of surgery, post-surgery length of hospital stay, serious complications, readmissions, a range of PROMs, and patient satisfaction), continuous quality improvement reports are provided routinely to individual health care providers, public health agencies, hospitals and clinics on their performance regarding hip and knee arthroplasty at all of the 14 hospitals that perform hip and knee arthroplasty across the province. These data are used for routine monitoring and surveillance by ABJHI as part of their continuous quality improvement program at the provincial level through Alberta Health Services and the Bone and Joint Health Strategic Clinical Network and the clinic level for all hip and knee clinics in Alberta to inform performance against benchmarks, and at the individual surgeon level to inform patient management [23,24,25]. Despite the desire to provide real-time feedback of PROMs during the clinical encounter, several technological barriers complicate the integration of web-based data capture with multiple EMR systems across multiple locations in near real-time.
At the macro level, PROMs data can be used to evaluate the performance of the healthcare system to compare outcomes across different jurisdictions or over time and support health policy decision making. Combined with administrative databases as a key source for estimating the burden and epidemiologic trends of chronic diseases such as osteoarthritis , and simulation models that reflect how patients transition through the continuum of care from disease onset through end-stage care , PROMs data can provide insight into the magnitude of the patient population and burden of disease, healthcare resource requirements and associated healthcare costs. Such tools can be used for estimating how many patients will need care, the nature of that care, and when the care will be needed. This is essential for planning healthcare resource allocation at a system level. These are powerful tools when applied to inform decision making for health care delivery that consider not only the technological aspects of effective care, but also the healthcare context – including the people and processes for the delivery of care [28, 29]. As part of their continuous quality improvement program noted above, ABJHI also reports these data at the macro provincial level through Alberta Health Services and the Bone and Joint Strategic Clinical Network [24, 25]. For instance, PROMs data are being used to develop appropriateness criteria for same-day discharge and accelerated discharge to improve patient throughput and address demand backlog . As examples of how PROMs and PREMs have had an impact, we have seen protocol changes around analgesic use (e.g. elimination of intravenous morphine) due to patient-reported post-op nausea and vomiting reported via PREMS. We have also seen changes in communication methods to address patient-reported deficiencies in coordination of care (e.g. whiteboards in room with discharge planning). Consequently, changes in education and awareness have led to reduced variability in selection of patients for surgery over time (i.e. reduced variance in pre-op WOMAC scores over time).
We have met some general challenges that other PROMs programs might also expect to experience. The transition from the EQ-5D-3L to the 5L was performed gradually between 2013 and 2016. To minimize expense when the paper copies of EQ-5D-5L questionnaires were sent to each clinic, the 3L version questionnaires had to be exhausted before the clinics were willing to deploy the 5L survey. The two versions were in mixed-use for a period, and some patients were administered different versions throughout their care journey. This introduced difficulties in data analysis and interpretation, as the 3L scores had to be up converted to 5L values.
In addition, the integration of electronic data capture (EDC) into clinic workflow remains one of the greatest challenges in PROMs administration. Though some clinics provide patients with a web URL to complete their surveys in advance of the appointment, many patients only complete their surveys at the clinic. This can introduce delays in clinic flow including patients requiring additional time to complete their surveys while in the waiting or exam room, additional staff time required to set up technology for patients to use and to address any patient queries regarding the survey questions, and gaps in data collection when patients abandon surveys before completion due to late arrival at the clinic or faster than expected wait time for the appointment. The transition from paper to EDC highlighted the implementation challenges when deploying at provincial scale. The critical success factors that eased adoption of EDC were a) the availability of staff to assist patients with the technology, b) availability of WIFI connectivity, and c) sufficient or dedicated space for the patients to use while completing their PROMs. Furthermore, patient follow-up processes vary across clinics, with some using automated reminders for patients to complete outstanding surveys, while other clinics are unwilling to rely on technology to support electronic data capture due to limited availability of technologically proficient staff or IT support. Moreover, while some small clinics have staff capacity to follow up with patients to complete their paper surveys, this would overwhelm some other clinics. Although guidelines suggest that 1 year post-operatively is most appropriate for collecting PROMs and PREMs , in practice, we included data collection at 3 months post-operatively. With the ever-present pressure to improve efficiencies, some clinics no longer routinely conduct face-to-face follow up appointments at 12 months, which reduces data capture rates. Since most of the functional improvement is achieved by 3-months post-surgery, it has become standard practice in the province to focus analysis and reporting on 3-month results using the substantially larger sample sizes at 3-months.
However, one of the greatest challenges that we faced was building a culture of evidence-based decision making. A critical aspect of the pathway implementation was the establishment of provincial committees to oversee the adoption of new evidence, including the Hip and Knee Arthroplasty Clinical Committee that maintains the clinical standards of care and the Hip and Knee Replacement Working Group that operationalizes the standards into clinical practice. These committees review more than a dozen key performance indicators, including aggregate pre- and post-surgery PROMs results to identify opportunities for improvement, and craft targeted quality improvement initiatives to investigate and address gaps in quality of care.
Based on our experiences, we suggest the following when considering implementation, collection and reporting on the use of PROMs and PREMs in routine clinical practice, our experiences in Alberta have highlighted some important aspects to consider, and we recommend the following:
Carefully consider the context in which data collection will occur – e.g. if using electronic data collection technology, be sure to confirm that there is reliable and secure WIFI connectivity everywhere;
Be sure that the clinic staff are familiar and comfortable with the technology before implementation and are able to assist patients in completing the PROMs and PREMs instruments if needed;
Conduct adequate testing of the data collection instruments with patients to ensure both functionality and usability of the instruments;
Consider implementation of PROMs and PREMs instruments as a package in one step rather than slow, phased deployments to minimize transition time when making changes (e.g. one instrument followed by another);
Tailor reports to the audience (clinician, clinic, health system) and include context against benchmarks and trends over time to inform changes in practice.
Further plans for PROMs applications in other areas of bone and joint health
The implementation of the new model of care for hip and knee arthroplasty has had significant impact on care delivery across the province in terms of multiple dimensions of quality of care including improvements in quality of life post-surgery (effectiveness), satisfaction with overall health care experience (acceptability), appropriateness of resources (appropriateness), reduced length of stay (efficiency), and reduced wait times (accessibility) . The application of PROMs in hip and knee arthroplasty has served as a reference example of at the national and international levels [1, 33]. Furthermore, at a time when wait times continue to grow due to an aging population and unforeseen demands for acute care on the healthcare system, long delays to receive elective surgery such as hip and knee arthroplasty will continue to have negative consequences on patient outcomes that need to be measured and reported to inform health policy . The learnings about the key elements of success for the successful implementation of models of care to reduce waiting times are relevant and transferable to other areas of care delivery .
The addition of PROMs data in clinical repositories and analyses enables the system to identify and address issues of inequity, appropriateness, and to explicitly recognize the trade-offs that are inherent in any resource-constrained system. For instance, the ability to empirically quantify the impact of waiting for service on patient quality of life elevates the discussions that are possible to include society’s “willingness to wait” and the appropriateness of global access benchmarks.
Availability of data and materials
Alberta bone and joint health institute
Health Utility Index Version 3
Optical character recognition
The organization for economic co-operation and development
Patient-reported outcome measures
Short Form 36
Total knee arthroplasty
Western Ontario and McMaster Universities Osteoarthritis Index
OECD. Health at a glance, (2017) OECD indicators. OECD Publishing, Paris, p 2017
Insight and Feedback Team (NHS England). National patient reported outcome measures (PROMs) programme consultation report. London2017.
The Swedish Hip Arthroplasty Register. The Swedish hip arthroplasty register annual report 2016.
OECD. OECD Patient-Reported Indicator Surveys (PaRIS) Initiative: Patient-Reported Outcome Measures (PROMs) for Hip and Knee Replacement Surgery - International Data Collection Guidelines. Ottawa, ON2019.
Gooch KL, Smith D, Wasylak T, Faris PD, Marshall DA, Khong H et al (2009) The Alberta hip and knee replacement project: a model for health technology assessment based on comparative effectiveness of clinical pathways. Int J Technol Assess Health Care 25(02):113. https://doi.org/10.1017/S0266462309090163
Gooch K, Marshall DA, Faris PD, Khong H, Wasylak T, Pearce T et al (2012) Comparative effectiveness of alternative clinical pathways for primary hip and knee joint replacement patients: a pragmatic randomized, controlled trial. Osteoarthritis Res Soc 20(10):1086–1094. https://doi.org/10.1016/j.joca.2012.06.017
Bellamy N, Buchanan WW, Goldsmith CH, Campbell J, Stitt LW (1988) Validation study of WOMAC: a health status instrument for measuring clinically important patient relevant outcomes to antirheumatic drug therapy in patients with osteoarthritis of the hip or knee. J Rheumatol 15(12):1833–1840
Feeny D, Furlong W, Boyle M, Torrance GW (1995) Multi-attribute health status classification systems. Pharmacoeconomics 7(6):490–502. https://doi.org/10.2165/00019053-199507060-00004
Ware JE Jr, Sherbourne CD (1992) The MOS 36-item short-form health survey (SF-36): I. Conceptual framework and item selection. Medical care. 30(6):473–83
Brooks R (1996) EuroQol: the current state of play. Health Policy 37(1):53–72. https://doi.org/10.1016/0168-8510(96)00822-6
Herdman M, Gudex C, Lloyd A, Janssen MF, Kind P, Parkin D et al (2011) Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 20(10):1727–1736. https://doi.org/10.1007/s11136-011-9903-x
Ortech Software. https://ortechsystems.com Accessed December 14, 2020.
REDCap Project. https://project-redcap.org Accessed December 14, 2020.
Jin X, Al Sayah F, Ohinmaa A, Marshall DA, Johnson JA (2019) Responsiveness of the EQ-5D-3L and EQ-5D-5L in patients following total hip or knee replacement. Qual Life Res. https://doi.org/10.1007/s11136-019-02200-1
Jin X, Al Sayah F, Ohinmaa A, Marshall DA, Smith C, Johnson JA. The EQ-5D-5L Is Superior to the -3L version in measuring health-related quality of life in patients awaiting THA or TKA. Clin Orthopaed Relat Res®. 2019;477(7):1632–44. doi: https://doi.org/10.1097/corr.0000000000000662.
Al Sayah F, Johnson J, Ohinmaa A, Dean S, Lahtinen M, Marshall D, et al. Enhancing the use of patient-reported outcome measures (PROMs) in the healthcare system in Alberta - a white paper. Edmonton, AB2020.
Mrklas K, Barber T, Campbell-Scherer D, Green L, Li L, Marlett M et al (2020) Co-design in the development of a mobile health app for the management of knee osteoarthritis by patients and physicians: qualitative study. JMIR Mhealth Uhealth 8(7):e17893. https://doi.org/10.2196/17893
Ravi B, Croxford R, Reichmann WM, Losina E, Katz JN, Hawker GA (2012) The changing demographics of total joint arthroplasty recipients in the United States and Ontario from 2001 to 2007. Best Pract Res Clin Rheumatol 26(5):637–647. https://doi.org/10.1016/j.berh.2012.07.014
Losina E, Thornhill TS, Rome BN, Wright J, Katz JN (2012) The dramatic increase in total knee replacement utilization rates in the United States cannot be fully explained by growth in population size and the obesity epidemic. J Bone Joint Surg 94(3):201–207. https://doi.org/10.2106/JBJS.J.01958
Hawker G, Connor-Spady B, Bohm E, Dunbar M, Jones C, Ravi B et al (2020) Patients pre-operative expectations of total knee arthroplasty and satisfaction with outcomes at one year: a prospective cohort study. Arthritis and Rheumatology. https://doi.org/10.1002/art.41510
Ostermann J, Brown D, van Til J, Bansback N, Legare F, Marshall D et al (2019) Support tools for preference-sensitive decisions in healthcare: Where are we? Where do we go? How do we get there? Patient 12:439–443. https://doi.org/10.1007/s40271-019-00372-z
Bansback N, Trenaman L, MacDonald K, Hawker G, Johnson J, Stacey D, et al. An individualized patient-reported outcome measure (PROM) based patient decision aid and surgeon report for patients considering total knee arthroplasty: Protocol for a pragmatic randomized controlled trial. BMC Musculoskeletal Disorders. 2019;20(89). doi: https://doi.org/10.1186/s12891-019-2434-2.
Marshall D, Christiansen T, Smith C, Squire Howden J, Werle J, Fyie K et al (2012) Voluntary versus involuntary waiting for joint replacements: new alberta wait times rules for hip and knee arthroplasties, with provincial consensus. Healthc Q 15(3):36–45
Marshall D, Christiansen T, Smith C, Squire-Howden J, Werle J, Faris P et al (2015) Continuous quality improvement program for hip and knee replacement surgical care. Am J Med Qual 30(5):425–431. https://doi.org/10.1177/1062860614540512
Werle J, Dobbelsteyn L, Feasel L, Hancock B, Job B, Makar L, Manning H, Quigley S, Teppler A, Smith C, Kelly S, Wasylak T (2010) A study of the effectiveness of performance-focused methodology for improved outcomes in Alberta public healthcare. Healthc Manage Forum 23:169–174
Marshall DA, Vanderby S, Barnabe C, MacDonald KV, Maxwell C, Mosher D et al (2015) Estimating the burden of osteoarthritis to plan for the future. Arthritis Care Res 67(10):1379–1386. https://doi.org/10.1002/acr.22612
Vanderby SA, Carter MW, Noseworthy T, Marshall DA (2015) Modelling the complete continuum of care using system dynamics: the case of osteoarthritis in Alberta. J Simul 9(2):156–169. https://doi.org/10.1057/jos.2014.43
Marshall DA, Burgos-Liz L, IJzerman MJ, Osgood ND, Padula WV, Higashi MK, et al. Applying dynamic simulation modeling methods in health care delivery research-the SIMULATE checklist: report of the ISPOR simulation modeling emerging good practices task force. Value Health. 2015;18(1):5–16. doi: https://doi.org/10.1016/j.jval.2014.12.001.
Marshall DA, Burgos-Liz L, IJzerman MJ, Crown W, Padula WV, Wong PK, et al. Selecting a dynamic simulation modeling method for health care delivery research-part 2: report of the ISPOR Dynamic Simulation Modeling Emerging Good Practices Task Force. Value Health. 2015;18(2):147–60. doi: https://doi.org/10.1016/j.jval.2015.01.006.
Alberta Bone and Joint Health Institute. The Osteoarthritis Crisis in Alberta: Access, Quality and Long-Term Planning. 2019. p. 27.
Rolfson O, Bohm E, Franklin P, Lyman S, Denissen G, Dawson J, et al. Patient-reported outcome measures in arthroplasty registries Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries Part II. Recommendations for selection, administration, and analysis. Acta Orthop. 2016;87 Suppl 1:9–23. doi: https://doi.org/10.1080/17453674.2016.1181816.
Frank C, Marshall D, Faris P, Smith C. Essay for the CIHR/CMAJ award: improving access to hip and knee replacement and its quality by adopting a new model of care in Alberta. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2011;183(6):E347-E50. doi: https://doi.org/10.1503/cmaj.110358.
Canadian Institutes for Health Information. PROMs background document. 2015.
Lopatina E, Damani Z, Bohm E, Noseworthy T, Conner-Spady B, MacKean G et al (2017) Single-entry models (SEMs) for scheduled services: towards a roadmap for the implementation of recommended practices. Health Policy 121(9):963–970. https://doi.org/10.1016/j.healthpol.2017.08.001
About the supplement
This article has been published as part of Journal of Patient-Reported Outcomes Volume 5, Supplement 2 2021: The Use of PROMs in Health Systems – Implementation Stories from Alberta, Canada. The full contents of the supplement are available at https://jpro.springeropen.com/articles/supplements/volume-5-supplement-2.
DM is supported by the Arthur J.E. Child Chair in Rheumatology and a Canada Research Chair in Health Systems and Services Research (2008–2018). Article processing fees for the publication of this supplement were paid by a grant from the EuroQol Research Foundation to Alberta PROMs and EQ-5D Research and Support Unit (APERSU), University of Alberta. Authors received no direct remuneration.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Marshall, D.A., Jin, X., Pittman, L.B. et al. The use of patient-reported outcome measures in hip and knee arthroplasty in Alberta. J Patient Rep Outcomes 5 (Suppl 2), 87 (2021). https://doi.org/10.1186/s41687-021-00362-6
- Patient-reported outcome measures (PROMs)
- Patient-reported experience measures (PREMs)
- Hip and knee arthroplasty
- Decision aid
- Key performance indicators