Skip to main content

Selection of patient-reported outcome measures (PROMs) for use in health systems

Abstract

Many healthcare systems around the world have been increasingly using patient-reported outcome measures (PROMs) in routine outcome measurement to enhance patient-centered care and incorporate the patient’s perspective in health system performance evaluation. One of the key steps in using PROMs in health systems is selecting the appropriate measure(s) to serve the purpose and context of measurement. However, the availability of many PROMs makes this choice rather challenging. Our aim was to provide an integrated approach for PROM(s) selection for use by end-users in health systems.

The proposed approach was based on relevant literature and existing guidebooks that addressed PROMs selection in various areas and for various purposes, as well as on our experience working with many health system users of PROMs in Canada. The proposed approach includes the following steps: (1) Establish PROMs selection committee; (2) Identify the focus, scope, and type of PROM measurement; (3) Identify potential PROM(s); (4) Review practical considerations for each of the identified PROMs; (5) Review measurement properties of shortlisted PROMs; (6) Review patient acceptance of shortlisted PROMs; (7) Recommend a PROM(s); and (8) Pilot the selected PROM(s). The selection of appropriate PROMs is one step in the successful implementation of PROMs within health systems, albeit, an essential step. We provide guidance for the selection of PROMs to satisfy all potential usages at the micro (patient-clinician), meso (organization), and macro (system) levels within the health system. Selecting PROMs that satisfy all these purposes is essential to ensure continuity and standardization of measurement over time. This is an iterative process and users should consider all the available information from all presented steps in selecting PROMs. Each of these considerations has a different weight in diverse clinical contexts and settings with various types of patients and resources.

Background

Many healthcare systems around the world have been increasingly using patient-reported outcome measures (PROMs) in routine outcome measurement to enhance patient-centered care and incorporate the patient’s perspective in health system performance evaluation [1,2,3,4,5,6]. Hundreds of PROMs are available and there are many more in development [7]. One of the key steps in using PROMs in health systems is selecting the appropriate measure(s) to serve the purpose and context of measurement. However, the availability of many PROMs makes this choice rather challenging.

Unlike their application in comparative effectiveness research or population health surveillance, the use of PROMs in health systems is intended to serve multiple purposes for several users [2, 3]. At the micro level, PROMs are used by clinicians to support individual patients’ clinical management. At the meso level, PROMs data are aggregated and used by administrators and managers to evaluate programs and inform service delivery and quality improvement initiatives at an organization level. At the macro level, aggregated PROMs data are used by executives and policy-makers for health system performance evaluation and for public reporting. This poses a challenge on the process of selecting a PROM that could satisfy all these potential purposes and users’ needs.

An important consideration in the use of PROMs in health systems is the standardization and consistency of measurement across the system [8]. For example, in England, the National Health Service (NHS) initiated the first system-level use of PROMs in four elective surgeries under the NHS PROMs program [5]. The EQ-5D is used as a generic PROM across all surgeries, alongside a condition-specific PROM for each clinical area. In Sweden, national quality registers use EQ-5D as a generic PROM across all registers, alongside condition-specific PROMs in some registers [6]. In Canada, the Edmonton Symptom Assessment Scale, a generic PROM of symptom burden, is used across all cancer care centers [9], and a modified version for renal care is collected in many hemodialysis units [10, 11]. In the province of Alberta, the EQ-5D-5L is used as a generic PROM in routine outcome measurement across the healthcare system [2]. This standardization in measurement is crucial as it allows generating population norms and setting benchmarks against which to compare PROMs in specific clinical groups.

The need to serve multiple purposes and users and the importance of consistency and standardization in measurement make the selection of PROMs for use within health systems a crucial component of successful PROMs programs. Previous literature focused on providing guidance for selecting PROM(s) for research purposes in specific clinical areas [12,13,14,15]. In this paper, we expand on previous work and provide an integrated approach for PROM(s) selection for use in health systems.

Taxonomy of PROMs

PROMs could be categorized in several ways; here we provide a taxonomy of PROMs based on the focus, scope, and type of measurement.

Focus of measurement

Health outcomes are broadly defined as changes in health status that occur as a result of a health-related event or illness, or a healthcare intervention. Some health outcomes are patient-reported and are measured using PROMs; these include outcomes such as symptom burden (e.g. pain, dizziness), functional status (e.g. physical functioning) and health-related quality of life [16]. Identifying a patient-reported outcome of interest is crucial to PROMs selection. In health system applications, it is important to consider overall health as an outcome of interest given its usefulness in performance evaluations and comparative analyses [17]. Nonetheless, other outcomes, such as symptom burden and disease-specific quality of life, could be considered.

Scope of measurement

PROMs could be categorized as generic or condition-specific. Generic PROMs are designed to assess general aspects of health that are not specific to a particular disease (e.g., EQ-5D, SF-36, PROMIS), while condition-specific PROMs assess aspects of health specific to a disease (e.g., WOMAC, EORTC-QLQ-C30) [18]. Generic and condition-specific PROMs each have their advantages and disadvantages. Generic PROMs are useful when comparing patients across different health conditions; however, changes in specific aspects of health are better assessed using condition-specific PROMs. Because of these advantages and disadvantages, it is often recommended to use a generic and a condition-specific PROM. In using a combination of generic and disease-specific measures, it is important for users to check and compare the domains or items in each of the measures, and avoid redundancy between the two measures which imposes undue respondent burden and potentially annoyance and reduce compliance with repeated assessment over time.

Type of measurement

PROMs could be also categorized as profile or preference-based. Profile measures (e.g., SF-36, WHOQOL, PROMIS) are used to determine the position of some characteristic on specific domains, which are measured by multiple items and reported by domain [19]. Preference-based measures (e.g., EQ-5D, HUI) are used to place a value on health states or conditions, and typically yield a single multi-domain index score. Profile measures are useful in applications that aim to assess health or an aspect of it from a patient perspective in the evaluation of treatments or services. While preference-based measures could be used for these purposes as well, they are also used to generate utility values to calculate quality-adjusted life years for use in health economic evaluations.

Selecting PROMs for use in health systems

The selection of PROMs requires a balance among several factors and pragmatic considerations. A health system may wish to establish a common measure, while also enabling a degree of flexibility for more tailored/specific PROM collection. The following approach is based on previous literature and guidebooks on PROMs selection [12,13,14,15, 17, 20,21,22,23,24] identified via a brief scoping review and included key and high quality articles on this topic, as well as our experience working with many health system users of PROMs in Canada, which helped identify important practical issues around PROMs selection. This approach to PROMs selection for use within health systems could be modified to align with contextual factors in certain settings or clinical areas. The following steps are presented in a sequential fashion; however, this process is iterative and users may need to go back and forth between the different steps until completion.

Step 1: Establish PROMs selection committee

PROMs program managers should establish a PROMs selection committee that includes system administrators, PROMs methodologic experts, patient or care provider representative, clinicians, and policy-makers. The committee would be involved in all steps of the PROMs selection process, and would periodically review, deliberate and provide feedback on various aspects of the process.

Step 2: Identify the focus, scope, and type of PROM measurement

The committee would identify the target population, which could be based on a clinical condition or procedure, or may include all patients within the system. Additionally, the committee should identify the focus of measurement, which may include one or more patient-reported outcomes (e.g., symptom burden, health-related quality of life), as well as the scope (generic, condition-specific), and type of measurements (profile, preference-based) to guide PROMs selection.

The type of PROM(s) to use depends on the purpose of measurement. In selecting PROM(s) for use in health systems, the measures need to satisfy two key purposes: use at the individual patient (i.e., micro-) level to support patient clinical management, and use at the aggregate (i.e., meso-, macro-) level to support performance evaluation, healthcare delivery planning, comparative effectiveness analysis, as well as economic and policy analyses. Given the multiple purposes and users of PROMs within the system, multiple measures may need to be identified to serve all these purposes.

From a patient perspective, PROMs represent an opportunity to report specific aspects of health or problems, which vary depending on the clinical context. While cancer patients may consider symptom burden an important outcome, physical functioning would be important for patients after a hip arthroplasty. Clinicians on the other hand focus on outcomes presented in clinical guidelines and tend to value condition-specific measures and symptom assessment scales given their clinical usefulness. Quality managers, administrators, and policy-makers value generic measures of overall health that allow comparisons across clinical conditions and performance evaluation at the system level, as well as the evaluation of health outcomes as they relate to health policy.

Step 3: Identify potential PROM(s)

After identifying the focus and type of measurements, a scoping review of the literature to identify existing measures that fit the purposes of measurement, with a focus on their conceptual and measurement model, should be conducted. This review should focus on studies that developed, modified or validated a certain measure. Some generic PROMs such as the EQ-5D and SF-36, and condition or symptom-specific measures such as the EORTC-QLQ-C30 and PHQ-9 have a long legacy of use and are often identified by PROMs experts. A list of all identified measures is compiled and evaluated in the next step.

Step 4: Review practical considerations

Practical considerations should be reviewed to shortlist the identified measures as these are determining factors for selecting PROMs in many contexts (Table 1). A key consideration is the ease of PROM integration into clinical workflow, which depends on the length of the measure (lengthy PROMs increase respondent burden and may lead to incomplete data), ease of administration (e.g., simple presentation of questions), and availability of various administration modes (e.g., paper-based, web, proxy) and languages. The choice of a collection mode depends on the proposed use of the data and availability of technology and resources for establishing collection and reporting systems. Consideration must also be given to regulatory expectations, data security and ownership, and ease of access and data interoperability, all of which are impacted by planned uses of the data.

Table 1 Practical considerations of PROMS implementation

The timing and frequency of PROM(s) administration, which also impacts the integration into clinical workflow, is also important. Timing and frequency of measurements depend on the clinical context. For example, when a specific intervention is delivered, such as surgeries, PROMs are often administered once before surgery and once or more afterwards, while patients with chronic illness may complete PROMs at each clinic visit. Users should also consider the reference period of the measure(s), which should align with the timing and frequency of measurements.

If a preference-based PROM is identified, users need to identify whether a country-specific value set is available. Finally, users need to consider licensing fees. While some PROMs are available in the public domain with published norms and scoring protocols, others are proprietary and fees associated with large-scale and long-term use within health systems need to be considered.

Step 5: Review measurement properties

Once the identified measures are shortlisted, a review of their measurement properties is done. Although many validated PROMs have been in use for decades, they must be evaluated for appropriateness with specific patient populations and conditions. For use in the health system, there needs to be sufficient evidence on PROM(s) reliability, validity, responsiveness and interpretability as they are intended to assess an outcome at one time point and changes in the outcome over time. Guidance is available to inform such considerations, such as the “Terwee criteria” [25] and COSMIN guideline [21]. Users could also identify systematic reviews examining measurement properties of shortlisted PROMs.

Step 6: Review patient acceptance

Given that patient representatives are members of the committee and are engaged in the selection process, patient acceptance would presumably be addressed in previous steps. However, it is important to consider some patient attributes that may influence PROMs selection such as language, literacy level, and cultural appropriateness, as well as other factors such as visual impairment, dexterity issues or other functional limitations that may affect patients’ ability to complete PROM(s). Patients or patient representatives who were not involved in the selection process could be consulted at this stage.

Step 7: Recommend a PROM(s)

After careful assessment of practical considerations, measurement properties, and patient acceptance, the PROMs selection committee would recommend a PROM or multiple PROMs for use in the target population. For example, the use of a common, generic measure across the system, while allowing the flexibility of integrating another PROM(s) for a specific population, geographic area or other sub-population of interest.

Step 8: Pilot the PROM(s)

Before large-scale use of the selected PROM(s) within the system, we encourage users to consider piloting the selected PROM(s) in the target setting. Evaluation measures, such as brief surveys, interviews and narratives among patients, users (collectors) and management should be included. This pilot would further inform patient and clinician acceptance of PROM(s), expose any challenges with integration into clinical workflow, and uncover other contextual issues that may have been missed during the selection process. Once complete, reports to the PROMS selection committee regarding the integration into clinical workflow, data analysis and value would strengthen the PROMs program and subsequent implementations.

The selection of appropriate PROMs is one step in the successful implementation of PROMs within health systems, albeit, an essential step. We provide guidance for the selection of PROMs to satisfy all potential usages at the micro (patient-clinician), meso (organization), and macro (system) levels within the health system. Selecting PROMs that satisfy all these purposes is essential to ensure continuity and standardization of measurement over time. This is an iterative process and users should consider all the available information from all presented steps in selecting PROMs. Each of these considerations has a different weight in diverse clinical contexts and settings with various types of patients and resources.

Availability of data and materials

Not applicable.

Abbreviations

COSMIN:

Consensus-based standards for the selection of health measurement instruments

EORTC-QLQ-C30:

European Organization for the Research and Treatment of Cancer, quality of life questionnaire

EQ-5D-5L:

EuroQol, 5-dimension, 5-level questionnaire

HUI:

Health utility index

NHS:

National Health Service

PHQ-9:

Patient health questionnaire, 9-items

PROMIS:

Patient-reported outcomes measurement information system

PROMs:

Patient-reported outcome measures

SF-36:

36-Item, Short-form health survey

WHOQOL:

World Health Organization quality of life assessment

WOMAC:

Western Ontario and McMaster universities arthritis index

References

  1. 1.

    (CMS), C.f.M.a.M.S. (2020) Health Outcomes Survey (HOS). https://www.cms.gov/Research-Statistics-Data-and-Systems/Research/HOS. Accessed 25 Aug 2021

  2. 2.

    Al-Sayah F et al (2020) Enhancing the use of patient-reported outcome measures (PROMs) in the healthcare system in Alberta. Alberta PROMs and EQ-5D Research and Support Unit (APERSU), Edmonton

    Google Scholar 

  3. 3.

    Devlin N, Appleby J (2010) Getting the most out of PROMs: putting health outcomes at the heart of NHS decision-making. The King’s Fund and The Office Health Economics, London

    Google Scholar 

  4. 4.

    Devlin NJ et al (2019) International meeting for health system PROMs users

  5. 5.

    Digital N (2020) Patient Reported Outcome Measures (PROMs). https://digital.nhs.uk/data-and-information/data-tools-and-services/data-services/patient-reported-outcome-measures-proms. Accessed 25 Aug 2021

  6. 6.

    Ernstsson O, Janssen MF, Heintz E (2020) Collection and use of EQ-5D for follow-up, decision-making, and quality improvement in health care—the case of the Swedish National Quality Registries. J Patient Rep Outcomes. https://doi.org/10.1186/s41687-020-00231-8

    Article  PubMed  PubMed Central  Google Scholar 

  7. 7.

    Pinotti R (2016) PROQOLID. J Med Libr Assoc 104(1):91–92

    Article  Google Scholar 

  8. 8.

    OECD (2017) Recommendations to OECD Ministers of Health from the high level reflection group on the future of health statistics: strengthening the international comparison of health system performance through patient-reported indicators. OECD

    Google Scholar 

  9. 9.

    Cancer, C.P.A. (2017) Effective, efficient, evolving: enhancing cancer control in Canada. CPAC, Toronto

    Google Scholar 

  10. 10.

    Davison SN, Jhangri GS, Johnson JA (2006) Longitudinal validation of a modified Edmonton symptom assessment system (ESAS) in haemodialysis patients. Nephrol Dial Transplant 21(11):3189–3195

    Article  Google Scholar 

  11. 11.

    Davison SN (2011) Integrating palliative care for patients with advanced chronic kidney disease: recent advances, remaining challenges. J Palliat Care 27(1):53–61

    Article  Google Scholar 

  12. 12.

    Bryan S et al (2014) Choosing your partner for the PROM: a review of evidence on patient-reported outcome measures for use in primary and community care. Healthc Policy 10(2):38–51

    PubMed  PubMed Central  Google Scholar 

  13. 13.

    Ju A, Tong A (2017) Considerations and challenges in selecting patient-reported outcome measures for Clinical Trials in nephrology. Clin J Am Soc Nephrol 12(11):1882–1884

    Article  Google Scholar 

  14. 14.

    Luckett T, King MT (2010) Choosing patient-reported outcome measures for cancer clinical research—practical principles and an algorithm to assist non-specialist researchers. Eur J Cancer 46(18):3149–3157

    CAS  Article  Google Scholar 

  15. 15.

    Reaney M, Black P, Gwaltney C (2014) A systematic method for selecting patient-reported outcome measures in diabetes research. Diabetes Spectr 24(4):229–232

    Article  Google Scholar 

  16. 16.

    Weldring T, Smith SM (2013) Patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs). Health Serv Insights 6:61–68

    PubMed  PubMed Central  Google Scholar 

  17. 17.

    Cella D et al (2015) Patient-reported outcomes in performance measurement. RTI International, Research Triangle Park

    Book  Google Scholar 

  18. 18.

    Patrick DL, Deyo RA (1989) Generic and disease-specific measures in assessing health status and quality of life. Med Care 27(3 Suppl):S217–S232

    CAS  Article  Google Scholar 

  19. 19.

    Elsevier., P.F.M.K.A.P. (2017) The measurement of health and health status: concepts, methods and applications from a multidisciplinary perspective, 1st edn. Academic Press, Berlin

    Google Scholar 

  20. 20.

    Aaronson N et al (2016) User s guide to implementing patient-reported outcomes assessment in clinical practice

  21. 21.

    Prinsen CAC et al (2018) COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 27(5):1147–1157

    CAS  Article  Google Scholar 

  22. 22.

    Prinsen CAC et al (2016) How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”—a practical guideline. Trials 17(1):449

    Article  Google Scholar 

  23. 23.

    Reeve BB et al (2013) ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 22(8):1889–1905

    Article  Google Scholar 

  24. 24.

    van der Wees PJ et al (2019) Development of a framework with tools to support the selection and implementation of patient-reported outcome measures. J Patient Rep Outcomes. https://doi.org/10.1186/s41687-019-0171-9

    Article  PubMed  PubMed Central  Google Scholar 

  25. 25.

    Terwee CB et al (2007) Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 60:34–42

    Article  Google Scholar 

Download references

Acknowledgements

Allison Soprovich for assisting with edits and the final submission.

About this supplement

This article has been published as part of Journal of Patient-Reported Outcomes Volume 5, Supplement 2 2021: The Use of PROMs in Health Systems – Implementation Stories from Alberta, Canada. The full contents of the supplement are available at https://jpro.springeropen.com/articles/supplements/volume-5-supplement-2.

Funding

Article processing fees for the publication of this supplement were paid by a grant from the EuroQol Research Foundation to Alberta PROMs and EQ-5D Research and Support Unit (APERSU), University of Alberta. Authors received no direct remuneration.

Author information

Affiliations

Authors

Contributions

All authors (FAS, XJ, JAJ) created the content and conceived the article. FAS drafted the first draft of this manuscript. All co-authors (FAS, XJ, JAJ) reviewed and revised the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Fatima Al Sayah.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Al Sayah, F., Jin, X. & Johnson, J.A. Selection of patient-reported outcome measures (PROMs) for use in health systems. J Patient Rep Outcomes 5, 99 (2021). https://doi.org/10.1186/s41687-021-00374-2

Download citation