Skip to main content
Journal of Patient-Reported Outcomes
Download PDF

Implementing patient-reported outcomes in routine clinical care for diverse and underrepresented patients in the United States

Journal of Patient-Reported Outcomes volume 6, Article number: 20 (2022) Cite this article

Abstract

Background

Patient-reported outcomes (PROs) are used increasingly in routine clinical care and inform policies, reimbursements, and quality improvement. Less is known regarding PRO implementation in routine clinical care for diverse and underrepresented patient populations.

Objective

This review aims to identify studies of PRO implementation in diverse and underrepresented patient populations, elucidate representation of clinical specialties, assess implementation outcomes, and synthesize patient needs, concerns, and preferences.

Methods

MEDLINE, Embase, Web of Science, CINAHL, and PsycINFO were searched September 2021 for studies aiming to study PRO implementation in diverse and underrepresented patient populations within the United States. Studies were screened and data extracted by three independent reviewers. Implementation outcomes were assessed according to Proctor et al. taxonomy. A descriptive analysis of data was conducted.

Results

The search yielded 8,687 records, and 28 studies met inclusion criteria. The majority were observational cohort studies (n = 21, 75%) and conducted in primary care (n = 10, 36%). Most studies included majority female (n = 19, 68%) and non-White populations (n = 15, 54%), while fewer reported socioeconomic (n = 11, 39%) or insurance status (n = 9, 32.1%). Most studies assessed implementation outcomes of feasibility (n = 27, 96%) and acceptability (n = 19, 68%); costs (n = 3, 11%), penetration (n = 1, 4%), and sustainability (n = 1, 4%) were infrequently assessed.

Conclusion

PRO implementation in routine clinical care for diverse and underrepresented patient populations is generally feasible and acceptable. Research is lacking in key clinical specialties. Further work is needed to understand how health disparities drive PRO implementation outcomes.

Plain English summary

Patient-reported outcomes (PROs) allow doctors and researchers to understand the patient perspective, such as how they are doing physically, mentally, or socially. When used, PROs can improve health and increase satisfaction of patients. Many clinics and hospitals are interested in using PROs in everyday care. Doctors, hospitals, and insurance companies are also using information from PROs to decide if the care they give is good quality. Unfortunately, certain groups of patients, such as racial and ethnic minorities and patients with low income, report worse PROs. Because of these differences, it will be important to make sure that PROs are being collected from all people, but not much is known regarding how this has been done. This study demonstrates what is known so far with regard to using PROs in everyday clinical care for these diverse patient groups. Findings from this study show that PROs can be successfully collected, but more work is needed in certain medical fields, and some types of patients have specific needs, concerns, or preferences with regard to PRO collection.

Background

Patient-reported outcomes (PROs) allow clinicians and researchers to better understand patients’ perceptions of their health, goals, treatment, and healthcare experiences across different domains, such as physical, mental, and social well-being [1, 2]. Patient-reported outcome measures (PROMs) are tools used to measure PROs in various disease contexts. Ideally PROMs are rigorously tested and validated. While many PROMs were developed for use in clinical trials [3], the use of PROMs in routine clinical care has become more widespread and has been shown to improve symptom control, supportive care measures, patient satisfaction, and health outcomes [4, 5]. In addition, there is increasing demand from healthcare payors, regulators, and administrators to incorporate PROs into routine clinical care for quality improvement and reimbursement purposes [6, 7].

Several logistical challenges regarding the implementation of PRO data collection into routine clinical practice have been noted, such as selection of appropriate and relevant PROMs, adequate staffing and data resources, patient compliance, and non-integration with electronic health records [6, 8,9,10]. However, an additional concern is ensuring equitable PRO implementation, such that PROs capture diverse and underrepresented patient populations [6, 11].

Given the current landscape of biomedical research, where less than 2% of 10,000 clinical trials included sufficient numbers of minority participants in one 2015 report [12], for example, it is unlikely that PRO implementation research would be immune to these issues of health equity and healthcare disparities in clinical research. PRO implementation in routine clinical care has been documented to have low adoption nationwide [13], which may further impede efforts to capture diverse populations. Additionally, reporting of certain factors such as race and ethnicity in clinical trials has not been widespread [14].

Routine PRO data collection is therefore likely predominant in White, higher socioeconomic, and English-speaking populations [15]. This is further supported by a systematic review of PRO implementation in routine care for patients with breast cancer, in which only 2 of 34 identified studies targeted minority patient populations [16] and a study of PRO implementation in integrative medicine, in which the majority of patients were White and had commercial insurance [17]. In addition, clinical PRO implementation studies have identified disparate response rates for many diverse and underrepresented populations across clinical specialties [18,19,20,21].

The lack of participation of certain patient populations in PRO data collection is concerning in light of a growing body of evidence demonstrating disparities in PROs among diverse and underrepresented patient populations. For example, PROs were significantly associated with race, education, and neighborhood poverty in a study of hip and knee arthroplasty [22], and lower income families experienced higher symptom burden and worse quality of life in a pediatric oncology study [23]. As the demand for PRO implementation in routine clinical care grows, it will therefore be critical to collect PROs from diverse and underrepresented patient populations to ensure representative sampling and reduce health disparities [24].

However, there is presently a limited understanding of the landscape of PRO implementation in these populations. This study therefore aims to review the literature to determine what is presently known regarding implementation of PROs in routine clinical care for diverse and underrepresented populations. The primary aim of this review is to characterize PRO implementation in routine clinical care, in terms of patient populations studied and clinical specialties that have evaluated PRO implementation. The secondary aims of this review are to analyze implementation outcomes and population-specific concerns, needs, and preferences regarding PRO collection. We believe that the review will advance our understanding of existing PRO programs and identify important areas of future research.

Methods

Given the lack of knowledge regarding available evidence and breadth of coverage of PRO implementation in diverse and underrepresented populations across clinical specialties, a scoping review methodology was adopted for this study. Per guidance by Munn et al. [25], a scoping review enables determination of the studies available and overall focus of a research area, aligning with this study’s aims to identify the landscape of available studies, populations and clinical specialties represented, and implementation outcomes evaluated to date. The review was conducted in accordance with the Joanna Briggs Institute (JBI) methodology for scoping reviews [26].

Data source and search

Studies were identified by searching the following electronic databases: MEDLINE (Ovid), Embase (Elsevier), Web of Science Core Collection (Clarivate Analytics), Cumulative Index to Nursing and Allied Health Literature (CINAHL: EBSCO), and PsycINFO (EBSCO). Searches were conducted between September 23 and 27, 2021. No date restriction was applied. The terms used to search each database were informed by previously published systematic reviews investigating PRO implementation in routine clinical care [16, 27, 28], as opposed to PROs used as evaluative assessments only for interventions in clinical trials, and relevant MeSH terms related to health disparities or diverse and underrepresented populations known to be affected by healthcare disparities [29]. The search was designed and conducted by a medical librarian with expertise in systematic and scoping reviews (PAB). Search strategy for the included databases is provided in the Additional file 1: Table S1.

Study selection and eligibility

Studies were compiled into the Covidence (Veritas Health Innovation) citation manager. First, abstracts and titles were screened by three independent reviewers (CJH, RG, RD) using inclusion and exclusion criteria. Full texts were then reviewed by three independent reviewers (CJH, RG, RD) to determine whether studies met inclusion criteria. Disagreements at each phase were resolved through discussion among the three reviewers.

Inclusion criteria were studies that (1) assessed PRO implementation in routine clinical care, (2) had a specific and explicit aim of studying PRO implementation in diverse or underrepresented populations, such as racial or ethnic minorities, sexual and gender minorities, elderly or geriatric populations ≥ 65 years of age, and populations with diverse literacy, educational, socioeconomic or geographic (rural or urban) status, (3) used a validated PROM, and (4) English language. Studies were limited to those conducted in the United States because diverse and underrepresented populations are in part defined by the specific historical, economic, and cultural contexts of their country of origin, as detailed in a previously published review [30]. In addition, certain sociodemographic factors such as insurance status and results concerning routine clinical care and PRO collection would be more generalizable to the unique healthcare system within the United States. Studies that focused on the development, creation, and/or validation of PROMs; use of PROMs as an outcome measure for some other primary intervention; and case reports, study protocols, editorials, dissertations, conference abstracts, and these were excluded. Citations within review articles from the search were manually reviewed to identify additional primary studies that met inclusion criteria. Results of the search as well as inclusion and exclusion of studies are reported according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Scoping Reviews (PRISMA-ScR) flow diagram (Fig. 1) [31].

Fig. 1
figure 1

Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Scoping Reviews (PRISMA-ScR) diagram detailing inclusion and exclusion of studies included in the analysis

Full size image

Data analysis

Data from relevant studies were extracted into an a priori defined form by three reviewers (CJH, RG, RD), including study aims, sample size and demographics, geographic location of study settings, type of study setting, clinical specialty, validated PROM used, study limitations, if technology was used and type, and implementation outcomes. Implementation outcomes were categorized according to previously defined conceptual distinctions for implementation research: acceptability, adoption, appropriateness, costs, feasibility, fidelity, penetration, and sustainability [32]. These categorizations have been further refined for evaluation of technological interventions in healthcare [33] and have been used in previously published reviews of PRO implementation [28]. The quality and level of evidence (1–7) of studies were determined based on previously developed criteria by Melnyk et al., where level 1 corresponds to the highest quality evidence (systematic reviews, meta-analyses of randomized controlled trials, etc.) [34, 35].

Results

Study selection

The electronic database search yielded 8,687 records, and 1 record was identified through a review article known to study authors [16]. After duplicates were removed, the titles and abstracts of 4,452 records were each screened in duplicate according to pre-defined inclusion criteria (agreement > 95%). After screening these records, a total of 76 full-text articles were reviewed, of which 55 were excluded. Manual review of bibliographies from review articles identified in the search yielded an additional 7 studies that met inclusion criteria. A total of 28 studies were included in the analysis (Fig. 1).

Quality assessment and study characteristics

The aims, quality, and general characteristics of studies are included in Table 1. Most studies were observational cohort studies with level 4 evidence (n = 21, 75%), followed by level 2 randomized controlled studies (n = 5, 17.9%), and level 6 qualitative studies (n = 2, 7.1%). Sample sizes within studies ranged from 10 [36] to 6454 [37]. Primary care was the most common specialty represented (n = 10, 35.7%) [11, 36, 38,39,40,41,42,43,44,45], followed by oncology (n = 4, 21.4%) [46,47,48,49,50,51], of which two studies were specific to breast oncology [46, 47] and two studies were specific to urologic/radiation oncology [49, 50]. Remaining specialties included rheumatology [52,53,54,55] (n = 4, 14.3%), psychology/psychiatry [56,57,58] (n = 3, 10.7%), and one study each (3.6%) for neurology [37], geriatrics [59], trauma [60], home health care [61], and orthopedic hand surgery [62].

Table 1 Characteristics of the studies (n = 28) included in the review
Full size table

The demographics and characteristics of study populations are detailed in Table 2. Most included a study population that was majority female (n = 19, 67.9%) compared to majority male (n = 3, 10.7%). Some studies reported multiple samples with variable gender majorities (n = 4, 14.3%) or did not report gender (n = 2, 7.1%). Most studies (n = 15, 53.6%) included majority non-White populations and reported educational attainment and/or literacy (n = 19, 67.9%). In contrast, most studies did not report income or employment status (n = 17, 60.7%) or insurance status (n = 19, 67.9%). For studies that reported participants’ age, the age range among studies was 26.7 to 75.9 years. The majority of studies (n = 23, 82.1%) used electronic PRO (ePRO) data collection.

Table 2 Study population demographics and characteristics
Full size table

Implementation outcomes

Implementation outcomes are detailed in Table 3. Most studies evaluated feasibility (n = 27, 96.4%), followed by acceptability (n = 19, 67.9%), adoption (n = 18, 64.3%), and appropriateness (n = 10, 35.7%). Four studies assessed fidelity (14.3%) and three studies assessed costs (10.7%). One study each evaluated for penetration (3.6%) and sustainability (3.6%). Specific details and examples of each implementation outcome are detailed in Additional file 2: Table S2. Of those assessing feasibility, the majority assessed PRO completion rates over time (n = 17, 60.7%). Fewer assessed time needed to complete PRO assessments (n = 7, 25%) or impact on clinic workflows, such as the need for staff assistance in PRO completion or work burden (n = 7, 25%). Of those assessing adoption, few reported patients’ intention to try the studied PRO collection method again or in another setting (n = 3, 10.7%) and one reported clinicians’ intention to adopt new PRO technology in practice [53]. Of the three studies that assessed costs, none assessed cost-effectiveness. With regard to the one study assessing penetration and sustainability, the number of providers interfacing with the PRO system increased over time and the program was able to be maintained over a 5-year study period [40].

Table 3 Implementation outcomes of studies classified according to Proctor et al. [32] taxonomy
Full size table

Overall concerns, needs, and preferences of populations studied

Overall concerns, needs, and preferences were abstracted from studies and detailed in Table 4. Among the concerns were disparities in PRO completion among racial and ethnic minorities [11, 38, 50, 62], Spanish-speaking patients [11], populations with low income/employment status and low educational or health literacy [62], and elderly or geriatric populations [11, 38, 62]. Needs among populations included additional help in completing surveys among patients with low income/education [55] and a suggestion for a tutorial video for ePRO use among elderly patients [59]. Preferences among populations included a higher likelihood of Black patients selecting automated telephone over web-based surveys [49, 50], conflicting results of Spanish-speaking patients preferring face-to-face interviews vs. electronic data collection [43,44,45], symptom report display using bar graphs with “emojis” for low health literacy, majority Black patients [41], and preference for using a finger over stylus for tablet-based PRO collection in elderly patients [61].

Table 4 Overall concerns, needs, and preferences of populations identified in the review
Full size table

Discussion

This review synthesizes studies to date that have explicitly aimed to evaluate PRO implementation in routine clinical care for diverse and underrepresented patient populations in the United States across all clinical specialties, thereby taking an important step in furthering our understanding in this area. Given growing demands for routine clinical PRO collection, this review specifically evaluated study quality, demographics, implementation outcomes, and patient perspectives in order to inform existing and emerging PRO programs as well as to identify areas requiring further research.

This review demonstrated that relatively few high-quality studies such as randomized-controlled trials have been conducted. In addition, studies are not representative of all clinical specialties and skew predominately toward primary care settings. In particular, there was a paucity of studies conducted in surgical, obstetric, and pediatric settings. While this may be partly explained by pre-existing disparities in access to and quality of care in these settings [63,64,65,66], it will be important to study PRO implementation across the entirety of the healthcare spectrum with particular attention to those presently unrepresented in research to date.

In recent years, several health information technology interventions have been developed to better facilitate electronic PRO (ePRO) implementation, such as web-based platforms, tablets, and mobile applications. While concerns have been raised, given that there are disparities in smartphone and tablet computer ownership as well as internet access [67], our study highlights that ePRO collection is widely acceptable and feasible among diverse and underrepresented patient populations. While some studies suggested that face-to-face PRO collection may be preferred by certain populations (e.g., primarily Spanish-speaking individuals [43, 44]), another study conducted in a similar population demonstrated preference for computerized data collection [45]. Moreover, these studies were published over 20 years ago and may not be representative of populations today that have had more time to adjust to new technologies. This is supported by recent research showing that reliance on smartphones for internet access has become increasingly more common among Americans with lower socioeconomic status as well as racial and ethnic minority populations [67]. However, relying on ePROs alone may not be sufficient to maintain equitable PRO collection, as evidenced by one report demonstrating profound racial and ethnic disparities when transitioning from tablet-based to web-based PRO collection [68]. Our study highlights several key findings regarding the unique needs and concerns of populations in various clinical settings using ePROs, such as inclusion of bar graphs with “emojis” for symptom reports [41] or touchscreens with visual and audio components, for example [48]. This not only emphasizes the need for further research in these populations, but also suggests that programs would likely benefit from specifically tailoring ePROs to the populations they serve.

Notably, the least studied implementation outcomes in studies to date were penetration and sustainability. While it is still helpful to understand elements of PRO implementation such as acceptability and feasibility in the short-term, it is evident that less is known regarding longer-term and institution-wide outcomes of these interventions. This is problematic, given that PRO implementation programs can require additional staffing and data resources [8, 10], which may be prohibitive for low-resourced settings where many diverse and underrepresented populations receive care, especially outside of short-term, intensive study settings. As evidenced by the 5-year study period in the one study in this review with sustainability and penetration outcomes [40], another potential barrier to studying these outcomes is the time needed for longer, prospective studies. Bolstering partnerships between higher-resourced academic centers with existing PRO programs and lower-resourced settings may therefore provide one solution.

Moreover, cost was another infrequently assessed outcome. Consequently, cost-effectiveness data of PRO implementation was largely missing. Alongside staffing and data resource concerns, the cost of PRO implementation may preclude implementation in resource-constrained settings where many diverse and underrepresented populations receive care. While one study reported on the ability to maintain long-term costs [40], the financial requirements and impacts of PRO implementation for diverse and underrepresented populations remains largely to be determined. Nonetheless, the reduction of healthcare disparities itself has been estimated to substantially reduce healthcare spending [69, 70]. With the aim of reducing disparities through these implementation programs, cost data will also be helpful in further characterizing the cost-effectiveness and healthcare value of routine PRO collection.

While the studies within this analysis elucidated important findings within underrepresented patient populations, it is important to note that more work is needed to elucidate which specific determinants of health within populations drive certain outcomes as well as the way in which identities and social determinants intersect [71]. Most studies, for example, reported racial and ethnic demographics of their study sample, however much fewer reported income, employment, and insurance status. This is problematic, given that racial or ethnic minority outcomes may be confounded by other unreported social factors, such as low income or unemployment. In addition, results and data analysis were often stratified by race or ethnic minority status alone, without delving deeper into the specific social determinants of health (i.e. transportation, perceived and actual racism, work environments). As such, future work evaluating implementation within these populations should more deeply investigate how social determinants of health drive disparate acceptability, adoption, or feasibility outcomes, for example.

Our study had some limitations. First, studies were limited to the United States in order to increase generalizability of results, given that diverse and underrepresented populations are defined in part by the history, demographics, culture, and economy of their country of origin. Although international studies such as those investigating routine PRO collection in rural Australia [72], use of robots for routine PRO collection among elderly adults in the Netherlands [73], and ePRO collection among a diverse, urban clinic population living with HIV in Canada [74] may have relevant and generalizable findings for clinical environments in the United States, we deemed these beyond the scope of this review. However, the narrower scope of this review allowed for greater specificity of findings for certain populations, such as Black Americans and non-English-speaking populations, within the unique context of PRO programs in the United States. Second, we only included studies that had a specific and explicit aim of studying PRO implementation in a diverse and/or underrepresented population. There is a possibility that studies investigating clinical PRO implementation more broadly had potentially relevant incidental findings for these populations, however this review importantly highlights studies that fill critical gaps in the literature by intentionally aiming to study these populations. In addition, this review studied diverse populations known to be affected by healthcare disparities in the United States. Third, our study focused on the implementation of already developed and validated PROMs. Addressing disparities in PRO data collection will also require equitable PROM development, testing, and validation within diverse and underrepresented populations before implementation occurs [15].

Conclusions

As existing healthcare systems expand and new systems develop PRO data collection programs, it will be imperative to ensure that PRO data collection is not only representative of all patients but also equitable in its implementation in routine clinical care [15]. While this study highlights several important considerations for PRO implementation in diverse and underrepresented populations, it simultaneously calls attention to the paucity of research in this area to date. Future studies of PRO implementation will be needed across the healthcare spectrum in order to address existing disparities and promote health equity alike.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

PRO:

Patient-reported outcome

PROM:

Patient-reported outcome measure

CINAHL:

Cumulative Index to Nursing and Allied Health Literature

ePRO:

Electronic patient-reported outcome

PRISMA-ScR:

Preferred Reporting Items for Systematic Reviews and Meta-analyses for Scoping Reviews

References

  1. Marshall S, Haywood K, Fitzpatrick R (2006) Impact of patient-reported outcome measures on routine practice: a structured review. J Eval Clin Pract 12:559–568

    PubMed  Google Scholar 

  2. Lavallee DC et al (2016) Incorporating patient-reported outcomes into health care to engage patients and enhance care. Health Aff (Millwood) 35:575–582

    Google Scholar 

  3. Refolo P, Minacori R, Mele V, Sacchini D, Spagnolo AG (2012) Patient-reported outcomes (PROs): the significance of using humanistic measures in clinical trial and clinical practice. Eur Rev Med Pharmacol Sci 16:1319–1323

    CAS  PubMed  Google Scholar 

  4. Boyce MB, Browne JP (2013) Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res 22:2265–2278

    PubMed  Google Scholar 

  5. Kotronoulas G et al (2014) What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 32:1480–1501

    PubMed  Google Scholar 

  6. Sisodia RC et al (2020) Patient reported outcomes measures in gynecologic oncology: a primer for clinical use, part I. Gynecol Oncol 158:194–200

    PubMed  PubMed Central  Google Scholar 

  7. Squitieri L, Bozic KJ, Pusic AL (2017) The role of patient-reported outcome measures in value-based payment reform. Value Health J Int 20:834–836

    Google Scholar 

  8. Snyder CF et al (2012) Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 21:1305–1314

    PubMed  Google Scholar 

  9. Zhang B, Lloyd W, Jahanzeb M, Hassett MJ (2018) Use of patient-reported outcome measures in quality oncology practice initiative-registered practices: results of a national survey. J Oncol Pract 14:e602–e611

    PubMed  Google Scholar 

  10. Philpot LM et al (2018) Barriers and benefits to the use of patient-reported outcome measures in routine clinical care: a qualitative study. Am J Med Qual 33:359–364

    PubMed  Google Scholar 

  11. Scholle SH et al (2018) Implementation of the PROMIS-29 in routine care for people with diabetes: challenges and opportunities. J Ambul Care Manag 41:274–287

    Google Scholar 

  12. Oh SS et al (2015) Diversity in clinical and biomedical research: a promise yet to be fulfilled. PLoS Med 12:e1001918

    PubMed  PubMed Central  Google Scholar 

  13. Basch E (2017) Patient-reported outcomes - harnessing patients’ voices to improve clinical care. N Engl J Med 376:105–108

    PubMed  Google Scholar 

  14. Paul RW et al (2021) Reporting and analyzing race and ethnicity in orthopaedic clinical trials: a systematic review. J Am Acad Orthop Surg Glob Res Rev 5:e21.00027

    PubMed Central  Google Scholar 

  15. Ortega G et al (2021) Prioritizing health equity in patient-reported outcome measurement to improve surgical care. Ann Surg. https://doi.org/10.1097/SLA.0000000000005169

    Article  PubMed  Google Scholar 

  16. van Egdom LSE et al (2019) Implementing patient-reported outcome measures in clinical breast cancer care: a systematic review. Value Health J 22:1197–1226

    Google Scholar 

  17. Dusek JA et al (2018) Rationale for routine collection of patient reported outcomes during integrative medicine consultation visits. Complement Ther Med 37:43–49

    CAS  PubMed  Google Scholar 

  18. Berlin NL et al (2018) Nonresponse bias in survey research: lessons from a prospective study of breast reconstruction. J Surg Res 224:112–120

    PubMed  Google Scholar 

  19. Garcia SF et al (2019) Implementing electronic health record-integrated screening of patient-reported symptoms and supportive care needs in a comprehensive cancer center. Cancer 125:4059–4068

    PubMed  Google Scholar 

  20. Tosi LL et al (2019) Assessing disease experience across the life span for individuals with osteogenesis imperfecta: challenges and opportunities for patient-reported outcomes (PROs) measurement: a pilot study. Orphanet J Rare Dis 14:23

    PubMed  PubMed Central  Google Scholar 

  21. Shaw BE et al (2017) Centralized patient-reported outcome data collection in transplantation is feasible and clinically meaningful. Cancer 123:4687–4700

    CAS  PubMed  Google Scholar 

  22. Rubenstein WJ, Harris AHS, Hwang KM, Giori NJ, Kuo AC (2020) Social determinants of health and patient-reported outcomes following total hip and knee arthroplasty in veterans. J Arthroplasty 35:2357–2362

    PubMed  Google Scholar 

  23. Ilowite MF et al (2018) The relationship between household income and patient-reported symptom distress and quality of life in children with advanced cancer: a report from the PediQUEST study. Cancer 124:3934–3941

    PubMed  Google Scholar 

  24. Braveman P (2014) What are health disparities and health equity? We need to be clear. Public Health Rep 129(Suppl):5–8

    PubMed  PubMed Central  Google Scholar 

  25. Munn Z et al (2018) Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol 18:143

    PubMed  PubMed Central  Google Scholar 

  26. Aromataris E, Munn Z (2020) JBI manual for evidence synthesis. Adelaide: JBI. https://synthesismanual.jbi.global. Accessed 9 Aug 2021

  27. Howell D et al (2015) Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol 26:1846–1858

    CAS  PubMed  Google Scholar 

  28. Sokas C et al (2021) A review of PROM implementation in surgical practice. Ann Surg. https://doi.org/10.1097/SLA.0000000000005029

    Article  PubMed  Google Scholar 

  29. U.S. Department of Health and Human Services (2021) Disparities HealthyPeople.gov

  30. Feit NZ et al (2022) Healthcare disparities in laryngology: a scoping review. Laryngoscope 132:375–390

    PubMed  Google Scholar 

  31. Tricco AC et al (2018) PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169:467–473

    PubMed  PubMed Central  Google Scholar 

  32. Proctor E et al (2011) Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 38:65–76

    PubMed  Google Scholar 

  33. Hermes EDA, Lyon AR, Schueller SM, Glass JE (2019) Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes. J Med Internet Res 21:e11752

    PubMed  PubMed Central  Google Scholar 

  34. University of Michigan Library (2021) Melnyk levels of evidence. University of Michigan. https://guides.lib.umich.edu/c.php?g=282802&p=1888246. Accessed 11 Oct 2021

  35. Melnyk BM, Fineout-Overholt E (2015) Evidence-based practice in nursing & healthcare: a guide to best practice, 3rd edn. Wolters Kluwer Health, Philadelphia

    Google Scholar 

  36. Arcia A, George M, Lor M, Mangal S, Bruzzese J-M (2019) Design and comprehension testing of tailored asthma control infographics for adults with persistent asthma. Appl Clin Inform 10:643–654

    PubMed  PubMed Central  Google Scholar 

  37. Lapin BR et al (2019) Association between patient experience with patient-reported outcome measurements and overall satisfaction with care in neurology. Value Health J 22:555–563

    Google Scholar 

  38. Hinami K et al (2015) Health perceptions and symptom burden in primary care: measuring health using audio computer-assisted self-interviews. Qual Life Res 24:1575–1583

    PubMed  Google Scholar 

  39. Jiwani R et al (2020) Changes in patient-reported outcome measures with a technology-supported behavioral lifestyle intervention among patients with type 2 diabetes: pilot randomized controlled clinical trial. JMIR Diabetes 5:e19268

    PubMed  PubMed Central  Google Scholar 

  40. Loo S et al (2020) Capturing relevant patient data in clinical encounters through integration of an electronic patient-reported outcome system into routine primary care in a Boston community health center: development and implementation study. J Med Internet Res 22:e16778

    PubMed  PubMed Central  Google Scholar 

  41. Stonbraker S, Porras T, Schnall R (2020) Patient preferences for visualization of longitudinal patient-reported outcomes data. J Am Med Inform Assoc 27:212–224

    PubMed  Google Scholar 

  42. Sarkar U et al (2016) Usability of commercially available mobile applications for diverse patients. J Gen Intern Med 31:1417–1426

    PubMed  PubMed Central  Google Scholar 

  43. González GM, Costello CR, Valenzuela M, Chaidez B, Nuñez-alvarez A (1995) Bilingual computerized speech-recognition screening for clinical depression: evaluating a cellular telephone prototype. Behav Res Methods Instrum Comput 27:476–482

    Google Scholar 

  44. González GM et al (2000) A bilingual telephone-enabled speech recognition application for screening depression symptoms. Prof Psychol Res Pract 31:398–403

    Google Scholar 

  45. Muñoz RF, González GM, Starkweather J (1995) Automated screening for depression: toward culturally and linguistically appropriate uses of computerized speech recognition. Hisp J Behav Sci 17:194–208

    Google Scholar 

  46. Anderson JN et al (2021) THRIVE intervention development: using participatory action research principles to guide a mHealth app-based intervention to improve oncology care. J Hosp Manag Health Policy 5:5

    PubMed  PubMed Central  Google Scholar 

  47. Anderson KO et al (2015) Automated pain intervention for underserved minority women with breast cancer. Cancer 121:1882–1890

    PubMed  Google Scholar 

  48. Hahn EA et al (2004) The talking touchscreen: a new approach to outcomes assessment in low literacy. Psychooncology 13:86–95

    PubMed  Google Scholar 

  49. Samuel CA et al (2021) Racial differences in user experiences and perceived value of electronic symptom monitoring in a cohort of black and white bladder and prostate cancer patients. Qual Life Res 30:3213–3227

    PubMed  Google Scholar 

  50. Smith AB et al (2021) Feasibility and delivery of patient-reported outcomes in clinical practice among racially diverse bladder and prostate cancer patients. Urol Oncol 39(77):e1-77.e8

    Google Scholar 

  51. Zullig LL et al (2019) Using a geriatric oncology assessment to link with services (GOAL). J Geriatr Oncol 10:164–168

    PubMed  Google Scholar 

  52. Kasturi S et al (2018) Feasibility, validity, and reliability of the 10-item patient reported outcomes measurement information system global health short form in outpatients with systemic lupus erythematosus. J Rheumatol 45:397–404

    PubMed  PubMed Central  Google Scholar 

  53. Liu LH et al (2020) Patient and clinician perspectives on a patient-facing dashboard that visualizes patient reported outcomes in rheumatoid arthritis. Health Expect 23:846–859

    PubMed  PubMed Central  Google Scholar 

  54. Wahl E et al (2017) Validity and responsiveness of a 10-item patient-reported measure of physical function in a rheumatoid arthritis clinic population. Arthritis Care Res (Hoboken) 69:338–346

    Google Scholar 

  55. Hirsh J et al (2019) Limited health literacy and patient confusion about rheumatoid arthritis patient global assessments and model disease states. Arthritis Care Res (Hoboken) 71:611–619

    Google Scholar 

  56. Calamia M et al (2021) Feasibility and validation of a web-based platform for the self-administered patient collection of demographics, health status, anxiety, depression, and cognition in community dwelling elderly. PLoS ONE 16:e0244962

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Ramsey AT, Wetherell JL, Depp C, Dixon D, Lenze E (2016) Feasibility and acceptability of smartphone assessment in older adults with cognitive and emotional difficulties. J Technol Hum Serv 34:209–223

    PubMed  PubMed Central  Google Scholar 

  58. Wolford G et al (2008) A clinical trial comparing interviewer and computer-assisted assessment among clients with severe mental illness. Psychiatr Serv 59:769–775

    PubMed  Google Scholar 

  59. Gabbard J et al (2021) Rapid electronic capturing of patient-reported outcome measures in older adults with end-stage renal disease: a feasibility study. Am J Hosp Palliat Care 38:432–440

    PubMed  Google Scholar 

  60. Jacoby SF, Robinson AJ, Webster JL, Morrison CN, Richmond TS (2021) The feasibility and acceptability of mobile health monitoring for real-time assessment of traumatic injury outcomes. mHealth 7:5

    PubMed  PubMed Central  Google Scholar 

  61. Nyirenda J, Sun C, Wright S, Rezek G (2019) Feasibility testing of health information technology: enabled patient-reported outcome measurement in the home health setting. Home Healthc Now 37:222–226

    PubMed  Google Scholar 

  62. Shipp MM, Thakkar MY, Sanghavi KK, Means KRJ, Giladi AM (2021) Disparities limit the effect and benefit of a web-based clinic intake system. Orthopedics 44:e434–e439

    PubMed  Google Scholar 

  63. Wood KL, Haider SF, Bui A, Leitman IM (2020) Access to common laparoscopic general surgical procedures: do racial disparities exist? Surg Endosc 34:1376–1386

    PubMed  Google Scholar 

  64. Restrepo DJ et al (2020) Disparities in access to autologous breast reconstruction. Medicina (Kaunas) 56:281

    Google Scholar 

  65. Howell EA, Zeitlin J (2017) Quality of care and disparities in obstetrics. Obstet Gynecol Clin N Am 44:13–25

    Google Scholar 

  66. Federico MJ, McFarlane AE 2nd, Szefler SJ, Abrams EM (2020) The impact of social determinants of health on children with asthma. J Allergy Clin Immunol Pract 8:1808–1814

    PubMed  Google Scholar 

  67. Pew Research Center (2021) Mobile fact sheet. Pew Research Center

  68. Sisodia RC, Rodriguez JA, Sequist TD (2021) Digital disparities: lessons learned from a patient reported outcomes program during the COVID-19 pandemic. J Am Med Inform Assoc 28:2265–2268

    PubMed  PubMed Central  Google Scholar 

  69. LaVeist TA, Gaskin D, Richard P (2011) Estimating the economic burden of racial health inequalities in the United States. Int J Health Serv 41:231–238

    PubMed  Google Scholar 

  70. Nanney MS et al (2019) The economic benefits of reducing racial disparities in health: the case of minnesota. Int J Environ Res Public Health 16:742

    PubMed Central  Google Scholar 

  71. Braveman P, Gottlieb L (2014) The social determinants of health: it’s time to consider the causes of the causes. Public Health Rep 129(Suppl):19–31

    PubMed  PubMed Central  Google Scholar 

  72. Thewes B, Butow P, Stuart-Harris R (2009) Does routine psychological screening of newly diagnosed rural cancer patients lead to better patient outcomes? Results of a pilot study. Aust J Rural Health 17:298–304

    PubMed  Google Scholar 

  73. Boumans R, van Meulen F, Hindriks K, Neerincx M, Olde Rikkert MGM (2019) Robot for health data acquisition among older adults: a pilot randomised controlled cross-over trial. BMJ Qual Saf 28:793–799

    PubMed  PubMed Central  Google Scholar 

  74. Suri S et al (2021) Feasibility of implementing a same-day electronic screening tool for clinical assessment to measure patient-reported outcomes for eliciting actionable information on adherence to HIV medication and related factors in a busy Canadian urban HIV clinic. Int J STD AIDS. https://doi.org/10.1177/09564624211032796

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

C.J.H is supported through the Harvard Medical School Office of Scholarly Engagement and the Patient-Reported Outcome, Values, and Experience Center at Brigham and Women’s Hospital. M.N.K. is supported through the Canadian Institutes of Health Research Fellowship Award (2020-23).

Author information

Authors and Affiliations

  1. Harvard Medical School, Brigham and Women’s Hospital, Boston, MA, USA

    Colby J. Hyland, Ruby Guo, Manraj N. Kaur, Maria O. Edelen & Andrea L. Pusic

  2. Harvard School of Public Health, Brigham and Women’s Hospital, Boston, MA, USA

    Ravi Dhawan

  3. Harvard Medical School, Countway Library, Boston, MA, USA

    Paul A. Bain

Authors
  1. Colby J. Hyland
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruby Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ravi Dhawan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Manraj N. Kaur
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Paul A. Bain
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maria O. Edelen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrea L. Pusic
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors listed contributed substantially to the manuscript. CJH was involved in manuscript conception, literature review, data analysis, and manuscript writing. RG and RD were involved in literature review and data analysis. MNK was involved in manuscript writing and revision. PAB was involved in literature search methodology. MOE and ALP were involved in manuscript conception, writing, and revision. All authors approved the final version of the manuscript for submission.

Corresponding author

Correspondence to Colby J. Hyland.

Ethics declarations

Ethics approval and consent to participate

As this was a scoping review article, ethics approval and consent to participate are not applicable.

Consent for publication

Not applicable.

Competing interests

A.L.P is a co-developer of the QPROMS which are owned by Memorial Sloan-Kettering Cancer Center and receives a portion of licensing fees (royalty payments) when the QPROMS are used in industry sponsored clinical trials.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Table S1

. Search strategy by databases included in the review.

Additional file 2: Table S2

. Details of the specific implementation outcomes by studies included in the review.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hyland, C.J., Guo, R., Dhawan, R. et al. Implementing patient-reported outcomes in routine clinical care for diverse and underrepresented patients in the United States. J Patient Rep Outcomes 6, 20 (2022). https://doi.org/10.1186/s41687-022-00428-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-022-00428-z

Keywords

  • Patient-reported outcome
  • Patient-reported outcome measure
  • PRO
  • PROM
  • Clinical care
  • Implementation
  • Diverse
  • Underrepresented patient population