Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature

Milena Anatchkova; Sarah M. Donelson; Anne M. Skalicky; Colleen A. McHorney; Dayo Jagun; Jennifer Whiteley

doi:10.1186/s41687-018-0091-0

Review
Open Access

Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature

Milena Anatchkova¹Email author,
Sarah M. Donelson³,
Anne M. Skalicky¹,
Colleen A. McHorney¹,
Dayo Jagun⁴ and
Jennifer Whiteley²

Journal of Patient-Reported Outcomes20182:64

https://doi.org/10.1186/s41687-018-0091-0

Received: 30 March 2018
Accepted: 4 December 2018
Published: 27 December 2018

Abstract

Background

To explore the existing evidence of the real-world implementation of patient-reported outcomes (PROs) in oncology clinical practice and address two aims: (1) summarize available evidence of PRO use in clinical practice using a framework based on the International Society for Quality of Life Research (ISOQOL) PRO Implementation Guide; and (2) describe reports of real-world, standardized PRO administration in oncology conducted outside of scope of a research study.

Methods

A Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol was developed to guide the systematic literature review (SLR) that was conducted in MEDLINE and Embase databases. A two step search strategy was implemented including two searches based on previously completed reviews. Studies published from 2006 to 2017 were synthesized using a framework based on the ISOQOL PRO Implementation Guide.

Results

After screening 4427 abstracts, 36 studies met the eligibility criteria. Most elements of the ISOQOL PRO Implementation Guide were followed. Two notable exceptions were found: 1) providing PRO score interpretation guidelines (39% of studies); and 2) providing patient-management guidance for addressing issues identified by PROs (25% of studies). Of the 22 studies with an intervention component, 19 (86%) reported intervention effects on study outcomes. The European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire-Core 30 (EORTC QLQ-C30) was the most commonly used PRO (n = 10, 28%); use of 38 other PRO measures was also reported. Only three studies (8%) reported real-world PRO implementation.

Conclusion

Reports of real-world PRO implementation are limited. Reports from studies conducted in clinical settings suggest gaps in information on PRO score interpretation and the use of PRO results to inform patient management. Before the promise of practice-based PRO assessment in oncology can be truly realized, investigators need to advance the state-of-the-art of real-time PRO score interpretation as well as developing guidance on how to use PRO insights to drive clinically-meaningful patient-management strategies.

Keywords

Patient-reported outcomes
Clinical practice
Quality of care
PRO implementation
Patient-centered care
Cancer
Oncology

Introduction

There has been growing interest in the assessment of patient-reported outcomes (PROs) over the last 40 years, and the use of PROs in clinical and health services research is common [1]. PROs have been defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else.” [2] The incorporation of PROs in clinical practice can serve numerous purposes [3] including: (1) describing a patient’s overall state; (2) screening for incipient disease and undetected disability [4, 5]; (3) monitoring disease progression and response to treatment; (4) assessing patient-centered needs; (5) formulating treatment plans consistent with patient preferences [6–8]; (6) improving physician-patient communication [9–11]; (7) providing patient-based data for quality initiatives [12–14]; and (8) standardizing interactions between healthcare providers and patients [3]. While the use of PROs in clinical practice can help in all of these areas [1, 15], critical questions remain about how patient outcome data should be collected, shared, and used to improve the quality of care and patient health outcomes [16]. Some reports have emerged regarding the use of PROs in routine clinical practice across different conditions [1, 17–22], but the incorporation of these tools in oncology clinical practice has been slower than adoption in research [23–26].

More recently, routine use of PROs in oncology practice has been identified as a priority area by the President’s Cancer Panel [27] as well as by national oncology societies such as the American Society of Clinical Oncology (ASCO) [28]. There is increasing interest in bringing “the patient’s perspective” to cancer decision making which is demonstrated by a number of key initiatives of PRO application in oncology research and regulatory decisions [29]. However, little evidence has been generated with regards to clinical-practice implementation [30].

The interest in implementation of PROs specifically in oncology care is exemplified by the number of recent reviews on PRO clinical applications and their impact on health outcomes [31–34]. All of the recent oncology reviews provide some insights on gaps in existing evidence of PRO use in clinical practice related to both challenges in implementation and PRO use impact. For example, Howell identified that more attention needs to be paid to complexity of implementation and interpretation [32]. King and colleagues [33] found a scarcity of studies reporting data on actions and medical decisions [33]. Two others—Chen [31] and Kotronoulas [34]—examined PRO intervention evidence and identified weak signals specific to changes in patient management and improved health outcomes [31, 34]. While these reviews identified important evidence gaps, none of them used an existing implementation framework to organize findings or focused on a review of publications reporting on the actual implementation of PRO in real-world settings beyond the context of a feasibility study or intervention trial. The current review makes a unique contribution to the field, by summarizing currently existing evidence using an implementation framework based on the user’s guide for the implementation of PROs in clinical practice recently developed by the International Society for Quality of Life Research (ISOQOL) [35]. The guide includes recommendations for the following implementation elements: (1) identifying the goals for collecting PROs in clinical practice and which key patient outcomes or barriers need attention; (2) considering group of patients and the care settings; (3) determining which questionnaire(s) to use (e.g., whether to use generic or disease-specific questionnaires, profile or preference-based measures, single or multi-item scales, and static or dynamic questionnaires); (4) choosing how often a patient should complete the questionnaires and whether it should be one-time completion or repeated, tied to clinic visits, or a way to monitor patients between visits; (5) deciding how the PRO will be administered and scored; (6) identifying interpretation benchmarks for the PRO score and how scores requiring follow-up will be determined; (7) developing strategies for when the PRO results will be presented and discussed with the patient (such as during or after the visit), how the results will be presented (e.g., numeric, graphical, one-time results or trends over time), and who will see the PRO score reports; (8) determining what will be done to respond to issues identified by the PROs and follow-up; and (9) evaluating the impact and value of the PRO interventions on the practice and patient [35]. While previous publications have discussed various considerations and potential applications of PROs in clinical practice [36–38], the ISOQOL PRO Implementation Guide is most recent and provides specific implementation guidance developed by subject matter experts and endorsed by a professional organization.

The objective of this systematic literature review (SLR) was to explore and summarize the existing evidence of PRO use in oncology clinical practice. We address two key aims in the review: (1) summarize available evidence of PRO use in clinical practice using a framework based on the ISOQOL PRO Implementation Guide [35]; (2) describe reports of real-world implementation of PRO measures with oncology patients. Real-world implementation of PROs can provide evidence regarding the usage and potential benefits of PRO adoption derived from real-world clinical settings. For the purposes of this review, real-world implementation studies were defined as those reporting the process of ongoing standardized PRO administration and related clinical actions to manage patient care conducted in a routine clinical practice beyond the scope of a specific research study.

Methods

Study design

We conducted a SLR in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [39, 40]. The protocol was developed following PRISMA guidelines and the Guidance on the Conduct of Narrative Synthesis in Systematic Reviews [41].

Data sources

The literature search was conducted in two databases: MEDLINE and Embase. Blocks of medical subject heading (MeSH) terms were used to identify the most relevant articles and conference papers that describe PRO implementation in oncology clinical practice.

Search strategy

The SLR search strategy was developed in consultation with a professional librarian and used a two-step approach for identifying studies. The first step included a search strategy and approach based on a previously published systematic review of use of PROs in oncology care [31]. As part of the first step, two additional, closely-related systematic reviews were identified [32, 33]. References from these oncology literature reviews were examined, and studies meeting the selection criteria were incorporated into the review [32, 33]. As time has elapsed since the publication of these reviews, they had a narrower focus and used different search strategies and search terms, a second search was conducted to replicate and update the earlier Howell [32] and King [33] reviews to further ensure a comprehensive review of recent publications from the end date of published reviews. MeSH terms and free-text keyword groups (e.g., “neoplasm,” “PRO measure,” “clinical practice,” and “treatment”) were used in different combinations. These updated searches also included specific PROs as search terms, minimizing the risk of missing relevant articles that used these measure, but may have resulted in overrepresentation of these specific measures in the final results. Terminology adjustments were made according to the requirements of each database. Both searches were supplemented by a hand search of references of relevant articles. Appendix A shows the full search strategy.

Selection criteria

Articles were included in the review if they met the following inclusion criteria: (1) cancer focus; (2) articles published in the past 10 years from 2006 to 2016 (inclusive) and abstracts from meetings held in 2015–2016 to ensure review of the current state of the field; (3) published in English; (4) title, abstract, or article contained information pertaining to the measurement of treatment satisfaction, process of care, treatment adherence, treatment decision-making, patient activation, PROs of health-related quality of life (HRQoL), symptoms, or function; and (5) the study design was a randomized controlled trial or an observational study in a clinical-practice setting or a report of PRO implementation in clinical practice.

Exclusion criteria were: (1) articles focused on non-cancer populations; (2) measures did not pertain to clinical outcomes or PROs associated with cancer treatment; (3) basic science studies (e.g., molecular biomarkers, neuroimaging drug formulation); (4) study designs not relevant including study protocols, case studies, case reports, case series, editorials, reviews, commentary, news, or study protocols; and (5) non-English language.

Data screening and abstraction

All abstracts were reviewed using DistillerSR® [42]—a systematic literature review reporting software—to assist with the organization, extraction, and categorization of all literature. Abstract and article screening was performed by three trained reviewers in a two-step process. During the Level 1 review, in order to standardize the review process, a calibration exercise was conducted by the reviewers for all abstracts and titles to assess eligibility for inclusion in the full-text review. Full-text articles that met the inclusion criteria were retrieved. If a determination of eligibility was not possible from the abstract, the full-text article was reviewed. During Level 2, full-text articles were reviewed again for eligibility. Disagreements on eligibility of screened publications at both levels were resolved through discussion with reviewers and final adjudication of unresolved disagreements by the first author of this paper (M.A). For eligible articles, the data was abstracted into a detailed source table that included data fields on study country, study type, cancer type, study objectives, sample size, study duration, study inclusion/exclusion criteria, PRO intervention characteristics, PRO reporting characteristics, study endpoints, assessment timepoints, PRO study results, and limitations/contextualization. The data abstracted into the detailed source table was validated by a second independent senior reviewer to ensure the accuracy of data abstraction. The detailed source table was used to organize information in summary tables that were developed during data analysis and based on the ISOQOL PRO Implementation Guide Framework.

Data analysis

We used the ISOQOL PRO Implementation Guide [35] as the basis for developing a framework to address our first research aim (Table 1). Categories corresponding to each of the ISOQOL PRO Implementation Guide recommendations were created, and information from the articles was extracted into summary tables from the original detailed literature source table. This was done to explore relationships in the data and to establish if all recommended information was included in reports of PRO use in oncology clinical practice. The framework was used to explore the use of PROs in clinical care and their relationship to outcomes in the context of the ISOQOL PRO Implementation Guide. To inform the second aim of this review—to describe reports of real-world implementation of PRO measures settings with oncology patients—we examined the characteristics of all real-world implementation reports.

Table 1

ISOQOL PRO Implementation Guide Framework for SLR Data Synthesis

	Data Synthesis Categories	Data Codes
PRO Study Design Recommendations	Goals for Collecting PRO	Screening, Monitoring, Patient Centered Care, Decision Aid, Team communication, Quality of care
	Patients, Setting, Timing of Assessment	Patients: Type of Cancer, Adults vs. Children
		Setting: Clinic, Home, Hospital, Hospice
		Timing of assessment: Before Visit, During Visit, After Visit
	Selection of PROs	Types of PROs Used: Symptoms, Function, Disease-Specific Quality of Life, Generic Quality of Life, Other
	PRO Mode of administration	Paper and Pencil /Phone/ IVR/ePRO (Tablet, Web, Phone)
PRO Study Results Recommendations	Reporting of PRO Results	Where? Clinical Flow vs. Other
		How? Numbers, Graphs, Full Report
		Who? Clinical Team, Patient, Both
	Score Interpretation	Written guidelines, Cut Scores, Minimally Important Difference, Normative Scores
	Plans for Addressing Issues Identified by PRO	Plans in Place/No Plans
	Evaluation of PRO Impact on Clinical Practice	Research Designs Used (RCT, Quasi RCT, Survey)
	Evaluation of PRO Impact on Clinical Practice	Types of Outcomes Considered: Outcomes with Evidence of Impact

Abbreviations: ePRO = electronic patient-reported outcome, IVR = Interactive Voice Response, PRO = patient-reported outcome, RCT = randomized controlled trial

Results

As shown in the PRISMA diagram (Fig. 1), a total of 5754 records were identified, and they yielded 4427 unique publications (following removal of duplicates). A total of 117 full-text articles were retrieved for a second round of screening. Of these, 36 articles met the inclusion criteria and are presented herein (Table 2). The largest number of studies were from the United States (US) (n = 16, 44%) followed by the United Kingdom (UK) (n = 6, 17%), Germany (n = 3, 8%), Canada (n = 2, 6%), The Netherlands (n = 2, 6%), other individual European Union (EU) countries (n = 6, 17%), and Australia (n = 1, 3%).

Table 2

Summary Table with Key Elements of Included Studies

Citation (author, year, title)	Patient/ Type of Cancer	Research Design	Study Goal	PRO Used	Member of medical team receiving feedback	PRO score interpretation	Plans for addressing issues identified by PRO	Study Results
Álvarez-Maestroa, M. 2014 [60]	Adults with metastatic prostate cancer	Cohort study	Feasibility	PROSQoLI, Perceived health status	Physician	Score information provided	No instructions given, only usual best practice guidelines	• % Physicians finding useful:
								o 66.1% clinical decision making
								o 71.3% questionnaire characteristics
								o 73.4% doctor---patient communication.
Basch, E. 2016 [61]	Adults with any cancer type	RCT	Intervention	EQ-5D and CTCAE	Oncologist, nurse	Score information provided	No instructions given, only usual best practice guidelines	• EQ-5D (p < .001)
								• ER Visits (p < .02)
								• 1 year survival (p = .05)¹
								• # of Nursing calls (p = .93)
Basch, E. et al. 2007 [62]	Adults with any cancer type	Cohort study	Feasibility/ Intervention	EQ-5D and CTCAE	Clinicians	Score information provided	Standard AE reporting procedures	• 85% of participants logged in during clinical visits
								• 66% logged in from home
								• 57 Grade 3 or 4 Toxicity reported
Berry, D. 2014 [63]	Adults with any cancer type	Cluster RCT	Intervention	ESRA-C and SDS	Clinicians	Score information provided	Questions for patient to ask clinician	• ESRA Intervention group had a significant reduction (p < .05)
Berry, D. 2015 [64]	Adults with any cancer type	RCT	Intervention	ESRA-C and SDS	Patient	Graphical data display	Questions for patient to ask clinician	• Significant reductions of symptom distress in ESRA-C intervention group (p < .01)
Berry, L., et al., 2011 [65]	Adults with any cancer type	RCT	Intervention	ESRA-C and SDS	Clinicians	Score information provided	No instructions given, only usual best practice guidelines	• Significant interaction effect for reporting of symptom/QoL issues (p = .032).
								• More discussion in intervention group if SQoLs reported as problems.
								• 50–60% of clinicians found PRO report useful
								• No difference in study length
Blum, D. 2014 [66]	Adults with advanced cancer	Cohort study	Feasibility	E-MOSAIC	Physician	Graphical data display	Discussion of PRO results	• % Clinicians agreeing:
								• 77% Useful monitoring system
								• 38–95% Better symptom control
Boyes, A., et al. 2006 [67]	Adults with any cancer type	RCT	Intervention	Symptoms, HADS, Care needs	Oncologist	Score information provided	No instructions given, only usual best practice guidelines	• No significant differences between groups in HADS outcomes
Boyes, A., et al. 2006 [67]	Adults with any cancer type	RCT	Intervention	Symptoms, HADS, Care needs	Oncologist	Score information provided	No instructions given, only usual best practice guidelines	• Majority of patients founds the survey easy to complete, good way to communicate with doctors, willing to complete again.
Chiang., A. 2015 [68]	Adults with any cancer type	Cohort study	Intervention for quality Improvement	NCCN-EDT	Clinical team	None	Referral to social worker	• EMR documentation increased from 19% to 34% over 6mo before and after intervention
								• Barriers:
								• Insufficient time with patients, lack of social work resources, lack of privacy and space to discuss, and patient discomfort in discussing.
Compaci, G., 2015 [69]	Adults with lymphoma	Cohort study	Feasibility	HADS, PTSD-CL, SF-36, eCRF	Oncologist, nurse or GP	Score information provided	Phone follow-up, clinic consultations	• Time for whole intervention 55 min per quarter
								• Anxiety decrease 20% to 14% baseline to 12 months
								• Depression decrease 10% to 6.5% baseline to 12 months
								• PTSD 14.8% to 17.6%
Cox, A., et al., 2011 [70]	Adults with lung cancer	Qualitative interviews and Cross-sectional study	Feasibility	ESAS and EQ-5D	Clinicians	Score information provided	Questions for patient to ask clinician	• Clinicians found PRO beneficial, but only considered them complementary.
Engelen, V., et al., 2012 [71]	Children with any cancer type	Cohort study	Intervention	QLIC-ON, PEDSQL, TAPQoL	Oncologist	Not reported	Discussion of PRO results	• HRQoL domains discussed more in intervention group (p < .05)
								• Significantly more emotional and cognitive problems were identified in the intervention group compared to the control group.
								• Better HRQoL outcomes in intervention group for children 5–7 old, but not other age groups.
Epstein, R., 2017 [72]	Adults with any cancer type	Cluster RCT	Intervention	McGill QoL scale single item, McGill Psychological Well-Being subscale, McGill Existential Well-Being subscale, FACT-G Physical Functioning subscale, and FACT-G Social Functioning subscale.	Clinicians	None	Coaching session, follow-up phone calls, list of follow-up questions	• Significant intervention effect for the composite of patient centered communication measure reported (p < .02)
Erharter, A., et al., 2010 [73]	Adults with primary brain tumor	Cohort study	Feasibility	EORTC-QLQ-C30, EORTC-BN20	Physician	Score information provided	No instructions given, only usual best practice guidelines	• Time for completion decreased from 10 to 5 min over the course of the study.
Erharter, A., et al., 2010 [73]	Adults with primary brain tumor	Cohort study	Feasibility	EORTC-QLQ-C30, EORTC-BN20	Physician	Score information provided	No instructions given, only usual best practice guidelines	• Majority of patients and physicians found the PRO acceptable.
Hilarius, D., et al., 2008 [74]	Adults with any cancer type	Cohort study	Intevention	EORTC QLQ-C30, EORTC QLQ-CR38, EORTC QLQ-LC13, EORTC QLQ-BR23	Nurse	Score information provided	No instructions given, only usual best practice guidelines	• HRQoL-related topics discussed significantly more often in the intervention group (p = .009)
								• Awareness of HRQoL issues significantly better in intervention group at visit 4 (p = .05)
								• Significantly more HRQoL chart notations in intervention group (p < .001)
								• No significant differences in patient satisfaction
								• No significant differences in patient HRQoL at visit 4.
								• Evaluation of
								• Intervention: All nurses reported: the summary provided useful information, facilitated communication, resulted in a more efficient use of their time with the patients, a desire to continue using the HRQoL summary profiles in their daily practice.
								• Patients
								o 89% reported the
								HRQoL summary profile provided an accurate picture of their HRQoL, 69% reported it was used explicitly during treatment 89% believed that the summary enhanced their nurses’ awareness of their health problems, and 99% believed that it would be useful to introduce the intervention as a standard part of the outpatient clinic procedure.
Izard, J. 2014 [75]	Adults with prostate cancer	Cohort study	Feasibility	REALM-SD, SNS, Graphic Literacy Scale	Researcher	Graphical data display	Not reported	• Pictograph was the least preferred format
								• Patients favored the bar chart (mean rank, 1.8 [P = .12] vs line graph [P < .01] vs table and pictograph);
								• Providers favored bar (34%), and line (34% and Table (30%) formats.
Kallen, M., et a., 2012 [76]	Adults with any cancer type	Qualitative interviews and Cross-sectional study	Feasibility	ELVIS and ESAS	Providers, patients and caregivers	Graphical data display	Not reported	• Provider interview themes: Improved communication, Barriers to implementation
								• Patient interview themes: improved comprehension, improved communication, and improved patient peace-of-mind
								• Usability: physicians, nurses, patients, and caregivers endorsed the usability of the system (SUS score 83.9)
Mooney, K. 2014 [77]	Adults with any cancer type	RCT	Intervention	Symptoms	Oncologist, nurse, preferred provider	Cut-offs/Thresholds	Structured interview	• There were a total of 6509 calls into the system by 223 patients. The overall daily call adherence was 65.0% of expected days.
Mooney, K. 2014 [77]	Adults with any cancer type	RCT	Intervention	Symptoms	Oncologist, nurse, preferred provider	Cut-offs/Thresholds	Structured interview	• No significant intervention effect
Nicklasson, M., et al., 2013 [78]	Adults with lung cancer or mesothelioma	RCT	Intervention	EORTC QLQ-C30, EORTC LC13	Physician	Interpretation of PRO score reports	No instructions given, only usual best practice guidelines	• Only emotional functioning was more frequently discussed in the intervention group both by doctors (p = 0.018) and by doctors or patients taken together (p = 0.015).
Nicklasson, M., et al., 2013 [78]	Adults with lung cancer or mesothelioma	RCT	Intervention	EORTC QLQ-C30, EORTC LC13	Physician	Interpretation of PRO score reports	No instructions given, only usual best practice guidelines	• More interventions aimed at emotional and social issues and dyspnea in intervention group (p < .05
Rogers, S.N. 2016 [79]	Adults with any cancer type	Cohort study	Intervention	UWQoLv4, PCI	Physician	Not reported	Not reported	• Median number of concerns 3 (range 1–6)
Rogers, S.N. 2016 [79]	Adults with any cancer type	Cohort study	Intervention	UWQoLv4, PCI	Physician	Not reported	Not reported	• Significant association with number of dysfunction scores (p < .005)
Rosenbloom, S., 2007 [80]	Adults with advanced breast, lung or colorectal cancer	RCT	Intervention	FACT-G, FLIC, PSQ-III, POMS-17	Nurse	Score information provided	Structured interview	• No significant differences between groups on study outcomes
Ruland, C., 2010 [81]	Adults with acute myelogenous leukemia (AML), lymphatic leukemia (ALL), multiple myeloma, Hodgkin disease, or non-Hodgkin lymphoma,	RCT	Intervention	SF-36, CES-D, MOS-SS	Patient, clinicians	Patient importance or bothersome ranking	No instructions given, only usual best practice guidelines	• Significantly more symptoms and problems address in intervention group (p < .0001)
								• Positive intervention
								• Effect for discomfort, eating/drinking, sleep/rest, and sexuality.
								• Significant reduction in 10 of 19 symptom distress categories (p < .01)
								• Symptom management: Group differences statistically significant in favor of the intervention group in 13 of 19 (68%) categories.
Schuler, M. 2016 [82]	Adults with any cancer type	Cohort study	Feasibility	PRO-Onc, EQ-5D	Clinic staff	Not reported	Not reported	• Only 4 patients refused to participate
								• 68% staff reported PRO easy to handle
								• 22% used information for care planning during hospital stay; 52% rated additional time as minimal
								• HRQoL scores from admission to discharge (no control group)
Seow, H., et al., 2012 [43]	Adults with lung and Breast Cancer	Retrospective chart reviews	Implementation	ESAS	Oncologist and nurse	Clinically significant severity levels	Not reported	• ESAS score associated with increase in both documentation and actions for pain and shortness of breath
Siekkinen, M. 2015 [83]	Adults with breast cancer	Single-blinded, RCT	Intervention	FACT-Breast Cancer, STAI	Oncologist, nurse	Not reported	Patient education materials	• As ESAS severity score category increase associated with
								• Significant increase in proportion of visits with symptom documentation (pain and shortness of breath) (p < .0001)
								• Increase proportion of visits with symptom related actions (p < .0001)
Snyder, C., et al., 2010 [84]	Adults with breast or prostate cancer	Qualitative interviews and Cross-sectional study	Feasibility	None	Not applicable	Not applicable	Not applicable	• Only 2 domains that over 70% of patients reported discussing (pain and information needs), while 9 domains reported by physicians
								• Barriers to using PROs in clinical practice: (1) time constraints, (2) varying relevance of questions, (3) value of the conversational approach, (4) decreased usefulness in established relationships, and (5) respondent burden.
								• Benefits of PROs in clinical practice include (1) identifying problems, (2) serving as a reminder of topics to discuss, and (3) tracking changes over time.
Synder, C. 2014 [85]	Adults with breast or prostate cancer	RCT	Intervention	EORTC QLQ-C30, SCNS-SF34, PROMIS	Clinicians	Problematic scores noted	No instructions given, only usual best practice guidelines	• Patient feedback suggested differences in ratings for included PROs in order QLQ-C30, PROMIS, SCNS-SF34 (P < .05).
Synder, C. 2014 [85]	Adults with breast or prostate cancer	RCT	Intervention	EORTC QLQ-C30, SCNS-SF34, PROMIS	Clinicians	Problematic scores noted	No instructions given, only usual best practice guidelines	• Clinicians did not prefer one questionnaire over the others.
Taenzer, P. 2000 [86]	Adults with lung cancer	Controlled trial (no randomization)	Intervention	EORTC QLQ-C30, PSQ, PDIS	Nurse, physician	Score information provided	Exit Interview	• No group differences in patient satisfaction
								• More QL issues identified were addressed in the experimental group (p < .001)
								• No statistical differences in charting of issues and action taken.
Takeuchi, E. et al., 2011 [87]	Adults with any cancer type	RCT	Intervention	EORTC QLQ-C30, HADS	Oncologists	Score information provided, graphical display	No instructions given, only usual best practice guidelines	• Significant intervention effect for the discussion of symptoms (p-.008)
								• Discussion of most symptom initiated by patients, no group differences.
								• Severity of symptom associated with clinical discussion. No group differences
Trautmann, F. 2016 [45]	Adults with any cancer type	Cohort study	Implementation	EORTC QLQ-C30, NCCN-DT, BPI	Physician	1 SD above/below mean	Physician-patient consultation using traffic light color-coded scoring	• 79% of patients agreed to participate
								• 67% provided complete PRO information
								• Mean completion time 30 min
								• Rates of approaching patients to participate increased over time
Veilkova, G. 2010 [88]	Adults with any cancer type	RCT	Intervention	EORTC QLQ-C30, HADS	Physician	Not reported	Not reported	• Continuity of care communication rated better in intervention group (o < 03).
Veilkova, G. 2010 [88]	Adults with any cancer type	RCT	Intervention	EORTC QLQ-C30, HADS	Physician	Not reported	Not reported	• Stakeholders found PRO to be useful
Wagner, L. 2015 [52]	Adults with ovarian, uterine, or cervical malignancies, non-gynecologic malignancy	Cohort study	Implementation	PROMIS CAT, NCCN- DT, NCCN-Prostate Cancer, PGA	Oncologist, nurse	T scores provided, severity range information	No instructions given, only usual best practice guidelines	• 92% of patients completed at least one assessment
								• Of multiple assessments:
								o 79% Message requests read
								o 37% Assessments started
								o 93% Assessments completed of those started
								• Physical function generated most alerts (4% based on severe problems)
Whittle, A.K., 2016 [89]	Adults with any cancer type: urological, lung,colorectal, breast, and gynecological	Cohort study	Feasibility/ Intervention	CGA GOLD	Clinical team	Score information provided	Questions for patient to ask clinician	• Phase I Observational
								o 42% consent and completion rate
								o 11.7 min mean completion time
								o 86.3% completed CGA-GOLD without assistance.
								o 3.1% missing response rate
								• Phase II Intervention
								o 39% consent and completion rate
								o 89% unchanged decision after comparison of PRO results with clinical notes
Wolfe, J. 2014 [90]	Children with any cancer type	Cluster RCT	Intervention	PediQUEST (PQ): MSAS, PedsQL4.0, sickness	Clinic staff, palliative care service, pain service, patient	Graphical data display	No instructions given, only usual best practice guidelines	• PRO feedback did not have an effect on symptoms and HRQOL in the study, but effect found for children > 8 years surviving > 20 weeks.
								• Report found useful by:
								o 50% of providers
								o 54% of parents
								o 28% of patients
Wolpin, S., 2008 [91]	Adults with any cancer type	RCT	Feasibility/ Intervention	ESRA-C and SDS	Physician	Not reported	Not reported	• Mean completion time 15.20 min 5 out of 6 acceptability questions indicated very high acceptability (mean > 4, on a 1–5 range scale)
Wright, P., et al. 2007 [92]	Adults with any cancer type	Qualitative interviews and Cross-sectional study	Feasibility	SDI, HADS, EORTC QLQ-C30	Social worker investigator	Cut-offs/Thresholds	Under development	• Referral rates 24% for patients above PRO cutoff

¹ A post-hoc analyses of the data from the same study demonstrated symptom monitoring improves overall survival by 5 months (Basch, 2017). Results were published after the completion of the current review

Abbreviations: AE = adverse event, BPI = Brief Pain Inventory, CES-D = Center for Epidemiologic Studies Depression Scale; CGA GOLD = comprehensive geriatric assessment screening questionnaire, CTCAE = Common Terminology Criteria for Adverse Events, eCRF = electronic case report form, E-MOSAIC = electronic Monitoring Symptoms and Syndromes Associated with Advanced Cancer; EMR = electronic medical record, EORTC QLQ-BN20 = European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Brain Neoplasm, EORTC QLQ-BR23 = European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Breast, EORTC QLQ-C30 = European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Core 30, EORTC QLQ-CR38 = European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Colorectal, EORTC QLQ-LC13 = European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire Lung Cancer, EQ-5D = EuroQol Five Dimensions, ER = emergency room, ESAS = Edmonton Symptom Assessment Scale; ESRA-C = Electronic Self-Report Assessment for Cancer; FACT-G = Functional Assessment of Cancer Therapy General, GP = general practitioner, HADS = Hospital Anxiety and Depression Scale, HRQoL = health-related quality of life, MOS-SS = Medical Outcomes Study Social Support, MSAS = Memorial Symptom Assessment Scale, NCCN-DT = National Comprehensive Cancer Network Distress Thermometer, NCCN-EDT = National Comprehensive Cancer Network Emotional Distress Thermometer, PCI = patients Concerns Inventory, PDIS = patient-provider communication, PEDSQL = Pediatric Quality of Life Scale, PGA = physician’s global assessment, POMS-17 = Postoperative Morbidity Survey, PRO = patient-reported outcome, PROSQoLI = Prostate Cancer Specific Quality of Life Instrument, PSQ-III = Patient Satisfaction Questionnaire, PTSD = post-traumatic stress disorder; PTSD-CL = post-traumatic stress disorder civilian, QLIC-ON = Quality of Life in Childhood Oncology; QoL = quality of life, RCT = randomized controlled trial, REALM-SD = Rapid Estimate of Adult Literacy in Medicine;m SCNS-SF34 = Supportive Care Needs Survey Short Form, SDS = Symptom Distress Scale, SF-36 = Short-form 36, SQoL = subjective quality of life, STAI = State Trait Anxiety Inventory, TAPQoL = TNO-AZL Preschool Children Quality of Life, UWQoL = University of Washington Quality of Life

Results: Available evidence on reporting of ISOQOL PRO implementation guide categories

Data from all 36 reports of PRO measures used in clinical settings were summarized according to the ISOQOL PRO Implementation Guide Framework [35]. .Publications included research studies (intervention research (n = 19, 58%), feasibility research (n = 10, 28%), combination intervention and feasibility (n = 3, 8%), real-world implementation reports (n = 3, 8%); and an intervention for quality improvement (n = 1, 3%) We initially summarized findings related to design considerations of PRO integration in clinical practice within the specified ISOQOL categories (goals for collecting PROs, assessment details, PRO selection, and mode of administration) followed by evidence in categories related to reporting and use of PRO results (reporting of PRO results, PRO score interpretation, plans for addressing issues identified by the PRO, and evaluation of PRO impact on clinical practice). Table 2 presents a summary of key elements from each paper included in the review.

Goals for collecting PROs

As part of this synthesis step, we examined the stated clinical-practice goals for the PRO data collection. Planned use of PROs included monitoring symptoms (n = 13, 36%), improving quality of care (n = 10, 28%), enhancing patient-provider communication (n = 9, 25%), and delivering patient-centered care (n = 9, 25%) (Table 3).

Table 3

Reported Goals of PRO Inclusion in Clinical Care

PRO Use Goal*	N of Studies	Percent of Studies
Symptom Monitoring	13	36%
Patient-Centered Care	10	28%
Patient-Provider Communication	9	25%
Quality of Care	9	25%
Symptom Screening	5	14%
Symptom Screening/Symptom Monitoring	3	8%
Decision Aid	1	3%

Abbreviation: PRO = patient-reported outcome

Patients, setting, and timing of assessments

All studies provided relevant details on the type of patients included, study setting, and timing of assessment (before visit, during visit, after visit, at home). Most studies were conducted with adult populations (94%) in outpatient settings (92%). PROs were administered most often at the clinic immediately before seeing the doctor (36%) or during a visit (33%). The type of cancer patient varied with a majority of studies including three or more cancer types (69%), and only seven studies (19%) with a single cancer type.

PROs selected for use in the studies

A total of 46 PRO measures were used across the 36 studies; 33 of the PRO measures were rarely used and were included in only one or two studies suggesting wide variability in measures used. Studies predominantly reported measuring symptoms (n = 15, 42%) or cancer-specific HRQoL (n = 13, 33%) outcomes. The most widely-used measure was the EORTC QLQ-C30 (n = 10,28%) followed by the Hamilton Anxiety and Depression Scale (HADS) (n = 5, 14%).

PRO mode of administration

All but one of the studies reported the mode of PRO administration. Electronic administration was the most common mode (n = 31, 69%) followed by paper-and-pencil (n = 12, 27%). Two studies used interactive voice response (IVR) (n = 2, 5%).

Reporting of PRO results

The summary of information on PRO results indicated that the preferred format of results presentation was an electronic summary report (n = 19, 56%) or a printed copy of PRO results (n = 7, 19%) while e-mails/telephones (n = 5, 14%) were used less often. Results were most often presented only to the clinical team (n = 30, 83%); only three studies (11%) presented the PRO results to both the clinician and the patient, and one study presented results to patients only.

PRO score interpretation

A large proportion of studies (n = 17, 47%) failed to report information on how to interpret PRO scores. Eleven studies (31%) provided PRO scores alone with no interpretation guidance, and six articles (17%) did not report any information on PRO score interpretation. Fifteen studies (42%) provided scores in the form of a graphical display which may aid in score interpretation. About half of the papers reviewed (n = 19, 53%) reported PRO scores along with some information on threshold values, cut-off scores, or severity levels. Only three studies (8%) provided reference groups or norms information for the selected PRO.

Plans for addressing issues identified by the PRO

The majority of studies did not provide any instructions on follow-up steps when PRO scores raised areas of concern. In 13 studies (36%), no instructions were given on next steps or patient-management action items based on PRO results; eight studies (22%) did not include sufficient information on whether PRO results were addressed. The plans for addressing issues identified by the PROs were often related to discussing the identified issues with the provider (n = 10, 28%) with single studies also suggesting specialist referrals, reporting adverse events (AEs), and/or providing educational materials to patients.

Evaluation of PRO impact on clinical practice

Only 19 studies (53%) included results of PRO intervention on patient outcomes. The outcomes for which most studies reported evidence of PRO intervention effect included patient reported symptoms, functioining or quality of life scores (n = 13) and patient-provider communication (n = 8) (Table 4). Eleven of the 19 studies (58%) reported significant PRO intervention effects for all reported endpoints, and five studies (26%) had mixed results and reported significant PRO intervention effects for some—but not all—of the assessed outcomes. Only three of the 19 studies (16%) reported no intervention effect (Table 2).

Table 4

Intervention Study Outcomes and Evidence of PRO Intervention Effect

Type of Outcome*	N¹ Studies Reporting Intervention Effect	N¹ Studies Used Outcome	Percent of All Studies Using Outcome
PRO score (symptoms n = 8, functioning n = 1, HRQoL n = 6)	13	16	44%
Discussion of results/patient-centered communication	8	8	22%
Patient satisfaction with treatment	3	4	11%
Chart documentation	3	3	8%
Changes in clinical evaluation or treatment plan	3	3	8%
Number of provider visits	0	2	6%
Emergency-department visits	1	1	3%
One-year survival	1	1	3%

Abbreviation: HRQoL = health-related quality of life, PRO = patient-reported outcome

¹A total of 19 intervention studies reporting results on study outcomes were included in the review. Multiple outcomes assessed across studies

Results: Real-world implementation of PRO measures with oncology patients

Our review identified only three reports of real-world implementation of PRO measures in clinical practice which we defined as the ongoing administration of a standardized PRO and related clinical actions to manage patient care in routine clinical practice beyond the scope of a specific research study.

The first study [43] used retrospective chart review to investigate the relationship between standardized symptom screening and clinical actions to manage symptoms using the Edmonton Symptom Assessment Scale (ESAS) [44]. The ESAS was included in routine clinic visits though self-reporting via an electronic touch-screen kiosk. The ESAS measures the severity (scale of 0–10; 0 = none, 10 = worst) of nine common cancer physical and psychological symptoms (pain, shortness of breath, nausea, anxiety, depression, tiredness, drowsiness, appetite, and well-being). ESAS symptoms were categorized into four severity categories: none (0 score), mild (1–3 score), moderate (4–6 score), and severe (7–10 score) where scores of > 4 indicate clinically-significant symptom issues. Symptom-related actions included relevant drugs being prescribed, medication dosage titration, or a test, treatment, or referral being made. Pain and shortness of breath were documented in 52% and 30% of charts; a related action occurred in 17% and 4% of charts, respectively. However, the frequency of relevant clinical actions was not proportionate to the documented symptom severity [43].

Trautmann and colleagues [45] described the development, implementation, completeness, and first results of an electronic, real-time assessment program for the collection of PROs in a tertiary referral cancer center in Germany. The EORTC QLQ-C30 [46], National Comprehensive Cancer Network Distress Thermometer (DT) [47], and the Hornheider Screening Instrument (HIS) of need for psycho-oncological support [48] were measured. Nutritional status was assessed using the Short-Form Mini Nutritional Assessment (MNA) [49], and pain was assessed using the Brief Pain Inventory (BPI) [50, 51]. A traffic-light system was applied for visualized score interpretation using published cutoff values or means/standard deviations (SDs). A green light indicated scores below a critical clinical importance thresholds based on means and standard deviations of reference population of cancer patients thereby indicating no need for clinical action; a red light indicated scores above a critical cut-off that indicated need for further action. Overall, 67% of patients provided complete information on 12 PROs. Rates of approach and participation varied between the different departments with the highest completion rates in patients presenting for oncological surgical consultation. The number of patients approached to complete PROs increased from 17% to 56% over three months. The percentage of patients completing the PROs increased from 70% to 92% over three months. The majority of patients (62%) reported a score of five or higher on the NCCN Distress scale indicating moderate to high burden; 53% of the patients had a score of four or higher on the HSI indicating a need for psycho-oncological support. Very few participants reported on pain outcomes as EORTC QLQ-C30 pain intensity, and impairment scores were only documented in patients reporting moderate to severe pain. Findings revealed that physician usage of PRO during the clinical consultation was limited. Limiting factors reported by physicians were the lack of knowledge of the PRO reporting system and perceived irrelevancy of some of the assessed PRO data. Rates of clinical action were not reported in the study.

The authors acknowledged a number of obstacles in the study—even though there was an increased number of patients recruited, the usage of PROs in the patient-physician interaction was limited due to physician turnover and lack of completion time provided to patient prior to consultation. The authors concluded that PRO assessments should be more carefully selected to be more clearly of benefit to the health care provider and patient. Additionally, sustaining the implementation and interpretation of PROs should be constantly reinforced with clinicians [45].

Wagner and colleagues [52] assessed cancer-related symptoms with electronic health record (EHR) integration to communicate assessment results to clinical teams in real time. PROMIS computer adaptive tests (CATs) use a computer algorithm developed with item response theory to administer the items. The psychosocial assessment was adapted from the National Comprehensive Cancer Network Distress Thermometer and Problem Checklist [53]. Over the course of three years, 636 patients completed a total of 1493 assessments with 636 patients completing the assessment at least once (301 twice, 184 three times, and 129 four times). Most patients (90.1%) completed the assessment at home rather than at the clinic (9.3%). Severe PROMIS symptom scores (≥70 or 75 depending on symptom) triggered a message to the oncology team. PROMIS T-score clinical severity thresholds (normal, mild, moderate, or severe) have been previously determined with a standard setting exercise that converged clinician expert ratings and patient self-reported severity scores [54]. Overall, one-third of the patients reported current psychosocial health needs. The authors consider that this study demonstrates that precise measurement of symptoms can be implemented while maintaining the brevity required for clinical implementation. EEHR integration also facilitated automated triage for psychosocial and supportive care [52].

Discussion

Evidence on reporting of ISOQOL PRO implementation guide categories

In order to be able to follow the ISOQOL PRO Implementation Guide, researchers need to have a body of evidence to guide choices on recommended categories.The first aim of our review was to examine existing information on recommended implementation based on published information from oncology clinical settings. While no studies in our review directly referenced the ISOQOL PRO Implementation Guide [35], publications on PRO use in oncology clinical practices were well aligned in their reporting of most recommended implementation elements. However, a gap exists in the description of PRO interpretation guidelines and attendant patient-management recommendations necessary to improve PRO outcomes.

Most studies adequately described the planned goal for PRO data collection, study setting, and selection of PRO and mode of administration. Electronic administration, which allows for flexible integration of PROs in clinical care, was used by the majority of studies, but most were not formally integrated into the electronic health record. In addition, formal integration into the EHR may require intensive resources and stakeholder buy-in that have been lacking perhaps due to limited evidence of the improvement in patient outcomes or lack of financial alignment (or incentives).

While the EORTC QLQ-C30 was the most commonly-used measure across studies, a wide variety of PRO measures was reported suggesting there is little consensus on core domains or the “best” PRO to use or consideration whether a measure is developed for clinical trial or clinical practice use. Such variability may be an implementation barrier for PROs in everyday oncology practice—the burden to individual organizations associated with selecting PRO measures, developing assessment guidelines for the selected PROs, and interpreting PRO results may discourage adoption in routine clinical care.

The main gap in evidence identified by this review was the sparsity of interpretation guidelines for PRO results provided to care providers. While most of the reviewed studies provided PRO scores, fewer added interpretation guidelines to these scores or provided follow-up instructions or procedures in case a problem was identified by the PRO. This is an important gap. Without clarity on the meaning, significance, and interpretation of collected PRO data, how can clinical actions be effected to result in improved health care processes and outcomes?

Real-world implementation of PRO measures with oncology patients

Based on the review of the published literature, the use of PRO measures in routine cancer clinical practice outside the context of feasibility or research intervention studies is seldom reported. Only three reports of routine implementation of PROs in clinical settings were identified by the current review and provided limited information for our first key aim The multi-stage process that is required for developing, introducing, testing, integrating, and monitoring PROs in EHR systems has been achieved in only a few US medical centers [55]. Numerous barriers to implementation have been discussed in the literature including: (1) perception among clinicians that PRO completion consumes valuable time during the patient visit; (2) EHR systems have limited abiity to deliver PROs in user-friendly formats for patients; and (3) clinical ecosystem workflow demands challenge full implementation and integration into clinical practice [56].

While our review suggests that PRO implementation in real-world settings outside of research context is scarce, it is possible that PRO implementation in real-world oncology clinical practice may be underreported in the research literature. Implementation efforts may be viewed more as quality-improvement efforts that are building on existing evidence but not always viewed as generating evidence warranting dissemination [23]. Therefore, it is plausible that PRO implementation may be more widespread than indicated by existing peer-reviewed publications. In a recent article, Basch and colleagues [55] noted that a handful of institutions have successfully integrated systematic PRO collection into routine clinical practice; however, no published data have been generated from these institutions related to real-world PRO implementation in oncology clinical care.

Comparison to other systematic literature reviews

Some of our findings are consistent with the results from earlier literature reviews evaluating different aspects of PROs in the context of oncology care (Table 5). We confirmed earlier findings that there is evidence for the effectiveness of PROs on improving provider-patient communication and increased discussion of mental health issues [23, 31–34]. The EORTC QLQ-C30 was also found to be the most commonly-evaluated PRO in oncology clinical practice settings [32]. The wide variability of PRO measures used has also been noted [55] and continues to be a challenge as confirmed by our findings. Several of the earlier reviews also pointed out the need for increased attention in providing guidance for PRO implementation in oncology clinical practice [33, 34, 57]. Since the completion of our review window, several additional reviews have appeared that focus on some aspect of PRO use in oncology clinical care such as mechanisms through which PROs facilitate increase in patient-physician communication [58] and use of PROs specifically in treating lung cancer [59]. The unique contributions of the current review remains, as no other review focused on separately examining PRO real-world implementation reports or used the ISOQOL PRO Implementation Guide as the framework in analyzing the identified articles. The use of this framework has allowed us to identify specific gaps in the PRO implementation cycle that need to be addressed to encourage use in clinical practice―mainly, the insufficient focus on developing and providing clear PRO score interpretation guidelines and patient-management action plans related to PRO results.

Table 5

Summary of Earlier Relevant SLR of Implementation of PRO in Cancer Clinical Care

Review Reference	Review goal	Databases + Search strategy	# References Screened	# Articles reviewed	Timeframe	Major Conclusions
Howell et al. 2015 [32]	To identify PROMs used in routine cancer clinical practice, their impact on patient, provider, and system outcomes, and the implementation factors influencing uptake.	Ovid Medline CINAHL PsycINFO Grey Literature	3297	30	2003–2013	The EORTC QLQ30 was the most commonly used PRO Use of PROMs for screening for emotional distress, unmet supportive care needs, or social difficulties Wide variety of PROMs were used with little standardization across studies PROMs implementation improves communication about symptoms and QoL More attention needs to be paid to complexity of implementation and interpretation of PROMS
King et al. 2016 [33]	To examine the use and impact of using quality of life measures on health care of cancer patients within a clinical setting, particularly those with brain cancer.	PubMed, EMBASE, Cochrane (SR & Trials), Web of Science [SCI]) Grey literature	18,483	19	2000–2015	QoL data may improve patient–physician communication, increase discussion of emotional functioning in particular. Scarcity of data on actions/medical decisions.
Chen et al. 2013 [31]	To provide a comprehensive review update including all relevant quantitative studies investigating the effectiveness of routine PRO collection in cancer patients.	NR Two-step search strategy building on existing reviews	1182	27	2000–2011	Strong evidence: PROs enhances patient-provider communication, improves patient satisfaction. Moderate evidence: PRO improves monitoring of treatment response and the detection of unrecognized problems. Weak Evidence: Changes to patient management, improved health outcomes No evidence: changes to patient health behavior, quality improvement, increased transparency, accountability, public reporting and better health care system performance
Jensen et al. 2013 [93]	To identify existing PRO systems and their administration of PRO assessments, integration of information into the clinic workflow and EHR systems, and the reporting of PRO information.	PubMed MEDLINE	190 plus conference abstracts and gray literature	33 ePRO systems reviewed	Not specified, conferences 2009–2011	Identified systems were generally developed to improve symptom management, identify psychosocial problems, and facilitate patient-provider communication. Data on actual impact was not part of review scope
Antunes et al. 2014 [57]	To systematically identify facilitators and barriers to the implementation of patient-reported outcome measures in different palliative care settings for routine practice	Medline, PsycINFO, Cumulative Index to Nursing Allied Health Literature, Embase British Nursing Index	3863	31	1985–2011	There is a need for guidance on implementing PROMs in palliative care clinical practice.
Alsaleh 2013 [23]	To review the scientific evidence behind recommending the use of QoL scales routinely in outpatient evaluation.	Medline, Embase, PsycINFO	486	6	1990–2012	Evidence for the use of QoL scales in daily clinical practice is limited. Some weak evidence suggesting that this might improve communication between patients and health caregivers. No good evidence that routine administration of QoL questionnaires improve patient’s QoL or changes management. The overall impression is that routine administration of questionnaires in medical oncology outpatient clinics is currently hardly justified.
Luckett et al. 2009 [94]	To identify future strategies for PRO interventions to impact patient outcomes in cancer clinics	MEDLINE PsycINFO References from earlier review included	576	6	2006–2008	More trials are urgently needed to build a satisfactory evidence base for the routine clinical use of patient-reported data in oncology. Evidence for improvement in patient outcomes as result of PROM use has been limited
Kotronoulas et al. 2014 [34]	Is inclusion of PROM in routine clinical practice associated with improvements in patient outcomes, processes of care, and health service outcomes during active anticancer treatment	Medline, EMBASE, CINAHL, PsycINFO, PBSC	5015	26	Database inception −2012	Use of PROMs increases the frequency of discussion of patient outcomes. Some support for positive association between use of PROSM and improved symptom control, increased supportive care measures, and patient satisfaction. Need for additional effort to ensure patient adherence and clear system guidelines to guide clinicians response. More research needed to support cost-benefit.
Yang et al. 2017 [58]	To identify mechanisms through which PROs facilitate patient-clinician communication in the adult oncology population.	MEDLINE, EMBASE, CINAHL, PsycINFO, Cab Direct, CDSR	610	43	Prior to 2016	PROs facilitate patient-clinician communication through various mechanisms that could perhaps contribute to improvements in symptom management and survival. The impact of PROs on clinical outcomes, however, remains poorly studied.
Bouazzaa et al. 2017 [59]	To analyze the use of PROMs in the treatment of lung cancer with the aim of improving the quality of care. The secondary objective is to evaluate PROMS currently being used in the care of lung cancer.	PubMed, Web of Science and Google Scholar	1118	51	2010–2016	There has yet to be a study on the routine implementation of lung cancer specific PROMs, but PROMs have a promising role.

Abbreviations: EHR = electronic health record, ePRO = electronic patient-reported outcome, EORTC QLQ-C30 = European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire-Core 30, PRO = patient-reported outcome, PROM = patient-reported outcome measure, QoL = quality of life

Strengths and limitations

The strengths of this review include the compliance with PRISMA guidelines, development of a comprehensive two-step search strategy, and review of results in the context of a framework built on the ISOQOL PRO Implementation Guide [35]. The results of the review help further the conversation on PRO implementation in oncology clinical practice by identifying gaps in guidance on interpretation of PRO results and action-oriented patient management based on PRO results. The review also has some limitations including the relatively small number of databases included in the review.

Conclusion

The existing evidence of PRO implementation in real-world clinical care in the published literature is very limited. It is unclear whether implementation efforts are not being studied, not being reported in peer-reviewed journals, simply being published in the grey literature, or not taking place at all.

While publication on PRO real-world implementation is uncommon, a good number of publications on PRO feasibility and/or PRO use in research in oncology clinical care exists as evidenced by our work and earlier literature reviews. This paper also aimed to organize findings of published studies in a framework informed by the ISOQOL PRO Implementation Guide [35]. Results suggested that, with the notable exception of PRO score interpretation and action strategies for PRO-identified problems, most studies report information suggested by the ISOQOL PRO Implementation Guide.

Based on the findings from our review, we offer two insights to help enable more widespread PRO implementation in routine clinical practice. First, adequate interpretation guidelines are needed for PRO results to be acted upon in clinical practice. Second, exploration should be conducted into how to best address issues raised by PRO results—particularly when the identified needs of patients extend beyond the expertise or training found in a routine oncology clinical practice such as depression or lack of social support. In the absence of available information on these key elements, implementation of PROs in clinical practice is unlikely to bridge the gap between perceived usefulness by researchers and routine uptake in oncology practice by clinicians.

Abbreviations

ASCO:: American Society of Clinical Oncology

BPI:: Brief Pain Inventory

DT:: Distress Thermometer

EHR:: Electronic health record

EORTC QLQ-C30:: European Organisation for Research and Treatment of Cancer Quality-of-Life Questionnaire-Core 30

ESAS:: Edmonton Symptom Assessment Scale

HIS:: Hornheider Screening Instrument

HRQoL:: Health-related quality of life

ISOQOL:: International Society for Quality of Life Research

MNA:: Mini Nutritional Assessment

PRISMA:: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

PRO:: Patient-reported outcome

SD:: Standard deviation

SLR:: Systematic literature review

US:: United States

Declarations

Acknowledgements

This work was supported by funding from Genentech, South San Francisco, CA. The authors thank Michael Celone, Owen Cooper, Ashley Duenas, and David Hengerer for their assistance with data collection, and Fritz Hamme for editorial assistance.

Funding

This study was funded by Genentech, Inc.

Availability of data and materials

The data collected during this study may be considered for release in response to reasonable requests to review the data for scientific and/or research purposes.

Authors’ contributions

Research project: A. Conception, B. Organization, C. Execution; A, B, C: MA, SD, DJ, CAM, AS, JW. Statistical Analysis: A. Design, B. Execution, C. Review and Critique; A: B: MA, CAM, AS, B: MA, AS, C: MA, SD, DJ, CAM, AS, JW. Manuscript: A. Writing of the first draft, B. Review and Critique. A: MA, CAM, AS, B: MA, SD, DJ, CAM, AS, JW. All authors read and approved the final manuscript.

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Milena Anatchkova, Anne M. Skalicky, and Colleen A. McHorney are employees of Evidera and received funding from Genentech to conduct the systematic literature review. Jennifer Whiteley is an employee of Genentech, Inc., a member of the Roche Group, and receives stock/options as part of her standard employment compensation.

Dayo Jagun and Sarah Donelson were full-time Genentech employees when this study was initiated and completed. Dayo Jagun and Sarah Donelson reviewed and approved the first and all subsequent drafts of the manuscript.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)

Patient-Centered Research, Evidera, Bethesda, MD, USA

(2)

Genentech Inc., South San Francisco, CA, USA

(3)

San Francisco, California, USA

(4)

Tampa, Florida, USA

References

Lohr, K. N., & Zebrack, B. J. (2009). Using patient-reported outcomes in clinical practice: Challenges and opportunities. Qual Life Res, 18(1), 99–107. https://doi.org/10.1007/s11136-008-9413-7.View ArticlePubMedGoogle Scholar
Food and Drug Administration. (2009). Guidance for industry on patient-reported outcome measures: Use in medical product development to support labeling claims. Fed Regist, 74(235), 65132–65133.Google Scholar
McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Qual Life Res, 4(4), 293–307.View ArticleGoogle Scholar
Almy, T. P. (1988). Comprehensive functional assessment for elderly patients. Ann Intern Med, 109(1), 70–72.View ArticleGoogle Scholar
Fried, L., Herdman, S., Kuhn, K., Rubin, G., & Turano, K. (1991). Preclinical disability: Hypotheses about the bottom of the iceberg. J Aging Health, 3(2), 285–300.View ArticleGoogle Scholar
Detmar, S. B., Aaronson, N. K., Wever, L. D., Muller, M., & Schornagel, J. H. (2000). How are you feeling? Who wants to know? Patients' and oncologists' preferences for discussing health-related quality-of-life issues. J Clin Oncol, 18(18), 3295–3301. https://doi.org/10.1200/JCO.2000.18.18.3295.View ArticlePubMedGoogle Scholar
Levine, M. N., Gafni, A., Markham, B., & MacFarlane, D. (1992). A bedside decision instrument to elicit a patient's preference concerning adjuvant chemotherapy for breast cancer. Ann Intern Med, 117(1), 53–58.View ArticleGoogle Scholar
Street, R. L., Jr., Gold, W. R., & McDowell, T. (1994). Using health status surveys in medical consultations. Med Care, 32(7), 732–744.View ArticleGoogle Scholar
Branch, W. T., & Malik, T. K. (1993). Using ‘windows of opportunities’ in brief interviews to understand patients’ concerns. JAMA, 269(13), 1667–1668.View ArticleGoogle Scholar
Delbanco, T. L. (1992). Enriching the doctor-patient relationship by inviting the patient's perspective. Ann Intern Med, 116(5), 414–418.View ArticleGoogle Scholar
Taylor, K. M., Macdonald, K. G., Bezjak, A., Ng, P., & DePetrillo, A. D. (1996). Physicians' perspective on quality of life: An exploratory study of oncologists. Qual Life Res, 5(1), 5–14.View ArticleGoogle Scholar
Avalere. (2014) A multi-stakeholder vision for patient-centered measurement in new payment and delivery models. http://avalere.com/expertise/life-sciences/insights/avalere-white-paper-facilitating-a-transition-to-using-pros-to-measure-perf.
Lansky, D., Butler, J. B., & Waller, F. T. (1992). Using health status measures in the hospital setting from acute care to ‘outcomes management’. Med Care, 30(5), MS57–MS73.PubMedGoogle Scholar
Stucki, G., & Sangha, O. (1996). Clinical quality management: Putting the pieces together. Arthritis Care Res, 9(5), 405–412.View ArticleGoogle Scholar
Deyo, R. A., & Carter, W. B. (1992). Strategies for improving and expanding the application of health status measures in clinical settings. A researcher-developer viewpoint. Med Care, 30(5), MS176–MS186 discussion MS196–209.PubMedGoogle Scholar
Baumhauer, J. F. (2017). Patient-reported outcomes - are they living up to their potential? N Engl J Med, 377(1), 6–9. https://doi.org/10.1056/NEJMp1702978.View ArticlePubMedGoogle Scholar
Jensen, R. E., Rothrock, N. E., DeWitt, E. M., Spiegel, B., Tucker, C. A., Crane, H. M., Forrest, C. B., Patrick, D. L., Fredericksen, R., Shulman, L. M., Cella, D., & Crane, P. K. (2015). The role of technical advances in the adoption and integration of patient-reported outcomes in clinical care. Med Care, 53(2), 153–159. https://doi.org/10.1097/MLR.0000000000000289.View ArticlePubMedPubMed CentralGoogle Scholar
Wintner LM, Sztankay M, Aaronson N, Bottomley A, Giesinger JM, Groenvold M, Petersen MA, van de Poll-Franse L, Velikova G, Verdonck-de Leeuw I, Holzner B, Group EQoL. (2016). The use of eortc measures in daily clinical practice-a synopsis of a newly developed manual. Eur J Cancer, 68, 73–81. https://doi.org/10.1016/j.ejca.2016.08.024.View ArticleGoogle Scholar
Detmar, S. B., Muller, M. J., Schornagel, J. H., Wever, L. D., & Aaronson, N. K. (2002). Health-related quality-of-life assessments and patient-physician communication: A randomized controlled trial. JAMA, 288(23), 3027–3034.View ArticleGoogle Scholar
Rose, M., & Bezjak, A. (2009). Logistics of collecting patient-reported outcomes (pros) in clinical practice: An overview and practical examples. Qual Life Res, 18(1), 125–136. https://doi.org/10.1007/s11136-008-9436-0.View ArticlePubMedGoogle Scholar
Santana, M. J., Feeny, D., Johnson, J. A., McAlister, F. A., Kim, D., Weinkauf, J., & Lien, D. C. (2010). Assessing the use of health-related quality of life measures in the routine clinical care of lung-transplant patients. Qual Life Res, 19(3), 371–379. https://doi.org/10.1007/s11136-010-9599-3.View ArticlePubMedGoogle Scholar
Velikova, G. (2004). Use of electronic quality of life applications in cancer research and clinical practice. Expert Rev Pharmacoecon Outcomes Res, 4(4), 403–411. https://doi.org/10.1586/14737167.4.4.403.View ArticlePubMedGoogle Scholar
Alsaleh, K. (2013). Routine administration of standardized questionnaires that assess aspects of patients' quality of life in medical oncology clinics: A systematic review. J Egypt Natl Canc Inst, 25(2), 63–70. https://doi.org/10.1016/j.jnci.2013.03.001.View ArticlePubMedGoogle Scholar
Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., & Schrag, D. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. https://doi.org/10.1001/jama.2017.7156.
Lipscomb, J., Gotay, C. C., & Snyder, C. F. (2007). Patient-reported outcomes in cancer: A review of recent research and policy initiatives. CA Cancer J Clin, 57(5), 278–300. https://doi.org/10.3322/CA.57.5.278.View ArticlePubMedGoogle Scholar
Stover, A. M., & Basch, E. M. (2016). Implementation of sumptom questionnaires into oncology workflow. J Clin Oncol, 12(10), 859–862.Google Scholar
President’s Cancer Panel Connected health: Improving patients’ engagement and activation for cancer-related outcomes: President’s cancer panel 2014–2015 series. https://deainfo.nci.nih.gov/advisory/pcp/glance/ConnectedHealth14-15.pdf. Accessed 27 Feb 2018.
Stover, A. M., Chiang, A. C., & Basch, E. M. (2016). Asco pro workgroup update: Patient-reported outcome measures as a quality indicator. J Clin Oncol, 34(7), 276.View ArticleGoogle Scholar
Basch, E., Snyder, C., McNiff, K., Brown, R., Maddux, S., Smith, M. L., Atkinson, T. M., Howell, D., Chiang, A., Wood, W., Levitan, N., Wu, A. W., & Krzyzanowska, M. (2014). Patient-reported outcome performance measures in oncology. J Oncol Pract, 10(3), 209–211. https://doi.org/10.1200/JOP.2014.001423.View ArticlePubMedPubMed CentralGoogle Scholar
Lipscomb, J., Reeve, B. B., Clauser, S. B., Abrams, J. S., Bruner, D. W., Burke, L. B., Denicoff, A. M., Ganz, P. A., Gondek, K., Minasian, L. M., O'Mara, A. M., Revicki, D. A., Rock, E. P., Rowland, J. H., Sgambati, M., & Trimble, E. L. (2007). Patient-reported outcomes assessment in cancer trials: Taking stock, moving forward. J Clin Oncol, 25(32), 5133–5140. https://doi.org/10.1200/JCO.2007.12.4644.View ArticlePubMedGoogle Scholar
Chen, J., Ou, L., & Hollis, S. J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res, 13, 211. https://doi.org/10.1186/1472-6963-13-211.View ArticlePubMedPubMed CentralGoogle Scholar
Howell, D., Molloy, S., Wilkinson, K., Green, E., Orchard, K., Wang, K., & Liberty, J. (2015). Patient-reported outcomes in routine cancer clinical practice: A scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol, 26(9), 1846–1858. https://doi.org/10.1093/annonc/mdv181.View ArticlePubMedGoogle Scholar
King, S., Exley, J., Parks, S., Ball, S., Bienkowska-Gibbs, T., MacLure, C., Harte, E., Stewart, K., Larkin, J., Bottomley, A., & Marjanovic, S. (2016). The use and impact of quality of life assessment tools in clinical care settings for cancer patients, with a particular emphasis on brain cancer: Insights from a systematic review and stakeholder consultations. Qual Life Res, 25(9), 2245–2256. https://doi.org/10.1007/s11136-016-1278-6.View ArticlePubMedPubMed CentralGoogle Scholar
Kotronoulas, G., Kearney, N., Maguire, R., Harrow, A., Di Domenico, D., Croy, S., & MacGillivray, S. (2014). What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol, 32(14), 1480–1501. https://doi.org/10.1200/JCO.2013.53.5948.View ArticlePubMedGoogle Scholar
International Society for Quality of Life Research prepared by Aaronson N ET, Greenhalgh J, Halyard M, Hess R, Miller D, Reeve B, Santana M, Snyder C User’s guide to implementing patient-reported outcomes assessment in clinical practice, Version: 2015.Google Scholar
McHorney, C. A. (1999). Health status assessment methods for adults: Past accomplishments and future challenges. Annu Rev Public Health, 20, 309–335. https://doi.org/10.1146/annurev.publhealth.20.1.309.View ArticlePubMedGoogle Scholar
Higginson, I. J., & Carr, A. J. (2001). Measuring quality of life: Using quality of life measures in the clinical setting. BMJ, 322(7297), 1297–1300.View ArticleGoogle Scholar
Santana, M. J., & Feeny, D. (2014). Framework to assess the effects of using patient-reported outcome measures in chronic care management. Qual Life Res, 23(5), 1505–1513. https://doi.org/10.1007/s11136-013-0596-1.View ArticlePubMedGoogle Scholar
Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & Group P-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015 statement. Syst Rev, 4(1). https://doi.org/10.1186/2046-4053-4-1.
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & Group P-P. (2015). Preferred reporting items for systematic review and meta-analysis protocols (prisma-p) 2015: Elaboration and explanation. BMJ, 349, g7647. https://doi.org/10.1136/bmj.g7647.View ArticleGoogle Scholar
Popay J, Roberts H, Sowden A, et al. (2006) Guidance on the conduct of narrative synthesis in systematic reviews: A product from the esrc methods programme. http://www.lancaster.ac.uk/shm/research/nssr/research/dissemination/publications/NS_Synthesis_Guidance_v1.pdf.
Borstlap, M., van de Laar, M., Zant, J., & van der Korst, J. (1993). Components of health: An analysis in rheumatoid arthritis using quality of life questionnaires and clinical and laboratory variables. Ann Rheum Dis, 52(9), 650–654.View ArticleGoogle Scholar
Seow, H., Sussman, J., Martelli-Reid, L., Pond, G., & Bainbridge, D. (2012). Do high symptom scores trigger clinical actions? An audit after implementing electronic symptom screening. J Oncol Pract, 8(6), e142–e148. https://doi.org/10.1200/JOP.2011.000525.View ArticlePubMedPubMed CentralGoogle Scholar
Bruera, E., Kuehn, N., Miller, M. J., Selmser, P., & Macmillan, K. (1991). The Edmonton symptom assessment system (esas): A simple method for the assessment of palliative care patients. J Palliat Care, 7(2), 6–9.PubMedGoogle Scholar
Trautmann, F., Hentschel, L., Hornemann, B., Rentsch, A., Baumann, M., Ehninger, G., Schmitt, J., & Schuler, M. (2016). Electronic real-time assessment of patient-reported outcomes in routine care-first findings and experiences from the implementation in a comprehensive cancer center. Support Care Cancer, 24(7), 3047–3056. https://doi.org/10.1007/s00520-016-3127-0.View ArticlePubMedGoogle Scholar
Aaronson, N. K., Ahmedzai, S., Bergman, B., Bullinger, M., Cull, A., Duez, N. J., Filiberti, A., Flechtner, H., Fleishman, S. B., de Haes, J. C., et al. (1993). The european organization for research and treatment of cancer qlq-c30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst, 85(5), 365–376.View ArticleGoogle Scholar
Roth, A. J., Kornblith, A. B., Batel-Copel, L., Peabody, E., Scher, H. I., & Holland, J. C. (1998). Rapid screening for psychologic distress in men with prostate carcinoma: A pilot study. Cancer, 82(10), 1904–1908.View ArticleGoogle Scholar
Strittmatter G, Mawick R, Tilkorn M (2000) Entwicklung und klinischer einsatz von screening-instrumenten zur identifikation betreuungsbedürftiger tumorpatienten. In: Leb. Aus med. –soziol. Perspekt. Hogrefe, Göttingen, Bern, Toronto, Seattle, pp 59–75.Google Scholar
Rubenstein, L. Z., Harker, J. O., Salva, A., Guigoz, Y., & Vellas, B. (2001). Screening for undernutrition in geriatric practice: Developing the short-form mini-nutritional assessment (mna-sf). J Gerontol A Biol Sci Med Sci, 56(6), M366–M372.View ArticleGoogle Scholar
Cleeland, C. S., & Ryan, K. M. (1994). Pain assessment: Global use of the brief pain inventory. Ann Acad Med Singap, 23(2), 129–138.PubMedGoogle Scholar
Cleeland, C. S. (2009). The brief pain inventory user guide. Houston: M.D. Anderson Cancer Center.Google Scholar
Wagner, L. I., Schink, J., Bass, M., Patel, S., Diaz, M. V., Rothrock, N., Pearman, T., Gershon, R., Penedo, F. J., Rosen, S., & Cella, D. (2015). Bringing promis to practice: Brief and precise symptom screening in ambulatory cancer care. Cancer, 121(6), 927–934. https://doi.org/10.1002/cncr.29104.View ArticlePubMedGoogle Scholar
Holland, J. C., Andersen, B., Breitbart, W. S., Compas, B., Dudley, M. M., Fleishman, S., Fulcher, C. D., Greenberg, D. B., Greiner, C. B., Handzo, G. F., Hoofring, L., Jacobsen, P. B., Knight, S. J., Learson, K., Levy, M. H., Loscalzo, M. J., Manne, S., McAllister-Black, R., Riba, M. B., Roper, K., Valentine, A. D., Wagner, L. I., Zevon, M. A., & Panel, N. D. M. (2010). Distress management. J Natl Compr Cancer Netw, 8(4), 448–485.View ArticleGoogle Scholar
Cella, D., Choi, S., Rosenbloom, S. K., et al. (2008). A novel irt-based caseranking approach to derive expert standards for symptom severity. Qual Life Res, 17(1), A32.Google Scholar
Basch, E. (2017). Patient-reported outcomes - harnessing patients' voices to improve clinical care. N Engl J Med, 376(2), 105–108. https://doi.org/10.1056/NEJMp1611252.View ArticlePubMedGoogle Scholar
Jensen, R. E., Potosky, A. L., Moinpour, C. M., Lobo, T., Cella, D., Hahn, E. A., Thissen, D., Smith, A. W., Ahn, J., Luta, G., & Reeve, B. B. (2017). United States population-based estimates of patient-reported outcomes measurement information system symptom and functional status reference values for individuals with cancer. J Clin Oncol, 35(17), 1913–1920. https://doi.org/10.1200/JCO.2016.71.4410.View ArticlePubMedPubMed CentralGoogle Scholar
Antunes B, Harding R, Higginson IJ, Euroimpact (2014) Implementing patient-reported outcome measures in palliative care clinical practice: A systematic review of facilitators and barriers. Palliat Med 28 (2):158–175. doi:https://doi.org/10.1177/0269216313491619.
Yang, L. Y., Manhas, D. S., Howard, A. F., & Olson, R. A. (2018). Patient-reported outcome use in oncology: A systematic review of the impact on patient-clinician communication. Support Care Cancer, 26(1), 41–60. https://doi.org/10.1007/s00520-017-3865-7.View ArticlePubMedGoogle Scholar
Bouazza, Y. B., Chiairi, I., El Kharbouchi, O., De Backer, L., Vanhoutte, G., Janssens, A., & Van Meerbeeck, J. P. (2017). Patient-reported outcome measures (proms) in the management of lung cancer: A systematic review. Lung Cancer, 113, 140–151. https://doi.org/10.1016/j.lungcan.2017.09.011.View ArticlePubMedGoogle Scholar
Alvarez-Maestro, M., Viladoms, J. M., Fernandez, A., & De la Cruz, G. (2014). Evaluation of the clinical usefulness of a health-related quality of life questionnaire in patients with prostate cancer. Actas Urol Esp, 38(10), 669–677. https://doi.org/10.1016/j.acuro.2014.03.007.View ArticlePubMedGoogle Scholar
Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., Sabbatini, P., Rogak, L., Bennett, A. V., Dueck, A. C., Atkinson, T. M., Chou, J. F., Dulko, D., Sit, L., Barz, A., Novotny, P., Fruscione, M., Sloan, J. A., & Schrag, D. (2016). Symptom monitoring with patient-reported outcomes during routine cancer treatment: A randomized controlled trial. J Clin Oncol, 34(6), 557–565. https://doi.org/10.1200/JCO.2015.63.0830.View ArticlePubMedGoogle Scholar
Basch, E., Artz, D., Iasonos, A., Speakman, J., Shannon, K., Lin, K., Pun, C., Yong, H., Fearn, P., Barz, A., Scher, H. I., McCabe, M., & Schrag, D. (2007). Evaluation of an online platform for cancer patient self-reporting of chemotherapy toxicities. J Am Med Inform Assoc, 14(3), 264–268. https://doi.org/10.1197/jamia.M2177.View ArticlePubMedPubMed CentralGoogle Scholar
Berry, D. L., Hong, F., Halpenny, B., Partridge, A. H., Fann, J. R., Wolpin, S., Lober, W. B., Bush, N. E., Parvathaneni, U., Back, A. L., Amtmann, D., & Ford, R. (2014). Electronic self-report assessment for cancer and self-care support: Results of a multicenter randomized trial. J Clin Oncol, 32(3), 199–205. https://doi.org/10.1200/JCO.2013.48.6662.View ArticlePubMedGoogle Scholar
Berry, D. L., Blonquist, T. M., Patel, R. A., Halpenny, B., & McReynolds, J. (2015). Exposure to a patient-centered, web-based intervention for managing cancer symptom and quality of life issues: Impact on symptom distress. J Med Internet Res, 17(6), e136. https://doi.org/10.2196/jmir.4190.View ArticlePubMedPubMed CentralGoogle Scholar
Berry, D. L., Blumenstein, B. A., Halpenny, B., Wolpin, S., Fann, J. R., Austin-Seymour, M., Bush, N., Karras, B. T., Lober, W. B., & McCorkle, R. (2011). Enhancing patient-provider communication with the electronic self-report assessment for cancer: A randomized trial. J Clin Oncol, 29(8), 1029–1035. https://doi.org/10.1200/JCO.2010.30.3909.View ArticlePubMedPubMed CentralGoogle Scholar
Blum, D., Koeberle, D., Omlin, A., Walker, J., Von Moos, R., Mingrone, W., deWolf-Linder, S., Hayoz, S., Kaasa, S., Strasser, F., & Ribi, K. (2014). Feasibility and acceptance of electronic monitoring of symptoms and syndromes using a handheld computer in patients with advanced cancer in daily oncology practice. Support Care Cancer, 22(9), 2425–2434. https://doi.org/10.1007/s00520-014-2201-8.View ArticlePubMedGoogle Scholar
Boyes, A., Newell, S., Girgis, A., McElduff, P., & Sanson-Fisher, R. (2006). Does routine assessment and real-time feedback improve cancer patients' psychosocial well-being? Eur J Cancer Care (Engl), 15(2), 163–171. https://doi.org/10.1111/j.1365-2354.2005.00633.x.View ArticleGoogle Scholar
Chiang, A. C., Buia Amport, S., Corjulo, D., Harvey, K. L., & McCorkle, R. (2015). Incorporating patient-reported outcomes to improve emotional distress screening and assessment in an ambulatory oncology clinic. J Oncol Pract, 11(3), 219–222. https://doi.org/10.1200/JOP.2015.003954.View ArticlePubMedGoogle Scholar
Compaci, G., Rueter, M., Lamy, S., Oberic, L., Recher, C., Lapeyre-Mestre, M., Laurent, G., & Despas, F. (2015). Ambulatory medical assistance--after cancer (ama-ac): A model for an early trajectory survivorship survey of lymphoma patients treated with anthracycline-based chemotherapy. BMC Cancer, 15, 781. https://doi.org/10.1186/s12885-015-1815-7.View ArticlePubMedPubMed CentralGoogle Scholar
Cox, A., Illsley, M., Knibb, W., Lucas, C., O'Driscoll, M., Potter, C., Flowerday, A., & Faithfull, S. (2011). The acceptability of e-technology to monitor and assess patient symptoms following palliative radiotherapy for lung cancer. Palliat Med, 25(7), 675–681. https://doi.org/10.1177/0269216311399489.View ArticlePubMedGoogle Scholar
Engelen, V., Detmar, S., Koopman, H., Maurice-Stam, H., Caron, H., Hoogerbrugge, P., Egeler, R. M., Kaspers, G., & Grootenhuis, M. (2012). Reporting health-related quality of life scores to physicians during routine follow-up visits of pediatric oncology patients: Is it effective? Pediatr Blood Cancer, 58(5), 766–774. https://doi.org/10.1002/pbc.23158.View ArticlePubMedGoogle Scholar
Epstein, R. M., Duberstein, P. R., Fenton, J. J., Fiscella, K., Hoerger, M., Tancredi, D. J., Xing, G., Gramling, R., Mohile, S., Franks, P., Kaesberg, P., Plumb, S., Cipri, C. S., Street, R. L., Jr., Shields, C. G., Back, A. L., Butow, P., Walczak, A., Tattersall, M., Venuti, A., Sullivan, P., Robinson, M., Hoh, B., Lewis, L., & Kravitz, R. L. (2017). Effect of a patient-centered communication intervention on oncologist-patient communication, quality of life, and health care utilization in advanced cancer: The voice randomized clinical trial. JAMA Oncol, 3(1), 92–100. https://doi.org/10.1001/jamaoncol.2016.4373.View ArticlePubMedPubMed CentralGoogle Scholar
Erharter, A., Giesinger, J., Kemmler, G., Schauer-Maurer, G., Stockhammer, G., Muigg, A., Hutterer, M., Rumpold, G., Sperner-Unterweger, B., & Holzner, B. (2010). Implementation of computer-based quality-of-life monitoring in brain tumor outpatients in routine clinical practice. J Pain Symptom Manag, 39(2), 219–229. https://doi.org/10.1016/j.jpainsymman.2009.06.015.View ArticleGoogle Scholar
Hilarius, D. L., Kloeg, P. H., Gundy, C. M., & Aaronson, N. K. (2008). Use of health-related quality-of-life assessments in daily clinical oncology nursing practice: A community hospital-based intervention study. Cancer, 113(3), 628–637. https://doi.org/10.1002/cncr.23623.View ArticlePubMedGoogle Scholar
Izard, J., Hartzler, A., Avery, D. I., Shih, C., Dalkin, B. L., & Gore, J. L. (2014). User-centered design of quality of life reports for clinical care of patients with prostate cancer. Surgery, 155(5), 789–796. https://doi.org/10.1016/j.surg.2013.12.007.View ArticlePubMedGoogle Scholar
Kallen, M. A., Yang, D., & Haas, N. (2012). A technical solution to improving palliative and hospice care. Support Care Cancer, 20(1), 167–174. https://doi.org/10.1007/s00520-011-1086-z.View ArticlePubMedGoogle Scholar
Mooney, K. H., Beck, S. L., Friedman, R. H., Farzanfar, R., & Wong, B. (2014). Automated monitoring of symptoms during ambulatory chemotherapy and oncology providers' use of the information: A randomized controlled clinical trial. Support Care Cancer, 22(9), 2343–2350. https://doi.org/10.1007/s00520-014-2216-1.View ArticlePubMedPubMed CentralGoogle Scholar
Nicklasson, M., Elfstrom, M. L., Olofson, J., & Bergman, B. (2013). The impact of individual quality of life assessment on psychosocial attention in patients with chest malignancies: A randomized study. Support Care Cancer, 21(1), 87–95. https://doi.org/10.1007/s00520-012-1496-6.View ArticlePubMedGoogle Scholar
Rogers, S. N., Lowe, D., & Kanatas, A. (2016). Suitability of the patient concerns inventory as a holistic screening tool in routine head and neck cancer follow-up clinics. Br J Oral Maxillofac Surg, 54(4), 415–421. https://doi.org/10.1016/j.bjoms.2016.01.018.View ArticlePubMedGoogle Scholar
Rosenbloom, S. K., Victorson, D. E., Hahn, E. A., Peterman, A. H., & Cella, D. (2007). Assessment is not enough: A randomized controlled trial of the effects of hrql assessment on quality of life and satisfaction in oncology clinical practice. Psychooncology, 16(12), 1069–1079. https://doi.org/10.1002/pon.1184.View ArticlePubMedGoogle Scholar
Ruland, C. M., Holte, H. H., Roislien, J., Heaven, C., Hamilton, G. A., Kristiansen, J., Sandbaek, H., Kvaloy, S. O., Hasund, L., & Ellison, M. C. (2010). Effects of a computer-supported interactive tailored patient assessment tool on patient care, symptom distress, and patients' need for symptom management support: A randomized clinical trial. J Am Med Inform Assoc, 17(4), 403–410. https://doi.org/10.1136/jamia.2010.005660.View ArticlePubMedPubMed CentralGoogle Scholar
Schuler, M. K., Trautmann, F., Radloff, M., Schmadig, R., Hentschel, L., Eberlein-Gonska, M., Petzold, T., Vetter, H., Oberlack, S., Ehninger, G., & Schmitt, J. (2016). Implementation of a mobile inpatient quality of life (qol) assessment for oncology nursing. Support Care Cancer, 24(8), 3391–3399. https://doi.org/10.1007/s00520-016-3163-9.View ArticlePubMedGoogle Scholar
Siekkinen, M., Pyrhonen, S., Ryhanen, A., Vahlberg, T., & Leino-Kilpi, H. (2015). Psychosocial outcomes of e-feedback of radiotherapy for breast cancer patients: A randomized controlled trial. Psychooncology, 24(5), 515–522. https://doi.org/10.1002/pon.3684.View ArticlePubMedGoogle Scholar
Snyder, C. F., Jensen, R. E., Geller, G., Carducci, M. A., & Wu, A. W. (2010). Relevant content for a patient-reported outcomes questionnaire for use in oncology clinical practice: Putting doctors and patients on the same page. Qual Life Res, 19(7), 1045–1055. https://doi.org/10.1007/s11136-010-9655-z.View ArticlePubMedGoogle Scholar
Snyder, C. F., Herman, J. M., White, S. M., Luber, B. S., Blackford, A. L., Carducci, M. A., & Wu, A. W. (2014). When using patient-reported outcomes in clinical practice, the measure matters: A randomized controlled trial. J Oncol Pract, 10(5), e299–e306. https://doi.org/10.1200/JOP.2014.001413.View ArticlePubMedPubMed CentralGoogle Scholar
Taenzer, P., Bultz, B. D., Carlson, L. E., Speca, M., DeGagne, T., Olson, K., Doll, R., & Rosberger, Z. (2000). Impact of computerized quality of life screening on physician behaviour and patient satisfaction in lung cancer outpatients. Psychooncology, 9(3), 203–213.View ArticleGoogle Scholar
Takeuchi, E. E., Keding, A., Awad, N., Hofmann, U., Campbell, L. J., Selby, P. J., Brown, J. M., & Velikova, G. (2011). Impact of patient-reported outcomes in oncology: A longitudinal analysis of patient-physician communication. J Clin Oncol, 29(21), 2910–2917. https://doi.org/10.1200/JCO.2010.32.2453.View ArticlePubMedGoogle Scholar
Velikova, G., Keding, A., Harley, C., Cocks, K., Booth, L., Smith, A. B., Wright, P., Selby, P. J., & Brown, J. M. (2010). Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: Secondary outcomes of a randomised controlled trial. Eur J Cancer, 46(13), 2381–2388. https://doi.org/10.1016/j.ejca.2010.04.030.View ArticlePubMedGoogle Scholar
Whittle, A. K., Kalsi, T., Babic-Illman, G., Wang, Y., Fields, P., Ross, P. J., Maisey, N. R., Hughes, S., Kwan, W., & Harari, D. (2016). A comprehensive geriatric assessment screening questionnaire (cga-gold) for older people undergoing treatment for cancer. Eur J Cancer Care (Engl). https://doi.org/10.1111/ecc.12509.
Wolfe, J., Orellana, L., Cook, E. F., Ullrich, C., Kang, T., Geyer, J. R., Feudtner, C., Weeks, J. C., & Dussel, V. (2014). Improving the care of children with advanced cancer by using an electronic patient-reported feedback intervention: Results from the pediquest randomized controlled trial. J Clin Oncol, 32(11), 1119–1126. https://doi.org/10.1200/JCO.2013.51.5981.View ArticlePubMedPubMed CentralGoogle Scholar
Wolpin, S., Berry, D., Austin-Seymour, M., Bush, N., Fann, J. R., Halpenny, B., Lober, W. B., & McCorkle, R. (2008). Acceptability of an electronic self-report assessment program for patients with cancer. Comput Inform Nurs, 26(6), 332–338. https://doi.org/10.1097/01.NCN.0000336464.79692.6a.View ArticlePubMedPubMed CentralGoogle Scholar
Wright, P., Smith, A., Roberts, K., Selby, P., & Velikova, G. (2007). Screening for social difficulties in cancer patients: Clinical utility of the social difficulties inventory. Br J Cancer, 97(8), 1063–1070. https://doi.org/10.1038/sj.bjc.6604006.View ArticlePubMedPubMed CentralGoogle Scholar
Jensen, R. E., Snyder, C. F., Abernethy, A. P., Basch, E., Potosky, A. L., Roberts, A. C., Loeffler, D. R., & Reeve, B. B. (2014). Review of electronic patient-reported outcomes systems used in cancer clinical care. J Oncol Pract, 10(4), e215–e222. https://doi.org/10.1200/JOP.2013.001067.View ArticlePubMedGoogle Scholar
Luckett, T., Butow, P. N., & King, M. T. (2009). Improving patient outcomes through the routine use of patient-reported data in cancer clinics: Future directions. Psychooncology, 18(11), 1129–1138. https://doi.org/10.1002/pon.1545.View ArticlePubMedGoogle Scholar