Skip to main content
Journal of Patient-Reported Outcomes
Download PDF
Download ePub

An implementation study of electronic assessment of patient-reported outcomes in inpatient radiation oncology

Journal of Patient-Reported Outcomes volume 6, Article number: 77 (2022) Cite this article

Abstract

Purpose

Despite evidence for clinical benefits, recommendations in guidelines, and options for electronic data collection, routine assessment of patient-reported outcomes (PROs) is mostly not implemented in clinical practice. This study aimed to plan, conduct and evaluate the implementation of electronic PRO (e-PRO) assessment in the clinical routine of an inpatient radiation oncology clinic.

Methods

The guideline- and evidence-based, stepwise approach of this single-center implementation study comprised preparatory analyses of current practice, selection of assessment instruments and times, development of staff training, and evidence-based recommendations regarding the use of the e-PRO assessment, as well as on-site support of the implementation. Process evaluation focused on potential clinical benefit (number of documented symptoms and supportive measures), feasibility and acceptance (patient contacts resulting in completion/non-completion of the e-PRO assessment, reasons for non-completion, preconditions, facilitators and barriers of implementation), and required resources (duration of patient contacts to explain/support the completion).

Results

Selection of instruments and assessment times resulted in initial assessment at admission (EORTC QLQ-C30, QSR 10), daily symptom monitoring (EORTC single items), and assessment at discharge (EORTC QLQ-C30). Recommendations for PRO-based clinical action and self-management advice for patients concerning nine core symptoms were developed. Staff training comprised group and face-to-face meetings and an additional e-learning course was developed. Analyses of clinical records showed that e-PRO assessment identified more symptoms followed by a higher number of supportive measures compared to records of patients without e-PRO assessment. Analysis of n = 1597 patient contacts resulted in n = 1355 (84.9%) completed e-PROs (initial assessment: n = 355, monitoring: n = 967, final assessment: n = 44) and n = 242 (15.2%) non-completions. Instructions or support to complete e-PROs took on average 5.5 ± 5.3 min per patient contact. The most challenging issue was the integration of the results in clinical practice.

Conclusion

E-PRO assessment in oncologic inpatient settings is acceptable for patients and can support symptom identification and the initiation of supportive measures. The challenge of making the “data actionable” within the clinical workflow and motivating clinical staff to use the results became evident.

Plain English summary

Cancer patients’ perceptions regarding their symptoms and functioning are important as they can differ from a professional assessment. Patients’ perceptions and self-assessment can be collected via electronic devices. Thus, the clinical staff can see a graphic overview of individual disease-related burden. Despite studies indicating the benefit of this assessment for care and symptom management, it is not integrated into routine care so far. The aim of our study was, to plan, conduct and evaluate the implementation of electronic patient-reported assessment in a radio-oncology inpatient clinic under “real-life” clinical conditions instead of study conditions. Patients could complete an electronic assessment at the beginning/end and during their treatment. Results indicate that electronic self-assessment can identify more symptoms than the assessment of physicians and nurses. Patients completing a self-assessment are more likely to receive supportive measures. The majority of 80–90% of patients were willing to complete a self-assessment. On average 5–6 min were needed to explain or support the completion. While the intervention was feasible and acceptable for patients, motivating clinical staff using its results was most challenging. The importance of technical support became evident.

Background

Patients’ perceptions of cancer or therapy-related burden are subjective and can differ from healthcare professionals (HCPs) perceptions [1, 2]. Therefore, routine assessment of patient-reported outcomes (PROs) is recommended to systematically capture individual perceptions of health conditions directly from patients, such as health-related quality of life (HRQOL) including functionality and symptoms [3, 4]. Studies have shown that PROs complementing routine assessment are a feasible intervention and acceptable for patients as well as HCPs. They can support communication and the integration of patient perspectives in clinical decision-making, treatment, and care, thus improving patient satisfaction and symptom management [5,6,7,8]. In a large randomized controlled trial, PRO assessment was associated with fewer unplanned hospital admissions and increased survival [9]. International guidelines and organizations for cancer care recommend the integration of PROs into clinical practice [10,11,12,13]. Electronic data capture with mobile devices can facilitate clinical implementation due to variable modes of access, computerized evaluation, and graphic display of results [14].

Despite this evidence, systematic PRO assessment is still not integrated into most oncological settings [15, 16], indicating that there is a large difference between studies favoring its feasibility and acceptability and actual clinical practice. Most studies focus on evaluating PROs within study conditions [17], including the recruitment of predefined participant groups for a certain period of time supported by additional personal and time resources. While study conditions are useful for developing and pretesting PRO-based interventions, they may reflect clinical practice not sufficiently enough to gain adequate knowledge regarding long-term integration in “real-life” application contexts. The implementation of systematic PRO assessment into practice is complex and challenging because of the number of stakeholders involved, organizational and financial aspects, and the need to change clinical routines [10]. Major barriers are the reluctance of organizations and HCPs to change established practice due to reservations regarding PROs, the lack of resources (time, staff, finances), and data protection or IT issues [18,19,20,21]. Existing guidelines provide general recommendations and advice for the implementation of PRO assessment into clinical practice [10, 11, 22]. However, when aiming for a successful implementation process, specific facilitators and barriers for each setting, current clinical practice, attitudes of all relevant stakeholders (e.g. presumptions and concerns), and the potential benefit of PROs should be explored and addressed [20, 23, 24]. Targeted training improving knowledge and competencies can facilitate the clinical use of PRO assessment [10, 25, 26]. Following these recommendations and guidelines this study aims to plan, conduct and evaluate the implementation of electronic PRO (e-PRO) assessment in inpatient clinical oncology. Addressing a lack of implementation studies regarding long-term applications in the clinical routine [17], our approach focuses on gaining comprehensive and quantifiable insight into the experiences of the implementation process under “real-life” clinical conditions by using a reproducible descriptive method of evaluation. Our methods and results can be used as a basis to further develop and adapt implementation and evaluation strategies in other oncology settings. The following research questions are addressed:

  • Which necessary preconditions, barriers, and facilitators can be identified?

  • Which are the potential benefits of e-PRO assessment regarding symptom management?

  • How feasible and acceptable is e-PRO assessment?

  • Which resources are needed for an e-PRO assessment?

Methods

Study design and setting

Aiming to examine preconditions and feasibility and to gain experiences and insights for the planning of broader implementation in other departments of the Krukenberg Cancer Center, University Hospital Halle (Saale), a single-center design was chosen. Because of the heterogeneity of diagnoses and treatments which include both chemotherapy and radiotherapy representing a broad spectrum of cancer treatment and care, the inpatient clinic of the radiation oncology department, comprising 34 beds, was chosen for this implementation.

The majority of radiotherapy in Germany is delivered in an outpatient setting. The reasons for inpatient radiotherapy are predominantly the delivery of concomitant chemoradiation (in particular the days of chemotherapy delivery, during a series of radiotherapy) and poor general condition with the need for intensive supportive therapy, e. g. intravenous medication or palliative therapy [27].

Stepwise approach

Due to the complexity of the implementation process, the Medical Research Council (MRC) framework for the development and evaluation of complex interventions [28] guided the approach of this study, containing the three steps development, implementation and evaluation. In addition, to model the procedure specifically for e-PRO assessment, the Manual of the European Organisation of Research and Treatment of Cancer (EORTC) for the use of EORTC measures in clinical practice [10] and evidence regarding relevant aspects like facilitators and barriers of PRO assessment were taken into account [5, 20, 23, 24].

The three steps and the procedure within are described following.

Development

Aiming to provide information on current practice, a purposive sample of n = 15 patient records comprising various cancer diagnoses and treatments was analyzed focusing on routinely documented PROs. Participating observation of the clinical practice of physicians and nurses comprised protocols of routine processes and clinical workflow, potential facilitators, barriers, and further aspects relevant for implementation. A self-developed, semi-structured survey of HCPs working in the radiation oncology department was conducted to explore their knowledge and views on e-PRO assessment. The survey consisted of six items regarding the understanding of HRQOL and attitudes about the potential of e-PRO assessment for supporting cancer care and personal practice, possible effects on the personal workload, usability, and acceptance or burden for patients. Each item could be answered via bipolar Likert scale (Yes, rather yes, rather no, no) with an additional, seperate option for “don’t know” and a free-text field.

Aiming to tailor the set-up of e-PRO assessment to the setting of radiation oncology, a multi-professional focus group with health care professionals (HCPs, including physicians, nurses, psycho-oncology) from the department was conducted to discuss the clinical procedure, decide on assessment instruments and times, and the requirements for the assessment software [23]. Minutes were taken and the decisions consented.

Interdisciplinary group-based training sessions for HCPs, focusing on the use of e-PRO assessment in clinical practice and the discussion of implementation issues and concerns, were developed and conducted, complemented by on-site face-to-face training regarding the technical use. To ensure sustainability and facilitate training of new HCPs, an e-learning course was developed, adapted, and finalized throughout the implementation process allowing the individual choice of timing and content according to the learning needs of the users.

In addition, after first experiences with the e-PRO assessment, the desire for easily accessible and structured guidance on how to react to reported symptoms was expressed. To meet this need, clinical recommendations for relevant symptoms were developed aiming to guide clinical consequences of e-PRO results [29].

Implementation

According to the study aims of implementation of e-PRO assessment in clinical routine including assessment of potential benefit, feasibility and barriers under routine conditions, in accordance with a declaration of no-objection by the ethics committee of the Medical Faculty, Martin-Luther-University Halle-Wittenberg (declaration from 08.01.2019, retrospectively assigned process number: 2022-061), no informed consent of patients was obtained to avoid a possible selection bias. Thus, the collection of e-PROs could be offered to all inpatients of the radio-oncology clinic except patients with severe cognitive impairment. All patient contacts to explain and/or support the completion of the e-PRO assessments were conducted by the study staff.

The implementation process consisted of the following three phases:

  • The initial phase focused on technical adjustments to achieve full functionality of the assessment software in connection with the clinical information system and electronic patient records.

  • The consolidation phase focused on optimized integration of e-PRO assessment into clinical routine and requirements for further adaptation to enhance clinical use of the e-PRO assessments and patients’ self-assessments.

  • The routine phase focused on constant data flow and analyses of necessary resources aiming to further facilitate the integration into clinical practice.

The implementation was accompanied by on-site monitoring in all three phases, resulting in process adaptation and optimization. Emerging issues were addressed in team meetings.

Evaluation

The software CHES (Computer-based Health Evaluation System) [30] was used to collect e-PROs, including scoring, compilation, and graphic presentation of the results within the clinical records.

Data collection for the evaluation focused on potential clinical benefit, feasibility/acceptance, and required resources. Potential clinical benefit was operationalized through comparison of the number of recorded symptoms (e-PROs vs. documentation of physicians and nurses) and initiated supportive measures (patients without vs. patients’ completion of e-PROs) for three samples of n = 100 each:

  • Sample 1: Without the opportunity for e-PRO assessment.

  • Sample 2: With e-PRO assessment, without integration of results into clinical records.

  • Sample 3: With e-PRO assessment, with the integration of results into clinical records.

These samples were extracted randomly through the automatized allocation of numbers for e-PRO assessment at therapy start (IA1, IA2, and IA3) and during the course of therapy (SM1, SM2, and SM3). The e-PRO results as well as sociodemographic and medical characteristics were extracted automatically from CHES, recorded symptoms and initiated supportive measures were extracted manually from the clinical documentation based on defined assignment criteria.

Furthermore, all patient contacts from the beginning of the consolidation phase were documented via standardized protocols by the study staff. Feasibility and acceptance were operationalized through the percentage of contacts resulting in completion or non-completion of the e-PRO assessment, characteristics of patients completing/not completing, and reasons for non-completion. Required resources were operationalized through the time required for patient contacts to explain and/or support the completion. Due to ethical requirements, these protocols comprised only anonymized data.

Notes were taken to document on-site monitoring, face-to-face coaching, and team meetings with a focus on relevant issues, e.g. facilitators or occurring barriers during the implementation process.

For data analysis, quantitative data were extracted or transferred into SPSS Version 25 and analyzed descriptively. Descriptive statistics covered frequency, mean-values, and standard deviation, minimum and maximum. All data were anonymized and aggregated for analysis and reporting. Notes were summarized narratively.

Since the study was not designed to examine or prove hypotheses, no inferential statistical analyses were performed.

Results

Analysis of current practice

The review of clinical records revealed a routine assessment of emotional burden (QSC-R10 [31]), nutrition (NRS [32]), and dichotomous questions for dyspnea, insomnia, constipation, and diarrhea at admission. Pain was assessed three times a day via a numeric rating scale (0–10). Documentation of other symptoms was not standardized.

Participating observation led to the identification of the following facilitators for the implementation: regular admission times and existing technical equipment with electronic clinical records, individual touchscreens next to each bed, and computer-supported ward rounds. Potential barriers comprised concerns of HCPs (e.g. doubts regarding patients’ ability to complete e-pro assessments) and time constraints due to temporary staff shortages.

The survey was completed by 20 out of 30 HCPs (67%), i.e. n = 12 (60%) physicians, n = 7 (35%) nurses and n = 1 (5%) case manager. All participants reported knowledge about the concept of PROs (n = 20, 100%), and the majority considered e-PRO assessment a contribution for improving cancer care (n = 18, 90%) or supporting their own work (n = 19, 95%). Twelve (60%) expected increased workload due to e-PRO assessment, n = 10 (50%) doubted or were unsure regarding easy use by patients, and n = 8 (40%) considered e-PRO assessment a potential burden for patients.

Set-up of e-PRO assessment

The focus group decided on an initial assessment of HRQOL at admission with the EORTC QLQ-C30 [33], complemented by the assessment of emotional burden using the QSC-R10 which was already part of the clinical routine as a paper–pencil version. During therapy, daily monitoring of eleven core symptoms with single items from the EORTC Item Library was planned [34]. In addition, specific symptoms of the six most frequent cancer sites treated in the clinic (lung, head-neck, colorectal, brain, breast, prostate) with 3–8 single items were included, allocated automatically based on the individual ICD-10 diagnosis. Before the end of the treatment, the completion of a final HRQOL assessment was planned (EORTC QLQ-C30).

E-PROs were assessed either via tablets or individual devices located next to each patient’s bed working as an entertainment system. The technical process of the e-PRO assessment is described in Additional file 1. Secure access to the bedside devices was provided through chip cards for individual identification. Data entry via touchscreen and automatic progression to the next question allowed easy use even for patients without prior IT experience (Fig. 1).

Fig. 1
figure 1

Overview of the assessment instruments and clinical procedure

Full size image

Graphic displays of the e-PRO results were made available in real-time in the electronic patient records including bar charts for a single assessment time (cross-sectional) or all available assessment times for a defined period (longitudinal e.g. monitoring during hospital stay). The color-coded graphic display allows intuitive interpretation of the results (e.g. red = severe burden); the respective scales include a mouseover option to access the scores. Via mouse click all underlying questions are displayed (Fig. 2). The color-coded categorization was based on the thresholds by Giesinger et al. [35] or on equal tripartition in case there were no scientifically based thresholds available for the chosen instrument.

Fig. 2
figure 2

Example for a cross-sectional (top-left) and longitudinal display (top-right) of results including single questions (bottom)

Full size image

Training of healthcare professionals

The group and on-site training covered basic knowledge such as the definition of PROs and clinical benefits, the process of e-PRO assessment, technical aspects and use of the system in connection with the electronic patient records, interpretation of results, PRO-based patient-communication, and supportive measures including counseling and self-management recommendations for patients.

The content of the e-learning course was presented mainly through written information and graphics (e.g. pictures, illustrations, flowcharts, partially interactive), complemented by interactive methods and quiz options to assess the learning progress. To demonstrate the integration of e-PROs in clinical situations and communication, videos with HCPs and actor-patients complemented the program.

Development of clinical recommendations

Guideline-based [36, 37], multi-professional recommendations for HCP were developed for a core set of nine symptoms (fatigue, nausea/vomiting, emotional burden, pain, sleep problems, appetite loss, constipation, diarrhea, skin problems). The recommendations included suggestions for PRO-based communication to gain additional information (e. g. questions to help patients specify their symptoms), further diagnostics (e. g. differential diagnosis), supportive measures, and patient consultations regarding measures of self-management.

In addition, patient self-management recommendations for these symptoms were developed and made available as handouts. These recommendations included suggestions for PRO-based self-observation and general or specific supportive measures patients can conduct independently.

Implementation

Technical adaptation during the initial phase comprised the optimization of functionality within the clinical IT (e. g., not all inpatients appeared in CHES) and the activation of CHES for the devices next to the patients’ beds. A secure port between CHES and the electronic patient records was created enabling HCPs to access the results without entering an additional password. The QSC-R10 was transferred subsequently into CHES.

During the consolidation phase, the e-PRO assessment results were integrated into the electronic patient records to further simplify their access. To enhance visibility, new results were highlighted in the same way as new medical reports for physicians. Some routine procedures such as ward rounds were accompanied by the first author presenting the results to encourage the use of the results of the assessment.

Within the routine phase, we aimed to facilitate the implementation by generating a constant data flow. Five research assistants were employed to guarantee a daily time slot of one person instructing the patients and documenting the patient contacts.

Results of the initial assessment and symptom monitoring

During the data collection period between 02/2019 and 02/2021, 568 patients (n = 201 female, n = 367 male) with heterogeneous cancer diagnoses and a mean age of 64.9 years (± 11.6), completed the initial assessment. Comprehensive sociodemographic and clinical characteristics of the sample are presented in Additional file 2, the complete results of the initial assessment with the EORTC QLQ-C30 in Additional file 3.

More than 50% of the patients reported moderate or severe burden on the EORTC QLQ-C30 (scales 0–100, thresholds for severe burden based on Giesinger et al. [35], thresholds for moderate burden on nomal values of EORTC QLQ-C30 scales in the general population [38]) for the following scales: fatigue (n = 451, 79.4%), physical function (n = 412, 72.5%), global quality of life (n = 386, 68.0%), pain (n = 353, 62.2%), insomnia (n = 342, 60.2%), emotional function (n = 312, 54.9%) and dyspnea (n = 296, 52.1%). The QSC-R10 was completed by fewer patients (n = 472) since the electronic assessment was added to the initial assessment during the study. A total of n = 109 (23.1%) stated a high emotional burden ≥ 15 points. From a total of 511 completing patients, n = 106 (20.7%) expressed the need for psycho-oncological support and n = 47 (9.2%) were already receiving support at the beginning of their therapy.

The daily symptom monitoring resulted in 1774 assessment time-points from 344 patients, mean age 63.7 (± 11.0, n = 128 female, n = 216 male). Comprehensive sociodemographic and clinical characteristics of the sample are given in Additional file 4, complete results of the symptom monitoring in Additional file 5.

Patients completed the monitoring questions repeatedly depending on their length of stay and compliance (1–36 times). The symptoms with a moderate or severe burden (scales 0–100, thresholds based on tripartition since there are no scientifically based thresholds for EORTC single items) rated at more than half of all assessment times were tiredness (n = 1306, 73.6%), weakness (n = 1083, 61.1%), and insomnia (n = 989, 55.8%).

Potential clinical benefit of initial assessment and symptom monitoring

Regarding the initial assessment, the three samples (IA0, IA1, IA2) had similar sociodemographic and clinical characteristics (Additional file 6). In comparison with clinical records, the initial assessment with e-PROs captured more symptoms. The smallest differences were found for symptoms that were already part of the standard documentation (pain, insomnia), the largest for fatigue, appetite loss, and financial difficulties. Results of the comparison of clinical records and the initial assessment regarding the number of identified symptoms are summarized in Table 1.

Table 1 Comparison of clinical records and initial assessment regarding the number of identified symptoms (n = 100/sample)
Full size table

The number of initiated supportive measures was lowest in the sample without initial assessment (IA0) followed by the sample with initial assessment (IA1) and highest in the sample where initial assessment results were represented in the clinical records (IA2). Results of the comparison of samples without and with an initial assessment regarding the number of initiated supportive measures are summarized in Table 2.

Table 2 Comparison of samples without/with initial assessment regarding the number of initiated supportive measures (n = 100/sample)
Full size table

Regarding symptom monitoring, the three samples (SM0, SM1, SM2) had similar sociodemographic and clinical characteristics (Additional file 7). In comparison with the clinical records, the monitoring identified many additional symptoms. Results of the comparison of clinical records and the symptom monitoring regarding the number of identified symptoms are summarized in Table 3.

Table 3 Comparison of clinical records and symptom monitoring regarding the number of identified symptoms (n = 100/sample)
Full size table

The number of initiated supportive measures was lowest in the sample without symptom monitoring (SM0) followed by the sample with symptom monitoring (SM1) and highest in the sample where monitoring results were represented in the clinical records (SM2). Results of the comparison of samples without and with an initial assessment regarding the number of initiated supportive measures are summarized in Table 4.

Table 4 Comparison of samples without/with symptom monitoring regarding the number of initiated supportive measures (n = 100/sample)
Full size table

Feasibility and acceptance

Not all patients could be approached. Barriers were time constraints of the patients (e.g. owing to scheduled treatments) resulting in a mismatch between patients’ availability and the availability of the research assistants. The COVID-19 pandemic resulting in the necessary adaptation of clinical routines caused further barriers.

If necessary (e.g. need for continuous support or reminder), some patients were approached repeatedly and the contacts documented accordingly. With respect to their autonomy and to reduce burden, patients who firmly stated their unwillingness to complete the e-PRO assessment at the first contact were not approached again.

Between 11/2019 and 02/2021, on 179 days a total of 1597 patient contacts were documented. Out of these contacts, n = 1355 (84.9%) resulted in the completion of one e-PRO assessment. Regarding the initial assessment, 436 contacts resulted in n = 355 (81.4%) completions; regarding symptom monitoring, 1117 contacts resulted in n = 967 (86.6%) completions; and regarding final assessment, which was introduced later in the process, 44 contacts resulted in n = 33 (75.0%) completions.

Depending on their actual health condition and their familiarity with electronic devices, patients could complete the e-PRO assessment on their own or with support from research assistants. Despite their ability to complete the assessment independently, many patients preferred the support (e.g. completing together with research assistants), appreciating the personal contact and communication. Regarding sex, age, and diagnosis of the patients, contacts resulting in non-completion were comparable to contacts resulting in completion (Additional file 8).

On average, more than one-third of the total number of all in-patients completed one e-PRO assessment every day. The main reasons for non-completion were health-related (e.g. high symptom burden, mild cognitive impairment) or lacking motivation and perception of benefit. Almost half of the reasons were related to specific conditions (e.g. forgetting to complete, other priorities) instead of a general refusal of the e-PRO assessment (Table 5).

Table 5 Reasons for non-completion of the e-PRO assessment (n = 255 reasons stated in n = 235 patient contacts)
Full size table

Barriers comprised technical and organizational issues e.g. Wi-Fi problems, interruption of the assessment completion (e.g. due to diagnostics, therapy), staff turnover as well as HCPs lack of time, doubts regarding the benefit of e-PROs, or a lack of use of the results by the majority of the clinical staff. The latter barriers were addressed during on-site and in team meetings supporting the implementation process.

Required resources

For all 1354 patient contacts resulting in the completion of an e-PRO assessment, data regarding the duration were documented. On average 5.5 ± 5.3 (1–60) minutes per contact were needed to explain or support the completion of an e-PRO assessment.

On average, 11.4 ± SD 6.8 (1–60) minutes were required for supporting the initial assessment (354 analyzed patient contacts), 3.3 ± 2.0 (1–17) minutes for supporting symptom monitoring (967 analyzed patient contacts), and 6.2 ± 3.8 (1–15) minutes for supporting the final assessment (33 analyzed patient contacts).

Discussion

This study focused on the implementation of e-PRO assessment into the routine practice of a radiation oncology clinic. So far, only a few pilot studies investigated a repeated assessment of e-PROs in an oncologic inpatient setting [39, 40]. Our study was conducted under “real-life” clinical conditions instead of study conditions to achieve results most representative for routine practice.

Comparison with clinical records indicated a large potential clinical benefit of e-PRO assessment regarding symptom identification. As in other studies [41], the assessments revealed an overall high symptom burden before and during treatment [5, 6]. While identifying many more issues compared to the documentation of clinical staff, e-PRO assessment seems to be most beneficial for detecting less “visible” symptoms such as fatigue or tiredness and often overlooked problems like financial difficulties, all of which can have a large impact on HRQOL [42, 43]. As in previous studies [5, 8], results also indicate the potential of e-PRO assessment to improve symptom management by leading to the initiation of more supportive measures. The integration into the electronic clinical records appears to further increase the awareness of the symptoms and the initiation of supportive measures.

The results provide comprehensive insight into the potential and requirements of e-PRO assessment complementing routine documentation. It became evident that the complex process of implementation requires flexibility, continuous optimization, and evaluation to adapt the procedure based on the knowledge gained throughout the process. In line with the EORTC manual on the use of EORTC measures in clinical practice [44], the importance of initial analysis of setting and current practice became evident for assessment of the general probability of a successful implementation. This includes also technical preconditions and integrability of the e-PRO assessment into clinical workflows.

Process evaluation indicated good feasibility, although the technical integration into the electronic patient records was challenging, a finding similar to other studies [45, 46]. As the support of the clinical IT turned out to be a crucial factor for successful integration, future implementations should consider an adequate amount of resources for IT personnel for a fast realization. Aiming to use existing ressources effectively, a multicenter implementation should start with a single center to realize the complex technical integration, gain experience, and make adaptations that can be used by the following centers. The basic e-PRO assessment implemented in inpatient radiation oncology will be easily adaptable for other inpatient clinics of the Krukenberg Cancer Centre considering setting-specific needs, patient characteristics and outcomes of relevance. An implementation into outpatient settings may require adapted approaches according to the different workflows. In a follow-up project for the outpatient radiation oncology clinic, e-PROs will be assessed via mobile tablets during the waiting times for therapy/consultation or from home via secure individualized access for patients with respective knowledge and technology.

Previous studies report cancer patients' willingness and ability to complete PROs regularly via electronic devices [5, 47] as relevant aspects for successful implementation [39, 40]. The results of this study confirm these findings and show an overall good acceptance. However, to maintain motivation, patient-reported issues must be addressed by HCPs and tailored measures initiated as patients have to recognize the consequences of their self-assessments [8, 29].

Instruments and procedures should also be acceptable for HCPs to facilitate the implementation into routine care [23, 24, 26]. To generate clinical benefit, the main challenge is the integration into the clinical workflow i.e. making “data actionable” [44]. In this respect, motivating HCPs to use the results in their routine practice seems to be most challenging. The main reason may be that changes in established clinical practice disrupt the existing "status quo", as they require questioning old beliefs and learning of new practices within existing contextual factors [48, 49]. Similar to other efforts to change established routines e.g. in quality improvement [50,51,52], the implementation and use of e-PRO assessment represents a multifaceted change process of existing systems and routines, influenced by the relationships between patients and clinical staff, their roles and structural conditions [53]. The experiences of our study suggest, that this complex process requires additional efforts concerning the motivation of all relevant stakeholders and the adjustment of overexpectations [54] as those changes need time.

Unlike other research from Germany [55], in our study HCPs only roughly self-estimated their familiarity with the concept of PROs as the survey was intended to gain basic knowledge regarding potential training needs. While a lack of basic knowledge regarding PROs may be a barrier for implementation, results suggest that even familiarity with the concept does not guarantee the use of PROs and their results in clinical practice. To enhance acceptance and motivation, we included HCPs in the planning process and decisions on instruments and clinical procedures [23], provided targeted training and recommendations for clinical use [56], and discussed relevant issues in team meetings [10]. However, these measures still appeared to be insufficient to ensure routine use of e-PRO assessments. As the results indicate improved symptom management already with few team members using the results of the e-PRO assessment, routine use by the whole multi-professional team may further increase the clinical benefit. Particularly motivated HCPs acting as “change champions” [46] could facilitate this development since their status as part of the clinical team may be more persuasive than external motivation. Furthermore, systematically developed targeted training for HCPs is needed regarding benefits of e-PRO assessment, PRO-based communication, PRO-based action with supportive measures, and self-management advice [26, 56,57,58]. Within this study, an e-learning course was developed but not yet used to educate HCPs. The future application may increase HCPs’ awareness and motivation to utilize the results of the e-PRO assessment.

To our knowledge, this is the first study evaluating the duration of patient contacts to explain and/or support the completion of e-PRO assessments, estimating necessary resources for routine implementation. Results indicate on average an additional workload of 5 to 6 min per patient to complete any e-PRO assessment, resulting in 150 to 180 additional minutes per day for 30 inpatients. As studies report that the integration of PROs did not increase the required time for consultations [47, 58], the explanation and/or support may be the most time-consuming aspect of the application of e-PRO assessment in clinical practice. However, due to our study design and limited resources of regular staff, this support was provided by the study staff. It might be expected that integration into the clinical routine of HCPs caring for the patients (e.g. reminding or supporting patients during ward rounds) would require less time. In addition, better symptom identification leading to more supportive measures, as indicated in this study, has the potential of reducing workload due to the prevention of adverse events, e.g. the risk of readmission [39]. Studies show that PROs are independent prognostic factors for important clinical outcomes such as survival [59, 60] and can therefore be used as decision aids when planning or adapting individual cancer therapy. The potential use of e-PRO assessment regarding the prevention of adverse events and optimization of therapy might be also beneficial for cost-efficacy in healthcare [61, 62].

Limitations of this implementation study are the single-center design and the inclusion of research assistants in the process. The majority of the patient contacts took place during the COVID-19 pandemic, resulting in extraordinary conditions e.g. lower number of admissions, only one patient per room, quarantined patients, different workflows/priorities for the clinical staff, and the need for interrupting the implementation process completely over several months. As described in another study [63], these pandemic conditions worked as a massive barrier by hindering and interrupting the implementation process, potentially reducing the acceptance of patients as well as staff and therefore preventing the e-PRO assessment to reach its full potential regarding potential benefit. Due to the anonymized documentation, contacts instead of persons were analyzed and the data have to be interpreted accordingly. A strength of this study is the combined evaluation of clinical benefit, feasibility/acceptance, and required resources of e-pro assessment based on a large number of analyzed patients and patient contacts, aiming to come as close to routine clinical conditions as possible. Our study did not aim for generalizability of the results but for a comprehensive evaluation of the complex and challenging implementation of e-PROs under routine conditionions to gain experience and knowledge for future implementations. With regard to setting-specific characteristics that may result in different symptoms and supportive measures of relevance, the methodologial approach, focusing on automated and criteria-based data extraction as well as proctocolled patient contacts, can be reproduced in other studies to systematically evaluate the implementation process.

Future research should focus on long-term implementations in different oncologic settings, apply a multi-center design starting with a pilot center and aim for routine clinical conditions to examine e-PRO assessment in “real-life” application contexts. Scientifically based implementation concepts [50,51,52] and models could be adapted and specified for this application to further enhance and facilitate the implementation of e-PRO assessment. Also, targeted educational interventions to use e-PRO-assessment [56], including communication strategies for the integration of e-PRO results into patient consultations [57] should be developed. From a technical perspective, the optimization of the user-friendliness of existing e-PRO assessment systems may facilitate their usability in the clinical routine.

Conclusion

E-PRO assessment in an oncologic inpatient setting can support symptom identification and documentation by capturing additional self-reported burden of patients compared to the documentation of HCPs. Furthermore, it may lead to more supportive measures, especially if the results are easily visible due to their integration into the electronic patient records. While a repeated completion of e-PRO assessment appears to be acceptable for many patients, the challenge of making data “actionable” within the clinical workflow and motivating clinical staff became evident. The implementation of e-PRO assessment can be regarded as a complex process of permanent adaptation and optimization focusing on the integrability into existing technical systems and clinical workflows. In this context, the time needed to explain or support the completion of the assessment also has to be considered.

Availability of data and materials

The scope of this implementation study was to improve the quality of care in clinical routine through the implementation of e-PRO assessment focusing on the evaluation of the process under “real-life” conditions and not under clinical trial conditions. In accordance with the ethics committee, no informed consent of patients completing the e-PRO assessment was obtained. Therefore, due to data protection and ethical regulations, the anonymized datasets generated or analyzed during the study cannot be provided.

Abbreviations

CHES:

Computer-based Health Evaluation System

e-PRO(s):

Electronic patient reported outcome(s)

EORTC:

European Organisation of Research and Treatment of Cancer

HCP(s):

Healthcare professional(s)

HRQOL:

Health-related quality of life

MRC:

Medical Research Council

PRO(s):

Patient-reported outcome(s)

References

  1. Atkinson TM, Ryan SJ, Bennett AV et al (2016) The association between clinician-based common terminology criteria for adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care Cancer 24:3669–3676. https://doi.org/10.1007/s00520-016-3297-9

    Article  PubMed  PubMed Central  Google Scholar 

  2. Bock D, Angenete E, Bjartell A et al (2019) Agreement between patient reported outcomes and clinical reports after radical prostatectomy—a prospective longitudinal study. BMC Urol 19:35. https://doi.org/10.1186/s12894-019-0467-3

    Article  PubMed  PubMed Central  Google Scholar 

  3. Higgins JPT, Thomas J, CHandler J et al (2021) Cochrane handbook for systematic reviews of interventions: version 6.2. https://training.cochrane.org/handbook/current. Accessed 07 Mar 2022

  4. Ramsey I, Eckert M, Hutchinson AD et al (2020) Core outcome sets in cancer and their approaches to identifying and selecting patient-reported outcome measures: a systematic review. J Patient Rep Outcomes. https://doi.org/10.1186/s41687-020-00244-3

    Article  PubMed  PubMed Central  Google Scholar 

  5. Kotronoulas G, Kearney N, Maguire R et al (2014) What is the value of the routine use of patient-reported outcome measures toward improvement of patient outcomes, processes of care, and health service outcomes in cancer care? A systematic review of controlled trials. J Clin Oncol 32:1480–1501. https://doi.org/10.1200/JCO.2013.53.5948

    Article  PubMed  Google Scholar 

  6. Chen J, Ou L, Hollis SJ (2013) A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Serv Res 13:211. https://doi.org/10.1186/1472-6963-13-211

    Article  PubMed  PubMed Central  Google Scholar 

  7. Ishaque S, Karnon J, Chen G et al (2019) A systematic review of randomised controlled trials evaluating the use of patient-reported outcome measures (PROMs). Qual Life Res 28:567–592. https://doi.org/10.1007/s11136-018-2016-z

    Article  CAS  PubMed  Google Scholar 

  8. Graupner C, Kimman ML, Mul S et al (2021) Patient outcomes, patient experiences and process indicators associated with the routine use of patient-reported outcome measures (PROMs) in cancer care: a systematic review. Support Care Cancer 29:573–593. https://doi.org/10.1007/s00520-020-05695-4

    Article  PubMed  Google Scholar 

  9. Basch E, Deal AM, Kris MG et al (2016) Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial. J Clin Oncol 34:557–565. https://doi.org/10.1200/JCO.2015.63.0830

    Article  CAS  PubMed  Google Scholar 

  10. EORTC (2016) EORTC quality of life group manual for the use of EORTC measures in daily clinical practice. Version 1. https://www.eortc.org/app/uploads/sites/2/2018/02/EORTC_QLQ_Clinical_Practice_User_Manual-1.0.pdf. Accessed 07 Mar 2022

  11. ISOQOL (2015) User’s Guide to implementing patient-reported outcomes assessment in clinical practice: version 2: January 2015. https://www.isoqol.org/wp-content/uploads/2019/09/2015UsersGuide-Version2.pdf. Accessed 07 Mar 2022

  12. Riba MB, Donovan KA, Andersen B et al (2019) Distress management, version 3.2019, NCCN clinical practice guidelines in oncology. J Natl Compr Cancer Netw 17:1229–1249. https://doi.org/10.6004/jnccn.2019.0048

    Article  Google Scholar 

  13. Hui D, Bohlke K, Bao T et al (2021) Management of dyspnea in advanced cancer: ASCO guideline. J Clin Oncol 39:1389–1411. https://doi.org/10.1200/JCO.20.03465

    Article  PubMed  Google Scholar 

  14. Bennett AV, Jensen RE, Basch E (2012) Electronic patient-reported outcome systems in oncology clinical practice. CA Cancer J Clin 62:337–347. https://doi.org/10.3322/caac.21150

    Article  PubMed  Google Scholar 

  15. Büttner M, Zebralla V, Dietz A et al (2017) Quality of life measurements: any value for clinical practice? Curr Treat Options Oncol 18:30. https://doi.org/10.1007/s11864-017-0470-4

    Article  PubMed  Google Scholar 

  16. Scheibe M, Herrmann A, Schmitt J et al (2020) Implementation of patient-reported outcome assessment in routine cancer care: a systematic review of multicentric programs in Europe. Z Evid Fortbild Qual Gesundhwes 156–157:11–23. https://doi.org/10.1016/j.zefq.2020.08.001

    Article  PubMed  Google Scholar 

  17. Anatchkova M, Donelson SM, Skalicky AM et al (2018) Exploring the implementation of patient-reported outcome measures in cancer care: need for more real-world evidence results in the peer reviewed literature. J Patient Rep Outcomes 2:64. https://doi.org/10.1186/s41687-018-0091-0

    Article  PubMed  PubMed Central  Google Scholar 

  18. Zbrozek A, Hebert J, Gogates G et al (2013) Validation of electronic systems to collect patient-reported outcome (PRO) data-recommendations for clinical trial teams: report of the ISPOR ePRO systems validation good research practices task force. Value Health 16:480–489. https://doi.org/10.1016/j.jval.2013.04.002

    Article  PubMed  Google Scholar 

  19. Smith KC, Brundage MD, Tolbert E et al (2016) Engaging stakeholders to improve presentation of patient-reported outcomes data in clinical practice. Support Care Cancer 24:4149–4157. https://doi.org/10.1007/s00520-016-3240-0

    Article  PubMed  Google Scholar 

  20. Antunes B, Harding R, Higginson IJ et al (2014) Implementing patient-reported outcome measures in palliative care clinical practice: a systematic review of facilitators and barriers. Palliat Med 28:158–175. https://doi.org/10.1177/0269216313491619

    Article  PubMed  Google Scholar 

  21. Duncan EAS, Murray J (2012) The barriers and facilitators to routine outcome measurement by allied health professionals in practice: a systematic review. BMC Health Serv Res 12:96. https://doi.org/10.1186/1472-6963-12-96

    Article  PubMed  PubMed Central  Google Scholar 

  22. Snyder CF, Aaronson NK, Choucair AK et al (2012) Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations. Qual Life Res 21:1305–1314. https://doi.org/10.1007/s11136-011-0054-x

    Article  PubMed  Google Scholar 

  23. Schmidt H, Merkel D, Koehler M et al (2016) PRO-ONKO-selection of patient-reported outcome assessments for the clinical use in cancer patients-a mixed-method multicenter cross-sectional exploratory study. Support Care Cancer 24:2503–2512. https://doi.org/10.1007/s00520-015-3055-4

    Article  PubMed  Google Scholar 

  24. Nic Giolla Easpaig B, Tran Y, Bierbaum M et al (2020) What are the attitudes of health professionals regarding patient reported outcome measures (PROMs) in oncology practice? A mixed-method synthesis of the qualitative evidence. BMC Health Serv Res 20:102. https://doi.org/10.1186/s12913-020-4939-7

    Article  PubMed  PubMed Central  Google Scholar 

  25. Santana MJ, Haverman L, Absolom K et al (2015) Training clinicians in how to use patient-reported outcome measures in routine clinical practice. Qual Life Res 24:1707–1718. https://doi.org/10.1007/s11136-014-0903-5

    Article  PubMed  Google Scholar 

  26. Yang LY, Manhas DS, Howard AF et al (2018) Patient-reported outcome use in oncology: a systematic review of the impact on patient-clinician communication. Support Care Cancer 26:41–60. https://doi.org/10.1007/s00520-017-3865-7

    Article  CAS  PubMed  Google Scholar 

  27. Medenwald D, Fietkau R, Klautke G et al (2021) Trends in radiotherapy inpatient admissions in Germany: a population-based study over a 10-year period. Strahlenther Onkol 197:865–875. https://doi.org/10.1007/s00066-021-01829-7

    Article  PubMed  PubMed Central  Google Scholar 

  28. Craig P, Petticrew M (2013) Developing and evaluating complex interventions: reflections on the 2008 MRC guidance. Int J Nurs Stud 50:585–587. https://doi.org/10.1016/j.ijnurstu.2012.09.009

    Article  PubMed  Google Scholar 

  29. Rosenbloom SK, Victorson DE, Hahn EA et al (2007) Assessment is not enough: a randomized controlled trial of the effects of HRQL assessment on quality of life and satisfaction in oncology clinical practice: A randomized controlled trial of the effects of HRQL assessment on quality of life and satisfaction in oncology clinical practice. Psycho Oncol 16:1069–1079. https://doi.org/10.1002/pon.1184

    Article  Google Scholar 

  30. ESD (2022) CHES components. https://ches.pro/project/ches-components/. Accessed 07 Mar 2022

  31. Book K, Marten-Mittag B, Henrich G et al (2011) Distress screening in oncology-evaluation of the Questionnaire on Distress in Cancer Patients-short form (QSC-R10) in a German sample. Psycho Oncol 20:287–293. https://doi.org/10.1002/pon.1821

    Article  Google Scholar 

  32. Kondrup J, Rasmussen HH, Hamburg O et al (2003) Nutritional risk screening (NRS 2002): a new method based on an analysis of controlled clinical trials. Clin Nutr 22:321–336. https://doi.org/10.1016/S0261-5614(02)00214-5

    Article  PubMed  Google Scholar 

  33. Aaronson NK, Ahmedzai S, Bergman B et al (1993) The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 85:365–376. https://doi.org/10.1093/jnci/85.5.365

    Article  CAS  PubMed  Google Scholar 

  34. EORTC (2022) EORTC Quality of Life Group Item Library. http://www.eortc.be/itemlibrary/#home/tab/official. Accessed 07 Mar 2022

  35. Giesinger JM, Loth FLC, Aaronson NK et al (2020) Thresholds for clinical importance were established to improve interpretation of the EORTC QLQ-C30 in clinical practice and research. J Clin Epidemiol 118:1–8. https://doi.org/10.1016/j.jclinepi.2019.10.003

    Article  PubMed  Google Scholar 

  36. Palliativmedizin AG (2017) Netzwerk-SOPs. http://www.ccc-netzwerk.de/netzwerk-sops/palliative-netzwerk-sops.html. Accessed 07 Mar 2022

  37. Leitlinienprogramm Onkologie (2017) S3 Leitlinie Supportive Therapie bei onkologischen PatientInnen: Langversion 1.1—April 2017. https://www.leitlinienprogramm-onkologie.de/fileadmin/user_upload/Downloads/Leitlinien/Supportivtherapie/LL_Supportiv_Langversion_1.1.pdf. Accessed 07 Mar 2022

  38. Lehmann J, Giesinger JM, Nolte S et al (2020) Normative data for the EORTC QLQ-C30 from the Austrian general population. Health Qual Life Outcomes 18:275. https://doi.org/10.1186/s12955-020-01524-8

    Article  PubMed  PubMed Central  Google Scholar 

  39. Nipp RD, El-Jawahri A, Ruddy M et al (2019) Pilot randomized trial of an electronic symptom monitoring intervention for hospitalized patients with cancer. Ann Oncol 30:274–280. https://doi.org/10.1093/annonc/mdy488

    Article  CAS  PubMed  Google Scholar 

  40. Bryant AL, Coffman E, Phillips B et al (2020) Pilot randomized trial of an electronic symptom monitoring and reporting intervention for hospitalized adults undergoing hematopoietic stem cell transplantation. Support Care Cancer 28:1223–1231. https://doi.org/10.1007/s00520-019-04932-9

    Article  PubMed  Google Scholar 

  41. Körner P, Ehrmann K, Hartmannsgruber J et al (2017) Patient-reported symptoms during radiotherapy. Strahlenther Onkol 193:570–577. https://doi.org/10.1007/s00066-017-1146-5

    Article  PubMed  Google Scholar 

  42. Smith GL, Lopez-Olivo MA, Advani PG et al (2019) Financial burdens of cancer treatment: a systematic review of risk factors and outcomes. J Natl Compr Cancer Netw 17:1184–1192. https://doi.org/10.6004/jnccn.2019.7305

    Article  Google Scholar 

  43. Butt Z, Rosenbloom SK, Abernethy AP et al (2008) Fatigue is the most important symptom for advanced cancer patients who have had chemotherapy. J Natl Compr Cancer Netw 6:448–455. https://doi.org/10.6004/jnccn.2008.0036

    Article  Google Scholar 

  44. Wintner LM, Sztankay M, Aaronson N et al (2016) The use of EORTC measures in daily clinical practice—a synopsis of a newly developed manual. Eur J Cancer 68:73–81. https://doi.org/10.1016/j.ejca.2016.08.024

    Article  PubMed  Google Scholar 

  45. Nguyen H, Butow P, Dhillon H et al (2021) A review of the barriers to using patient-reported outcomes (PROs) and patient-reported outcome measures (PROMs) in routine cancer care. J Med Radiat Sci 68:186–195. https://doi.org/10.1002/jmrs.421

    Article  PubMed  Google Scholar 

  46. van Egdom LSE, Oemrawsingh A, Verweij LM et al (2019) Implementing patient-reported outcome measures in clinical breast cancer care: a systematic review. Value Health 22:1197–1226. https://doi.org/10.1016/j.jval.2019.04.1927

    Article  PubMed  Google Scholar 

  47. Howell D, Molloy S, Wilkinson K et al (2015) Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol 26:1846–1858. https://doi.org/10.1093/annonc/mdv181

    Article  CAS  PubMed  Google Scholar 

  48. Gupta DM, Boland RJ, Aron DC (2017) The physician’s experience of changing clinical practice: a struggle to unlearn. Implement Sci 12:28. https://doi.org/10.1186/s13012-017-0555-2

    Article  PubMed  PubMed Central  Google Scholar 

  49. Myall M, May C, Richardson A et al (2020) Creating pre-conditions for change in clinical practice: the influence of interactions between multiple contexts and human agency. J Health Organ Manag. https://doi.org/10.1108/JHOM-06-2020-0240

    Article  PubMed  Google Scholar 

  50. Michie S, van Stralen MM, West R (2011) The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 6:42. https://doi.org/10.1186/1748-5908-6-42

    Article  PubMed  PubMed Central  Google Scholar 

  51. Skivington K, Matthews L, Simpson SA et al (2021) A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ 374:n2061. https://doi.org/10.1136/bmj.n2061

    Article  PubMed  PubMed Central  Google Scholar 

  52. Damschroder LJ, Aron DC, Keith RE et al (2009) Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 4:50. https://doi.org/10.1186/1748-5908-4-50

    Article  PubMed  PubMed Central  Google Scholar 

  53. Greenhalgh J, Gooding K, Gibbons E et al (2018) How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes 2:42. https://doi.org/10.1186/s41687-018-0061-6

    Article  PubMed  PubMed Central  Google Scholar 

  54. Dedehayir O, Steinert M (2016) The hype cycle model: A review and future directions. Technol Forecast Soc Chang 108:28–41. https://doi.org/10.1016/j.techfore.2016.04.005

    Article  Google Scholar 

  55. Sibert NT, Kowalski C, Pfaff H et al (2021) Clinicians’ knowledge and attitudes towards patient reported outcomes in colorectal cancer care - insights from qualitative interviews. BMC Health Serv Res 21:366. https://doi.org/10.1186/s12913-021-06361-z

    Article  PubMed  PubMed Central  Google Scholar 

  56. Sztankay M, Wintner LM, Roggendorf S et al (2021) Developing an e-learning course on the use of PRO measures in oncological practice: health care professionals’ preferences for learning content and methods. Support Care Cancer 30:2555–2567. https://doi.org/10.1007/s00520-021-06676-x

    Article  PubMed  PubMed Central  Google Scholar 

  57. Skovlund PC, Ravn S, Seibaek L et al (2020) The development of PROmunication: a training-tool for clinicians using patient-reported outcomes to promote patient-centred communication in clinical cancer settings. J Patient Rep Outcomes 4:10. https://doi.org/10.1186/s41687-020-0174-6

    Article  PubMed  PubMed Central  Google Scholar 

  58. Luckett T, Butow PN, King MT (2009) Improving patient outcomes through the routine use of patient-reported data in cancer clinics: future directions. Psycho Oncol 18:1129–1138. https://doi.org/10.1002/pon.1545

    Article  CAS  Google Scholar 

  59. Efficace F, Collins GS, Cottone F et al (2021) Patient-reported outcomes as independent prognostic factors for survival in oncology: systematic review and meta-analysis. Value Health 24:250–267. https://doi.org/10.1016/j.jval.2020.10.017

    Article  PubMed  Google Scholar 

  60. Mierzynska J, Piccinin C, Pe M et al (2019) Prognostic value of patient-reported outcomes from international randomised clinical trials on cancer: a systematic review. Lancet Oncol 20:e685–e698. https://doi.org/10.1016/S1470-2045(19)30656-4

    Article  PubMed  Google Scholar 

  61. Nixon NA, Spackman E, Clement F et al (2018) Cost-effectiveness of symptom monitoring with patient-reported outcomes during routine cancer treatment. J Cancer Policy 15:32–36. https://doi.org/10.1016/j.jcpo.2017.12.001

    Article  Google Scholar 

  62. Lizée T, Basch E, Trémolières P et al (2019) Cost-effectiveness of web-based patient-reported outcome surveillance in patients with lung cancer. J Thorac Oncol 14:1012–1020. https://doi.org/10.1016/j.jtho.2019.02.005

    Article  PubMed  Google Scholar 

  63. Patt D, Wilfong L, Hudson KE et al (2021) Implementation of electronic patient-reported outcomes for symptom monitoring in a large multisite community oncology practice: dancing the Texas two-step through a pandemic. JCO Clin Cancer Inform. https://doi.org/10.1200/CCI.21.00063

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Technical support was provided by the Department for Information and Communication of the University Hospital Halle (Saale), and Florian Santner, Evaluation Software Development. Clinical expertise regarding the review of the clinical and self-management recommendations was provided by Ursula Wolf (expert physician), Frank Sieker (expert physician) Kirsten Kusch (expert oncology nurse) and Janine Stingl (psychooncologist). Viktoria Badke de Guimet, Jo-Ella Bernardo, Marius Mix, Maxine Wirth and Kathleen Schüler supported the data collection as research assistants.

Funding

Open Access funding enabled and organized by Projekt DEAL. Purchase of the software tool (CHES) was funded by the European Regional Development Fund (ERDF). The first author received complementary funding through a grant for graduate students.

Author information

Authors and Affiliations

  1. Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

    Thomas Nordhausen, Gabriele Meyer & Heike Schmidt

  2. Department of Radiation Oncology, University Hospital Halle (Saale), Martin Luther University Halle-Wittenberg, Halle (Saale), Germany

    Katharina Lampe, Dirk Vordermark & Heike Schmidt

  3. University Hospital of Psychiatry II, University Hospital Innsbruck, Innsbruck, Austria

    Bernhard Holzner

  4. Evaluation Software Development, Innsbruck, Austria

    Bernhard Holzner

  5. Department of Internal Medicine IV, University Hospital Halle (Saale), Halle (Saale), Germany

    Haifa-Kathrin Al-Ali

  6. Krukenberg Cancer Center, University Hospital Halle (Saale), Halle (Saale), Germany

    Haifa-Kathrin Al-Ali

Authors
  1. Thomas Nordhausen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katharina Lampe
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dirk Vordermark
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bernhard Holzner
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haifa-Kathrin Al-Ali
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gabriele Meyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Heike Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TN: Responsible researcher, planning and execution of all phases of the study, writing of the manuscript. KL: Senior radio-oncologist, clinical expertise, review of the manuscript. DV: Head of the Department for Radiation Medicine, University Clinic and Outpatient Clinic for Radiotherapy, clinical expertise, review of the manuscript. BH: Technical and methodological e-PRO expertise, review of the manuscript. H-KA-A: Head of the Krukenberg Cancer Center Halle, clinical expertise, review of the manuscript. GM: Methodological expertise, review of the manuscript. HS: Principal investigator, study design, methodological expertise, co-writing and review of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Thomas Nordhausen.

Ethics declarations

Ethics approval and consent to participate

The ethics committee of the Medical Faculty, Martin Luther University Halle (Saale) approved the study protocol and had no concerns or objections regarding ethical aspects for conducting this study (declaration of no-objection from dated 08.01.2019, retrospectively assigned process number: 2022-061). Since the scope of this implementation study was to improve the quality of care in clinical routine under “real-life” conditions instead of human subject research, in accordance with the ethics committee, no informed consent of patients completing the e-PRO assessment was obtained to avoid selection bias.

Consent for publication

Not applicable.

Competing interests

Bernhard Holzner holds IPRs on the PROM software CHES.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:

Overview of the process of PRO assessment from a technical viewpoint.

Additional file 2:

Sociodemographic and clinical characteristics of patients completing the initial assessment with EORTC QLQ-C30 (n = 568), n (%) unless stated otherwise.

Additional file 3:

Results of the initial assessment with EORTC QLQ-C30, scales 0-100, traffic light system based on Giesinger et al. [35] and Lehmann et al. [38] (n = 568 patients).

Additional file 4:

Sociodemographic and clinical characteristics of patients completing the symptom monitoring with EORTC single items (n = 344), n (%) unless stated otherwise.

Additional file 5:

Results of symptom monitoring with EORTC single items, scales 0-100, traffic light system based on tripartition (n = 1774 assessment times of n=344 patients, 1-36 times per patient).

Additional file 6:

Sociodemographic and clinical characteristics of the three random samples (n=100 each) for the analysis of clinical records regarding the potential clinical benefit of the initial assessment.

Additional file 7:

Sociodemographic and clinical characteristics of the three random samples (n = 100 each) for the analysis of clinical records regarding the potential clinical benefit of the symptom monitoring.

Additional file 8:

Sociodemographic and clinical characteristics of patient contacts resulting in completion/non-completion of the e-PRO assessment, n (%)*.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nordhausen, T., Lampe, K., Vordermark, D. et al. An implementation study of electronic assessment of patient-reported outcomes in inpatient radiation oncology. J Patient Rep Outcomes 6, 77 (2022). https://doi.org/10.1186/s41687-022-00478-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-022-00478-3

Keywords