Real-world use and clinical impact of an electronic patient-reported outcome tool in patients with solid tumors treated with immuno-oncology therapy

Abstract

Background

Utilization of electronic patient-reported outcome (ePRO) tools to monitor symptoms in patients undergoing cancer treatment has shown clinical benefits. Tennessee Oncology (TO) implemented an ePRO platform in 2019, allowing patients to report their health status online. We conducted a real-world, multicenter, observational, non-interventional cohort study to evaluate utilization of this platform in adults with solid tumors who initiated immuno-oncology (IO) therapy as monotherapy or in combination at TO clinics.

Methods

Patients initiating IO therapy prior to platform implementation were included in a historical control (HC) cohort; those initiating treatment after implementation were included in the ePRO cohort, which was further divided into ePRO users (platform enrollment ≤ 45 days from IO initiation) and non-users. Data were extracted from electronic medical records; patients were followed for up to 6 months (no minimum follow up). Outcomes included patient characteristics, treatment patterns, duration of therapy (DoT), and overall survival (OS).

Results

Data were collected for 538 patients in the HC and 1014 in the ePRO cohort; 319 in the ePRO cohort were ePRO users (uptake rate 31%). Baseline age was higher, more patients had stage IV disease at diagnosis, and more received monotherapy (82 vs 52%, respectively) in the HC vs the ePRO cohort. Median follow-up was 181.0 days (range 0.0–182.6) in the HC and 175.0 (0.0–184.0) in the ePRO cohort. Median DoT of index IO regimen was 5.1 months (95% confidence interval [CI], 4.4–NE) in the HC cohort vs not estimable (NE) in the ePRO cohort. Multivariable regression adjusting for baseline differences confirmed lower risk of treatment discontinuation in the ePRO vs HC cohort: hazard ratio (HR) 0.83 (95% CI, 0.71–0.97); p < 0.05. The estimated 6-month OS rate was 65.5% in the HC vs 72.4% in the ePRO cohort (p < 0 .01). Within the ePRO cohort, DoT of index IO regimen and OS did not differ between users and non-users. In ePRO users, patient platform use was durable over 6 months.

Conclusion

Improvements in DoT and OS were seen after ePRO platform implementation. Conclusions are limited by challenges in separating the impact of platform implementation from other changes affecting outcomes.

Background

Through improvements in treatments and technologies, many cancers have become long-term chronic conditions rather than acute diseases with poor survival. Cancer treatments have also become more complex, thus symptom management has become an increasingly challenging and important component of care. Patients undergoing cancer treatment frequently experience disease- and treatment-related symptoms, which are often underreported and underestimated by physicians [1,2,3,4,5,6]. The reporting process usually relies on a physician’s interpretation of a patient’s recollection of symptoms or adverse events, which may result in inaccuracies [7]. Furthermore, patients may not be well-informed regarding which symptoms require urgent medical attention [8]. In patients receiving active treatment, patient-reported symptom information is important to inform treatment decisions; an increase in symptom burden may also indicate disease progression [9]. Assessment of toxicity helps guide decisions on prevention and management (eg, dose reductions/delays), and may improve outcomes by enabling patients to tolerate therapy for longer [10].

Immuno-oncology (IO) therapies have improved clinical outcomes in multiple cancers; however, these agents have toxicity profiles related to their mode of action that differ from those of standard treatments such as chemotherapy [11, 12]. These unique immune-related adverse events (irAEs) may have prolonged duration, delayed onset, and onset after treatment discontinuation [11, 12]. Timely identification of irAEs is important to allow rapid management of toxicities and improve symptom resolution, which could result in patients remaining on treatment for longer with fewer dose interruptions.

Utilization of electronic patient-reported outcome (ePRO) tools to monitor symptoms in clinical trials enrolling patients with cancer have shown clinical benefit regarding symptom-related distress, health-related quality of life, healthcare resource utilization (HCRU), and overall survival [13,14,15]. Real-world data also support a positive impact of ePRO use on HCRU [16,17,18]. The reasons for improved outcomes with ePROs are multifaceted and likely to involve a combination of improved communication between healthcare provider and patient, a move towards ‘whole’ person care resulting in better treatment of each individual’s needs, early detection of symptoms, enhanced monitoring of treatment efficacy, and increased patient engagement, which can contribute to increased adherence to treatment plans [18,19,20,21,22]. Overall, the result is enhanced, patient-centered care which seemingly improves patient outcomes.

While increasingly used in clinical trials, ePRO tools have not yet been widely adopted in community oncology practices [23,24,25,26]. In 2018, Tennessee Oncology (TO), one of the largest community-based cancer care practices in the US, initiated development of a customized electronic patient care coordination platform to allow patient-reported symptoms to be captured easily and enable triage and care coordination teams to process information in real time. Previously, ‘treatment callbacks’ following each round of therapy were an integral component of care to help manage patient symptoms and increase clinical intervention; these were time-consuming, and clinic staff may not have had capacity to make multiple call attempts [27]. Additionally, it was hoped that an ePRO tool would allow patient status information to be readily accessible and directly linked to patient records, whereas information in the electronic medical record (EMR) cannot necessarily be accessed by the care team quickly and easily, particularly when data are not captured in discrete fields. The ePRO platform was implemented by TO in 2019 to allow patients to easily report their health status and allow the healthcare team to proactively contact patients at scheduled times to provide prompt symptom management. The platform has since become the single electronic interface at TO for all patient communication activities. We conducted this real-world study to evaluate use of the ePRO platform and its impact on treatment patterns, most importantly duration of therapy (DoT), as well as overall survival (OS), in patients with solid tumors receiving IO therapy in community practice.

Methods

This was a real-world, multicenter, observational, non-interventional cohort study. The study population comprised adults (≥18 years of age at index date) with a documented diagnosis of solid tumor malignancy (non-small cell lung cancer [NSCLC], melanoma, renal cell carcinoma, bladder cancer, or head and neck cancer) who initiated IO therapy as monotherapy or in combination with any other therapy at TO clinics. IO therapy included atezolizumab, avelumab, durvalumab, ipilimumab, nivolumab, or pembrolizumab; other IO therapies that received approval after the study launch date for the specified tumor types were also considered. Exclusion criteria were enrollment in any clinical trial or pregnancy at index.

The study design is shown in Fig. 1. Study objectives were to describe and compare patient characteristics, treatment patterns, DoT, and OS in patients receiving IO before vs after ePRO platform implementation, and to evaluate use of the platform during the post-implementation period. Patients initiating IO therapy immediately before implementation of the ePRO tool (between January 1, 2017 and December 31, 2018) were included in the historical control (HC) cohort. Those initiating IO therapy after implementation (between September 1, 2019 and December 31, 2020) were included in the ePRO cohort. Patients in the ePRO cohort were invited to use the ePRO platform and could either accept or decline. Thus, the ePRO cohort was further divided into ePRO users (those accepting to use the platform within 45 days of the index date) and non-users. The index date for both the HC and ePRO cohorts was the date that the first cycle of IO therapy was initiated. All patients were followed for up to 6 months from index date until death, loss to follow-up, pregnancy, enrollment in a clinical trial, or end of the 6-month follow-up period; there was no minimum follow-up period.

Fig. 1

Study design. ^aIn the ePRO cohort, other IO therapies indicated for the solid tumors of interest could be considered after launch of the study. ^bVoluntary enrollment in the ePRO platform within 45 days of index date. ^cUntil death, loss to follow-up, pregnancy, enrollment in a clinical trial, or 6 months post-IO therapy initiation. ^dDate on which the first cycle of IO therapy was initiated after diagnosis. ePRO electronic patient-reported outcome; HC historical control; IO immuno-oncology

De-identified patient-level data were extracted from the TO EMR database for the HC and ePRO cohorts; additional data for the ePRO users subgroup were exported from the ePRO platform. Data for the HC cohort were collected retrospectively; data for the ePRO cohort were collected prospectively at regular intervals. Abstraction of data from EMRs was conducted by designated full-time employees at TO; data were then provided to Analysis Group for analysis.

EPRO platform

The ePRO platform implemented by TO was the Noona® patient outcomes management solution (Varian Medical Systems). Noona is a cloud-based, scientifically validated, mobile platform designed to capture PROs in routine clinical practice, allowing patients to report their health status online. Healthcare providers can interact with the platform online or at the clinic and respond to patient communications. Training for physicians and nurse practitioners was conducted at the 33 TO clinics. Care coordinators and other support staff who would use the platform received intensive training and continued to receive ongoing support. Patients also received training and support. Along with notifications sent to alert patients about the change, front-office staff supported patients in setting up online accounts. Patients received hands-on support to allay concerns and address any technological challenges. For ePRO users, questionnaires based on the PRO-Common Terminology Criteria for Adverse Events (PRO-CTCAE) were sent within a week of each IO infusion and could be completed using an internet browser or tablet/smartphone app. PRO-CTCAE was utilized as TO were familiar with this instrument, allowing adoption of the new ePRO platform alongside a known PRO instrument, meaning minimal disruption to typical routines.

Data collection

Demographic, clinical, and disease characteristics were described for each cohort; computer literacy was also described for ePRO users and non-users based on patient self-report at index visit. Treatment patterns (number of regimens, monotherapy vs combination therapy, therapy type) were described for each cohort. DoT was defined as time from index date until end of treatment and was assessed for the index IO regimen and the subsequent treatment regimen. Index IO regimen was defined as the treatment that included the patient’s first IO therapy for the index cancer (which may not be the patient’s first overall line of therapy for index cancer). The IO component could start after the beginning of the overall regimen, and/or end before the end of the overall regimen. The subsequent regimen was defined as the next treatment after index IO regimen (may or may not include an IO agent). For both index and subsequent regimens, treatment was considered to have ended when all agents in that regimen were stopped, as defined in patient charts, although the same chemotherapy may have been maintained across regimens. OS was measured from index date until date of death or censoring. The proportion of patients in the ePRO cohort who opted to use the ePRO platform was described, along with the level of patient/healthcare provider use of the platform in the ePRO users subgroup. Assessments were receipt of questionnaires, response to questionnaires, reported symptoms, alerts based on symptoms, and durability of use over time. Outcomes were assessed for two consecutive 3-month periods, Months 1–3 and Months 4–6, with month 1 starting on the index date.

Statistical analysis

Statistical analysis was performed using SAS Enterprise Guide 7.1 software (SAS Institute, North Carolina) and R (The R Foundation). All outcomes were analyzed descriptively and compared between the HC and ePRO cohorts; comparisons between ePRO users and non-users were also made for demographic and clinical characteristics, computer literacy, DoT, OS, and ePRO platform usage. Continuous variables were described using mean, standard deviation (SD), and median, and compared with the Wilcoxon rank sum test; categorical variables were described with frequencies and proportions and compared using the chi-squared test.

Kaplan–Meier analyses were conducted for DoT and OS; patients were censored at end of follow-up (6 months after index) or if they died, became pregnant, or enrolled in a clinical trial at the last contact date with TO. For DoT analysis, patients who had a dose interruption or treatment holiday were considered still on treatment if the treatment plan had not changed. Landmark analysis was conducted for DoT and OS to determine the proportion of patients still on therapy/alive at 1-month intervals after treatment initiation. A univariable Cox proportional hazards model was used to compare DoT and OS in the HC and ePRO cohorts. Additionally, multivariable fitted Cox proportional hazards regression models were used to adjust for differences in baseline characteristics between the cohorts in the DoT and OS analyses and included the covariates of cohort, age at index, sex, race, index cancer, and index cancer stage at diagnosis. Multivariable analysis for DoT comparing the ePRO users and non-users subgroups also included the covariate of insurance type.

To explore the potential impact of differences in treatment patterns, the same analyses were conducted comparing DoT and OS in the subgroup of all patients who received monotherapy for first IO regimen with all those receiving combination therapy for first IO regimen (from the combined HC and ePRO cohorts). The multivariable analysis included the covariates used for the HC and ePRO comparison plus insurance type and index year.

Results

Data were collected for 538 patients in the HC cohort and 1014 patients in the ePRO cohort. In the ePRO cohort, 319 patients opted to use the ePRO platform and so became ePRO users while 695 were non-users (ePRO uptake rate of 31%).

Demographic and clinical characteristics are shown in Table 1. Patients in the HC cohort were older at baseline than those in the ePRO cohort (70.2 vs 68.3 years; p < 0.01). More patients in the ePRO cohort had a college/graduate degree level of education (16.3 vs 4.5%; p < 0.001), though this information was missing for 55.6% of patients in the ePRO cohort and 88.8% in the HC cohort. A greater proportion of patients in the HC vs the ePRO cohort had stage IV disease (54.3 vs 47.0%; p < 0.01), had metastatic recurrence in those diagnosed with stage I to III cancer (61.8 vs 39.1%; p < 0.001), had a greater number of prior lines of therapy (mean 0.6 [SD 0.7] vs 0.3 [0.5], respectively; p < 0.001), and died during the study follow-up period (34.4 vs 26.9%, respectively; p < 0.01). Differences were also observed between the HC and ePRO cohorts in index cancer type, index IO therapy, and reason for end of follow-up (all p ≤ 0.01).

Table 1 Demographic, disease, and treatment characteristics in the HC and ePRO (users and non-users) cohorts

In the ePRO cohort, ePRO users were more likely than non-users to be female, White, married, living with a spouse, and have a college or graduate degree (all p < 0.05), and a greater proportion of patients covered by risk-share contracts were ePRO users (p < 0.001). Differences were also observed between ePRO users and non-users in index cancer type and reason for end of follow-up (p < 0.01).

Demographic and clinical characteristics for the subgroups of patients receiving monotherapy (n = 969) or combination therapy (n = 583) as first IO regimen are shown in Supplementary Table S1.

In the ePRO cohort, patient and caregiver access to a computer, tablet, or smartphone and frequency of patient email use were unknown in most cases (Table 2) but confirmed access and email use were more common among ePRO users than non-users (all p < 0.001).

Table 2 Computer literacy in the ePRO cohort (users vs non-users)

The median time to end of follow-up was 181.0 days (range 0.0–182.6) in the HC cohort and 175.0 (range 0.0–184.0) in the ePRO cohort.

Treatment patterns

The mean (SD) number of regimens from index was 1.1 (0.4) and 1.1 (0.3) in the HC and the ePRO cohorts, respectively. For their index IO regimen, most patients in the HC cohort received monotherapy (82.2% vs combination therapy 17.8%), whereas in the ePRO cohort approximately half received monotherapy (52.0%) and half combination therapy (48.0%). Use of chemotherapy as part of the index regimen was also more frequent in the ePRO than the HC cohort (37.1 vs 13.6%, respectively), as was use of targeted therapy drugs (3.1 vs 0.2%, respectively). For the subsequent regimen, the proportion of patients receiving combination therapy was similar in the HC and ePRO cohorts (57.6 vs 61.9%, respectively). The subsequent regimen included IO therapy in 15.2% of patients in the HC cohort (10 of 66 patients with data) and 25.0% in the ePRO cohort (21 of 84 with data).

Duration of therapy

DoT of the index IO regimen was longer in the ePRO cohort than in the HC cohort (Fig. 2a), with median time to end of index IO regimen not estimable (NE) compared with 5.1 months (95% confidence interval [CI], 4.4–NE), respectively. More patients in the ePRO than HC cohort remained on their index IO regimen at every timepoint (6 months: 53.8 vs 45.6%, respectively). For the subsequent regimen, DoT was again longer in the ePRO cohort (n = 81) than in the HC cohort (n = 63), but the difference was not significant (Fig. 2b). Median time to end of therapy was NE in the ePRO cohort (95% CI, 3.5–NE) compared with 2.9 months (2.3–NE) in the HC cohort. The proportion of patients still on their subsequent regimen at 5 months was 56.4% in the ePRO cohort vs 40.5% in the HC cohort. Univariable analysis showed a lower risk of index IO regimen being discontinued in the ePRO vs HC cohort (HR 0.83 [95% CI, 0.71–0.97]; p < 0.05); there was no difference regarding risk of subsequent regimen discontinuation (HR 0.81 [95% CI, 0.46–1.41]; p = 0.452). Multivariable regression analysis confirmed the univariable findings for index IO regimen, showing a lower risk of treatment discontinuation in the ePRO vs HC cohort (Table 3). There was a higher risk of discontinuation in the index IO regimen in patients with NSCLC (p < 0.05) and ‘other’ cancers (p < 0.001) vs melanoma, and a lower risk in patients with stage III vs stage I cancer at diagnosis (p < 0.05). There were no differences for the subsequent regimen (Table 3).

Fig. 2

Duration of therapy in HC vs ePRO cohort. a index IO regimen and b subsequent regimen. ^aPatients whose end of follow-up date (eg, date of last contact with TO was on their index date were excluded from the analytical subset (no observed follow-up time), as were patients with unknown line of therapy start or end dates. Kaplan-Meier curves were truncated at 6 months from the index date, the maximum observation period. Shaded areas around the curves represent 95% CIs. CI, confidence interval; ePRO electronic patient-reported outcome; HC historical control; IO immuno-oncology; NE not estimable; Pts patients; TO Tennessee oncology

Table 3 Multivariable fitted cox proportional hazards model for dot

Comparison of the ePRO users and non-users subgroups showed no difference in DoT of the index IO regimen (Fig. 3a), with a median time to the end of index IO regimen of NE in both subgroups and the proportion of patients still on index IO regimen at 6 months of 54.1 and 53.7%, respectively. For the subsequent regimen, DoT was longer among ePRO users (n = 29) than non-users (n = 52) but the difference was not significant (Fig. 3b); median time to end of therapy was not reached in either group and the proportion of patients still on the subsequent regimen at 4 months was 63.9 vs 54.7%, respectively. Univariable analysis showed no difference in the risk of either the index IO regimen (HR 1.00 [95% CI, 0.82–1.22]; P = 0.997) or subsequent regimen (HR 0.42 [95% CI, 0.15–1.12]; P = 0.081) being ended for ePRO users vs non-users. For the index IO regimen, multivariable regression analysis confirmed the findings of the univariable analysis, showing no difference in the risk of treatment being ended between ePRO users and non-users (Table 3). There were effects according to index cancer and index cancer stage at diagnosis for the index IO regimen. For the subsequent regimen, multivariable modeling showed a lower risk of the subsequent regimen being discontinued among ePRO users vs non-users (Table 3).

Fig. 3

Duration of therapy in ePRO cohort users vs non-users. a index IO regimen and b subsequent regimen. ^aPatients whose end of follow-up date (eg, date of last contact with TO) was on their index date were excluded from the analytical subset (no observed follow-up time), as were patients with unknown line of therapy start or end dates. Kaplan-Meier curves were truncated at 6 months from the index date, the maximum observation period. Shaded areas around the curves represent 95% CIs. CI confidence interval; ePRO electronic patient-reported outcome; IO immuno-oncology; NE not estimable; Pts patients; TO Tennessee oncology

The subgroup analysis comparing DoT in all patients receiving monotherapy with all those receiving combination therapy as first IO regimen showed no difference (Fig. 4), with median time to end of the index IO regimen being NE in both subgroups and proportion of patients still on the index IO regimen at 6 months being 51.2 and 50.6%, respectively (p = 1.0). Univariable (HR 1.00 [95% CI, 0.86–1.17]; p = 0.990) and multivariable analyses also showed no difference (Table 3). In the latter there were effects according to index cancer type and index cancer stage at diagnosis.

Fig. 4

Duration of therapy for first IO regimen in the monotherapy and combination therapy subgroups. ^aPatients whose end of follow-up date (eg, date of last contact with TO) was on their index date were excluded from the analytical subset (no observed follow-up time), as were patients with unknown line of therapy start or end dates. Kaplan-Meier curves were truncated at 6 months from the index date, the maximum observation period. Shaded areas around the curves represent 95% CIs. CI confidence interval; IO immuno-oncology; NE not estimable; Pts patients; TO Tennessee oncology

Overall survival

The estimated OS rate at 6 months was greater in the ePRO cohort than the HC cohort (72.4 vs 65.5%; p < 0 .01); median OS was NE in both cohorts (Fig. 5a). Multivariable regression confirmed longer OS in the ePRO vs HC cohort (Supplementary Table S2), with significant effects also seen for age at index date, gender, index cancer, and index cancer stage at diagnosis.

Fig. 5

Estimated OS. a HC vs ePRO cohort, b ePRO cohort users vs non-users, and c) monotherapy vs combination therapy subgroups for first IO regimen. ^aPatients who did not die or had unknown death date were censored at date of last contact with TO). Kaplan-Meier curves were truncated at 6 months from the index date, the maximum observation period. Shaded areas around the curves represent 95% CIs. CI confidence interval; Comb combination therapy; ePRO electronic patient-reported outcome; HC historical control; IO immuno-oncology; M month; Mono monotherapy; NE not estimable; OS overall survival; Pts patients; TO Tennessee oncology; TTE time to event

Within the ePRO cohort, the estimated OS at 6 months was similar for ePRO users and non-users (75.2 vs 71.1%; p = 0.2); median OS was NE in both groups (Fig. 5b). Multivariable regression showed no difference in length of OS in ePRO users vs non-users (Supplementary Table S2); significant effects were seen for age at index date and index cancer stage at diagnosis.

In the subgroup analysis of all patients receiving monotherapy vs all those receiving combination therapy for first IO regimen there was no difference in estimated OS rate at 6 months (69.3 vs 70.9%; p = 0.6); median OS was NE in both subgroups (Fig. 5c). Median time to end of follow-up was 179.0 days (range 0.0–184.0) in the monotherapy subgroup and 175.0 (range 0.0–184.0) in the combination therapy subgroup. Multivariable regression confirmed there was no difference in survival in the monotherapy vs combination therapy subgroups (Supplementary Table S2); significant effects were seen for age at index date, sex, index cancer, and index cancer stage at diagnosis.

EPRO platform usage

In the ePRO users group, use of the platform by both providers and patients was durable over the 6-month study period. There was no change in the total number of questionnaires sent to patients by providers from the first time period (Months 1–3) to the second time period (Months 4–6; Table 4). There was a small decrease in the number of questionnaires answered per patient and a small increase in the total number of questionnaires expired per patient.

Table 4 Use of ePRO platform in ePRO users subgroup

The number of patient/caregiver-reported symptoms decreased slightly in the second time period compared with the first period (mean 0.4 [SD 1.1] vs 0.7 [1.4], respectively; Table 4). However, most patients did not report any symptoms. The number of severe symptoms reported also decreased from the first time period to the second, with the frequency of 1–2 reported severe symptoms dropping from 6.2 to 3.3%, although most patients did not report any severe symptoms. From the first 3-month period to the second 3-month period, there was a reduction in the proportion of patients with any symptom alerts (11.2 vs 7.5%) and in the mean number of alerts (0.2 [SD 0.6] vs 0.1 [0.4]; Table 4). Fewer alerts were issued in the second 3-month period than the first 3-month period. In both time periods, the most frequent alert outcome was “resolved on phone/message”.

Discussion

This observational, non-interventional study was, to our knowledge, the first to evaluate the use of an ePRO platform and its impact on treatment patterns, particularly DoT, in patients with solid tumors receiving IO therapy in community practice. Treatment patterns for the index IO regimen differed between the HC and ePRO cohorts in that most patients in the HC cohort received monotherapy whereas in the ePRO cohort approximately half received monotherapy and half combination therapy. Consequently, use of chemotherapy and targeted therapy in the index IO regimen was more frequent in the ePRO cohort. These differences likely reflect changes in standard of care for solid tumors over the period of ePRO platform implementation, with the approval of several targeted therapies (including combinations) and increased used of IO-chemotherapy, IO-targeted therapy, and IO-IO combinations, which have shown improved outcomes compared with IO monotherapy, chemotherapy, and targeted monotherapy [28,29,30,31,32].

Index IO regimen DoT was longer in the ePRO vs HC cohort in this study, and multivariable regression analysis confirmed a lower risk of the index IO regimen being discontinued in the ePRO vs HC cohort. OS was also longer in the ePRO cohort than the HC cohort (although median OS was not reached in either cohort), with multivariable analysis confirming longer survival in the ePRO vs HC cohort. Baseline differences between the ePRO and HC cohorts in age and disease stage, which could potentially impact on DoT and OS outcomes, were adjusted for in the multivariable analysis. However, it is unclear whether use of the ePRO platform resulted in the improvements in DoT and OS given the voluntary nature of real-world adoption of the platform, and more importantly, the differences in treatment patterns between the two cohorts, reflecting broader changes in standards of care, which are expected to play a key role in the differing clinical outcomes. In the subgroup analysis comparing ePRO users and ePRO non-users, DoT for the index IO regimen and OS did not differ between groups, supporting the possibility that broader changes in clinical practice occurring after ePRO platform implementation may have been the reason for the improvements in the ePRO cohort, although the smaller sample sizes of ePRO users and ePRO non-users could be a confounding factor. However, the subgroup analysis comparing all patients who received monotherapy vs all those who received combination therapy for the first IO regimen showed no differences in DoT or estimated OS rate over 6 months, suggesting that the differences in mono- vs combination therapy patterns between the ePRO and HC cohorts had minimal impact on these outcomes.

Among patients in the ePRO cohort, uptake of the ePRO platform was 31%. This is comparable with two recent US studies reporting ePRO use in single institutions (initial response rates of 10–20% and 37%) [33, 34]. Lack of computer literacy and online access are likely to be significant factors affecting patient engagement with ePROs in the real world, and lower education level and non-working status are also possible barriers [15]. In our study, patient/caregiver access to suitable electronic devices and frequency of email use were unknown in most cases, but as might be expected, confirmed patient/caregiver access to a suitable device and frequency of patient email use were higher in ePRO users than in non-users. Social determinants also appeared to influence use of the platform, with greater proportions of White patients and patients having a college/higher-level education in the users group vs non-users, although information on education level was unknown in a substantial proportion of patients.

Implementation of the ePRO platform resulted in several learnings, some of which have already been addressed at TO to improve future uptake of the platform. An organizational commitment to understanding and navigating hurdles was required, eg, transitioning from the previous patient portal and telephone triage software to the new integrated platform needed substantial effort from operations, information technology, and clinical informatics teams to minimize disruption to established clinic workflows. Ensuring consistency in TO branding for all communications was also important to assure patients that they were legitimate. Questionnaires were based on the PRO-Common Terminology Criteria for Adverse Events typically used in clinical trials, but over time we realized that a better approach would be to phrase questions in an accessible way so they can be easily answered by the wide range of patients seen in our clinics. Despite increased reliance on technology during the COVID-19 pandemic, a substantial proportion of TO patients cannot/choose not to use an online care coordination platform. Lack of access to adequate internet or technology, lack of digital skills, and wariness around sharing personal information electronically continue to remain challenges for patients and, ultimately, for their care team.

In those patients opting to use the platform (ePRO users group), its use was durable over the 6-month period, with no change in total number of questionnaires sent to patients from the first time period (Months 1–3) to the second period (Months 4–6). A small decrease in the number of questionnaires answered per patient and a small increase in the total number of questionnaires expired per patient was observed, which is in line with findings on longitudinal PRO completion [35]. The platform was successfully employed by ePRO users to report their symptoms, although in the second time period (Months 4–6) there was a slight decrease in the number reporting symptoms and in the number of symptom alerts per patient compared with the first time period; it is not possible to determine the reasons for this, which may include better management of symptoms or a decrease in use of the platform. The majority of symptom alerts had the outcome of “resolved on phone/message.” Previous research has suggested adherence to PRO completion ranges from 50% to over 80%, with longitudinal PROs decreasing over time [36]. Our study had a completion rate of 31%, suggesting a need to improve patient adherence to PRO completion. Influences on factors that improve patient adherence to PRO completion are not clear from currently available evidence, although clinician and administrative engagement in the reporting of PROs has been suggested as a factor to enhance PRO completion [37]. Research has also shown that patients who complete PROs tend to have better functional capacity, meaning missing data may indicate worsening health, among other factors [38].

The strength of the study was the collection of data from the EMR database of one of the largest community-based cancer care practices in the US, which captured detailed information for a large number of patients with a range of solid tumors receiving IO therapies.

The main limitation was that inherent to chart abstraction studies, ie, missing, incomplete, or inaccurate data entries; however, such cases were queried with the treating TO physician where possible. Other limitations include the possibility that the ePRO platform may have been implemented differently for different patients or may have changed over time. Furthermore, it is challenging to separate the impact of ePRO platform implementation from other changes that occurred during the same period both at the individual patient level and the institutional management level, and from broader circumstances that impacted patient care and/or outcomes (eg, changes in standard of care, the COVID-19 pandemic). The time period studied was limited in an attempt to mitigate this. However, further studies are necessary to separate the effects of the ePRO platform from other changes in patient care.

Conclusion

In summary, improvements in DoT and OS were seen after implementation of the ePRO platform in both ePRO users and non-users. The implementation coincided with changes in the standard of care for solid tumors, which was reflected by the differences in treatment patterns between the HC and ePRO cohorts, although subgroup analysis suggested that differences based on use of monotherapy vs combination therapy had minimal impact on these outcomes. Voluntary patient participation in ePRO use outside the clinic is likely to confound results in the evaluation of real-world outcomes, and further investigation is warranted to determine whether the platform played a role in the improvements observed. The use of ePROs has the potential to facilitate improved care coordination and may enable patients to remain on IO therapy for longer. Our ultimate vision at TO is a comprehensive platform that touches on multiple aspects of clinical care. Careful, continuous evaluation of the implementation process and ongoing monitoring and adjustment of this innovation will be crucial to success.