- Short report
- Open Access
- Published:
A multicenter paper-based and web-based system for collecting patient-reported outcome measures in patients undergoing local treatment for prostate cancer: first experiences
Journal of Patient-Reported Outcomes volume 4, Article number: 56 (2020)
Abstract
Purpose
To give an overview of the multicenter Prostate Cancer Outcomes (PCO) study, involving paper-based and web-based collection of patient-reported outcome measures (PROM) in patients undergoing local treatment for prostate cancer in certified centers in Germany. The PCO study is part of the larger Movember-funded TrueNTH Global Registry. The article reports on the study’s design and provides a brief progress report after the first 2 years of data collection.
Methods
Prostate cancer centers (PCCs) certified according to German Cancer Society requirements were invited to participate in collecting patient-reported information on symptoms and function before and at least once (at 12 months) after treatment. The data were matched with disease and treatment information. This report describes progress in patient inclusion, response rate, and variations between centers relative to online/paper use, and also data quality, including recruitment variations relative to treatment in the first participating PCCs.
Results
PCC participation increased over time; 44 centers had transferred data for 3094 patients at the time of this report. Patient recruitment varied widely across centers. Recruitment was highest among patients undergoing radical prostatectomy. The completeness of the data was good, except for comorbidity information.
Conclusions
The PCO study benefits from a quality improvement system first established over 10 years ago, requiring collection and harmonization of a predefined clinical dataset across centers. Nevertheless, establishing a PROM routine requires substantial effort on the part of providers and constant monitoring in order to achieve high-quality data. The findings reported here may be useful for guiding implementation in similar initiatives.
Introduction
The mortality rate associated with prostate cancer is low in comparison with other cancers, but patients may suffer from severe impairment of their functional status due to disease and treatment, particularly with regard to erectile function and continence [1]. There have been several reports well describing the benefits for cancer patients that result from monitoring their functional status and using functional status indicators for quality assurance and quality improvement [2,3,4,5,6]. However, such monitoring has not yet been widely implemented in routine care for prostate cancer. The German Cancer Society — together with the OnkoZert certification institute and the Federal Association of German Prostate Cancer Patient Support Groups and the Movember Foundation — has launched an outcomes data collection system that combines previously collected clinical data with a paper-based and web-based system for collecting patient-reported outcome measures (PROM) in routine prostate cancer care. This effort, known as the “Prostate Cancer Outcome (PCO)” study, is part of the larger TrueNTH Global Registry (TNGR) funded by the Movember Foundation [7].
The idea underlying the TNGR is to compare and reduce outcome variation (for the better) in relation to the positive deviance framework [8]. Briefly, this means identifying better-performing providers and “best practices” in order to help others learn from these. The data in the PCO study and the TNGR are collected in accordance with the International Consortium for Health Outcomes Measurement (ICHOM) standard set for localized prostate cancer [9], with a few additional items. All prostate cancer centers (PCCs) that are certified in accordance with the requirements of the DKG [10] were invited to participate. Patient recruitment began July 1, 2016. A German validation paper on the patient reported outcome instrument, the EPIC-26, which is in use in many large ongoing studies [11], was published recently [12]. The present article reports on the design of the PCO study and provides a brief progress report after the first 2 years of data collection.
Materials and methods
Brief study protocol
Since 2008, PCCs have been able to apply for certification in accordance with criteria set out by the German Cancer Society [10]. Certification requires centers to document clinical and treatment data that are used for annual audits. These data represent one of the two sources for patient data in the PCO study.
All PCCs were invited to participate in the PCO study in August 2015. Twenty-four centers expressed initial interest, while others still had the option to join later. During study preparation, infrastructure development, and PCC training, the study protocol was developed and approved by the ethics committee of the Medical Association of Berlin (Eth-12/16) in June 2016, with an amendment approved in June 2018.Footnote 1 Patient recruitment started in July 2016. The study protocol was based on the TNGR protocol for collecting data in accordance with the ICHOM standard set, with few divergences reflecting specific aspects of certified centers and German legal matters, including an opt-in instead of an opt-out approach. The primary purposes of the study are to compare outcome variations (both patient-reported and clinical) and to make it possible to reduce differences (for the better) using individual symptom and function profiles. This includes identifying positively deviating providers and encouraging mutual learning. A further purpose is to evaluate whether the suggested data collection infrastructure works and to constantly improve it.
Participating PCCs invite patients who are undergoing local treatment for prostate cancer — radiotherapy, radical prostatectomy including radical cystoprostatectomy —, as well as active surveillance or watchful waiting to give informed consent for the collection of data on patient, disease, and treatment characteristics needed for purposes of case-mix adjustment. These data include age, comorbidities, date of diagnosis, prostate-specific antigen (PSA) level, TNM categories, Gleason score, treatment choice (prior to recruitment and definitive), Clavien–Dindo grading, margin status etc., and were already collected as “routine data” (“routine data in center”, Fig. 1) in the center with all of them being used for other purposes as well except for comorbidities, which are documented exclusively for the study. In addition, the patients complete an initial questionnaire, which includes the EPIC-26 [13] to assess patient-reported symptoms and function before the start of definitive treatment as well as information on education, insurance, and citizenship not yet part of routine data. Patients and centers can decide whether they want questionnaires to be filled out paper-based or online. If patients complete the questionnaire on paper, it is sent to the local data center (LDC) by mail where questionnaires are entered into the PROM database manually and were scores are calculated. If patients fill out the online questionnaire, PROMs are entered in the database immediately. Upon data entry in the PROM database, the PCCs are notified and receive the symptom and function profile for each patient. Thus, PCCs can review their patients’ symptoms and function to assist in clinical decision-making before initial treatment only if patients fill out the questionnaire online. The patients are surveyed again 12 months after the start of treatment (and also, if the PCCs so choose, additionally after 6 months and annually after 1 year). The follow-up survey is organized locally, i. e. centers send out questionnaires and reminders but are assisted by a centrally developed excel-based reminder tool to allow for uniform processes. The target recruitment rate for the centers is 75%.
Data collecting infrastructure
Linkage and transfer of data
Data are thus stored in two different environments, the PROM database that includes all PROMs, and the routinely data collected data in the center. At recruitment, patient identifiers are assigned to each participant with the consent form / the paper questionnaire / the online password. Patient identifiers together with the password allow accessing the questionnaire at baseline and follow-up. In case of paper questionnaire use, patient identifiers as part of the paper questionnaire are entered by the LDC into the PROM database. Either way, PCCs enter the patient identifiers into a software (“OncoBox Prostate”) that has been in use in PCCs before that and was originally meant to make plausibility tests for routine data. Within PCO it is also used to locally match routine data with PROM data using the patient identifier. Data are then pseudonymized in the OncoBox Prostate. To allow for the primary study purpose — comparing outcomes across providers — pseudonymized patient data (i.e., not identifiable for the receiver) are transferred to the LDC at least once a year. The matched dataset is transferred to the LDC via a secure file-sharing system. The following analyses are based on all data transferred by April 2018.
Statistical analyses
We present a description of a study sample with absolute and relative frequencies of risk classification, definitive treatment, androgen deprivation therapy, and active surveillance /watchful waiting prior to recruitment, as well as comorbidities, nationality, health-insurance status, and education. Statistical analyses were carried out with R version 3.5.1. For patients recruited in 2017, comparisons with the center population are also presented for risk classification and definitive treatment. Differences in recruitment rates between centers are reported, both overall and relative to treatment, as well as variation in the use of online or paper questionnaires. Chi-squared tests were conducted to investigate differences in frequency distributions between groups.
Results
Forty-four PCCs transferred data for 3094 patients who were recruited between July 2016 and March 2018, resulting in an overall response rate of 39.3% of the eligible patients, with wide variation between centers — ranging from 4.7% to 94.6% (not presented). The online option was used by 13.8% of patients. Seventy-seven percent of the patients in the sample were suffering from low-risk or intermediate-risk disease; 1.3% had advanced (N1) disease (Table 1). Of the patients sampled, 3.2% had received androgen deprivation therapy, 2.6% active surveillance, and 0.5% watchful waiting prior to recruitment. The patients’ mean age was 66 (range 39–85). Ninety-six percent of the patients had German nationality, 77% had statutory health insurance, and one-third of them had received a university entrance qualification certificate (high-school diploma). Sixty-three percent of the recruited patients had no reported comorbidities. Whereas the documentation was complete for risk classification, treatments, and age, there was a missing data rate of below 5% for patient-reported nationality, health-insurance status, and educational level. However, the missing data rate was over 20% for center-documented comorbidities, with 14 centers not documenting comorbidities at all.
An examination of patients recruited in 2017 alone allows insight into how well the sample represents the population. In comparison with the center population, in which patients with low-risk, localized disease represented 21%, the latter only made up 17% of the study sample, whereas patients with intermediate-risk disease represented a slightly higher proportion of the study sample (45% compared to 41% in the center population) (Table 2). Ninety percent of the recruited patients received radical prostatectomy or radical prostatectomy and radiotherapy. Patients treated with radical prostatectomy were therefore overrepresented in the sample, and all other treatment groups were underrepresented (Table 2).
Recruitment of patients relative to the different treatment options also varied widely between the centers, with medians of 51% and 6% for eligible patients treated with radical prostatectomy or radiotherapy and 0% of eligible patients treated with brachytherapy and active surveillance/watchful waiting (Supplementary Table 1).
Discussion
Drawing conclusions from observational data requires thoughtful study design and evaluation of the data, as well as a sensitive interpretation of findings. A thorough discussion of the data collection process and potential pitfalls of the data are necessary in order to achieve this.
The data from clinical documentation (routine data) in this study were close to complete with the study benefitting from documentation that was established beforehand. The rate of missing information in patient-reported items was below 5%. Particular emphasis should be given to the poor recruitment of nonsurgically treated patients in many PCCs. This reflects decentralized structures in the centers and lack of cooperation between urologists and radiotherapists at various sites. Active surveillance/watchful waiting patients are similarly underrepresented in the study sample, reflecting the office-based treatment of these patients by practitioners outside the PCCs. This may also explain the underrepresentation of low-risk patients.
It should be noted that the PCCs were invited to participate on a voluntary basis, without any financial compensation. Starting with 24 PCCs in 2016, the number soon rose. Although the target value for recruitment was set at 75% of eligible patients, centers that did not reach this number were not excluded, and measures (such as workshops and monthly monitoring) were developed to improve recruitment. A thorough evaluation if these measures affect recruitment however is difficult to conduct. Efforts need to be made not only to reduce the enormous variation in recruitment rates across PCCs, but also to ensure that a representative proportion of patients who receive nonsurgical treatment options are included. In addition to the poor recruitment of nonsurgical patients, the collection of data on comorbidities (optional in the TNGR) emerged as a major challenge for the PCCs: 14 of the 44 data-transferring centers did not report comorbidities at all. The overall burden of comorbidities is lower than in most similar studies, although some other studies have reported an even lower comorbidity burden [1, 15,16,17]. However, this is suggestive of underreporting and with some centers not documenting comorbidities at all, two modifications to collect these may be considered for the future: First, reimbursement of centers for this additional data collection, and second, collecting comorbidities via the patient questionnaire. The first option would further increase costs. One strategy to generally keep costs low from the beginning was to collect PROM data online. Yet, less than 14% of patients used the online survey. One strength of the data collection infrastructure described here is that the patient population is known at each PCC, allowing recruitment rates to be calculated for each PCC and for different patient strata — which is rarely possible in comparable studies, whether observational or interventional. Although a recruitment rate of 39% may seem low in comparison with other studies, the rate reflects the true value of all eligible patients, in contrast to many other studies with undefined or unclear denominators. 70.4% of participants filled out a follow-up questionnaire (not reported).
The primary purpose of the PCO study and the TNGR is to enable health-care providers who are treating patients with prostate cancer to compare their own work with others and to initiate processes of mutual learning. Since quality improvement initiatives are collective efforts that benefit from other researchers’ experience, the authors would be grateful to receive comments on the issues discussed in this article.
Availability of data and materials
The datasets generated during and/or analysed during the current study are not publicly available as per se patient consent form.
Notes
- 1.
The study protocol (in German and English) and ethics committee documents can be downloaded from the study website at https://www.pco-study.com/centersinfo.
Abbreviations
- EPIC-26:
-
Expanded prostate cancer index composite 26-item version
- ICHOM:
-
International consortium for health outcomes measurement
- LDC:
-
Local data center
- PCC:
-
Prostate cancer center
- PCO study:
-
Prostate cancer outcomes study
- PROM:
-
Patient-reported outcome measures
- PSA:
-
Prostate-specific antigen
- TNGR:
-
TrueNTH Global Registry
References
- 1.
Barocas, D. A., Alvarez, J., Resnick, M. J., et al. (2017). Association between radiation therapy, surgery, or observation for localized prostate cancer and patient-reported outcomes after 3 years. JAMA, 317(11), 1126–1140. https://doi.org/10.1001/jama.2017.1704.
- 2.
Basch, E., Deal, A. M., Kris, M. G., Scher, H. I., Hudis, C. A., Sabbatini, P., Rogak, L., Bennett, A. V., Dueck, A. C., Atkinson, T. M., Chou, J. F., Dulko, D., Sit, L., Barz, A., Novotny, P., Fruscione, M., Sloan, J. A., & Schrag, D. (2016). Symptom monitoring with patient-reported outcomes during routine Cancer treatment: A randomized controlled trial. J Clin Oncol, 34(6), 557–565. https://doi.org/10.1200/JCO.2015.63.0830.
- 3.
Basch, E., Deal, A. M., Dueck, A. C., Scher, H. I., Kris, M. G., Hudis, C., & Schrag, D. (2017). Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine Cancer treatment. JAMA, 318(2), 197–198. https://doi.org/10.1001/jama.2017.7156.
- 4.
Klinkhammer-Schalke, M., Koller, M., Steinger, B., Ehret, C., Ernst, B., Wyatt, J. C., Hofstadter, F., & Lorenz, W. (2012). Direct improvement of quality of life using a tailored quality of life diagnosis and therapy pathway: Randomised trial in 200 women with breast cancer. Br J Cancer, 106(5), 826–838. https://doi.org/10.1038/bjc.2012.4.
- 5.
Laugsand, E. A., Sprangers, M. A., Bjordal, K., Skorpen, F., Kaasa, S., & Klepstad, P. (2010). Health care providers underestimate symptom intensities of cancer patients: A multicenter European study. Health Qual Life Outcomes, 8, 104. https://doi.org/10.1186/1477-7525-8-104.
- 6.
Denis, F., Lethrosne, C., Pourel, N., Molinier, O., Pointreau, Y., Domont, J., Bourgeois, H., Senellart, H., Tremolieres, P., Lizee, T., Bennouna, J., Urban, T., El Khouri, C., Charron, A., Septans, A. L., Balavoine, M., Landry, S., Solal-Celigny, P., & Letellier, C. (2017). Randomized trial comparing a web-mediated follow-up with routine surveillance in lung Cancer patients. J Natl Cancer Inst, 109(9). https://doi.org/10.1093/jnci/djx029.
- 7.
Evans, S. M., Millar, J. L., Moore, C. M., Lewis, J. D., Huland, H., Sampurno, F., Connor, S. E., Villanti, P., & Litwin, M. S. (2017). Cohort profile: The TrueNTH global registry - an international registry to monitor and improve localised prostate cancer health outcomes. BMJ Open, 7(11), e017006. https://doi.org/10.1136/bmjopen-2017-017006.
- 8.
Bradley, E. H., Curry, L. A., Ramanadhan, S., Rowe, L., Nembhard, I. M., & Krumholz, H. M. (2009). Research in action: Using positive deviance to improve quality of health care. Implementation Sci : IS, 4, 25. https://doi.org/10.1186/1748-5908-4-25.
- 9.
Martin, N. E., Massey, L., Stowell, C., Bangma, C., Briganti, A., Bill-Axelson, A., Blute, M., Catto, J., Chen, R. C., D'Amico, A. V., Feick, G., Fitzpatrick, J. M., Frank, S. J., Froehner, M., Frydenberg, M., Glaser, A., Graefen, M., Hamstra, D., Kibel, A., Mendenhall, N., Moretti, K., Ramon, J., Roos, I., Sandler, H., Sullivan, F. J., Swanson, D., Tewari, A., Vickers, A., Wiegel, T., & Huland, H. (2015). Defining a standard set of patient-centered outcomes for men with localized prostate cancer. Eur Urol, 67(3), 460–467. https://doi.org/10.1016/j.eururo.2014.08.075.
- 10.
Kowalski, C., Ferencz, J., Albers, P., Fichtner, J., Wiegel, T., Feick, G., & Wesselmann, S. (2016). Quality assessment in prostate cancer centers certified by the German Cancer society. World J Urol, 34(5), 665–672. https://doi.org/10.1007/s00345-015-1688-z.
- 11.
Downing, A., Wright, P., Hounsome, L., Selby, P., Wilding, S., Watson, E., Wagland, R., Kind, P., Donnelly, D. W., Butcher, H., Catto, J. W. F., Cross, W., Mason, M., Sharp, L., Weller, D., Velikova, G., McCaughan, E., Mottram, R., Allen, M., Kearney, T., McSorley, O., Huws, D. W., Brewster, D. H., McNair, E., Gavin, A., & Glaser, A. W. (2019). Quality of life in men living with advanced and localised prostate cancer in the UK: A population-based study. Lancet Oncol. https://doi.org/10.1016/s1470-2045(18)30780-0.
- 12.
Sibert, N. T., Dieng, S., Oesterle, A., Feick, G., Carl, G., Steiner, T., Minner, J., Roghmann, F., Kaftan, B., Zengerling, F., Hinkel, A., Beyer, B., Heidenreich, A., Harke, N., Brehmer, B., Pfitzenmaier, J., Fichtner, J., Neisius, A., Hammerer, P., Wesselmann, S., & Kowalski, C. (2019). Psychometric validation of the German version of the EPIC-26 questionnaire for patients with localized and locally advanced prostate cancer. World J Urol. https://doi.org/10.1007/s00345-019-02949-7.
- 13.
Beyer, B., Huland, H., Feick, G., & Graefen, M. (2015). "Expanded prostate cancer index composite" (EPIC-26) : Funktionelles Behandlungsergebnis bei Patienten mit lokalisiertem Prostatakarzinom. Urologe A, 54(11), 1591–1595. https://doi.org/10.1007/s00120-015-3922-0.
- 14.
German Guideline Program in Oncology (ed) (2018) Interdisziplinäre Leitlinie der Qualität S3 zur Früherkennung, Diagnose und Therapie der verschiedenen Stadien des Prostatakarzinoms, Version 5.0, 1. Aktualisierung, AWMF-Registernummer: 043/022OL.
- 15.
Ennis, R. D., Hu, L., Ryemon, S. N., Lin, J., & Mazumdar, M. (2018). Brachytherapy-based radiotherapy and radical prostatectomy are associated with similar survival in high-risk localized prostate Cancer. J Clin Oncol, 36(12), 1192–1198. https://doi.org/10.1200/jco.2017.75.9134.
- 16.
Fosså, S. D., Storås, A. H., Steinsvik, E. A., Myklebust, T. A., Eri, L. M., Loge, J. H., & Dahl, A. A. (2016). Psychometric testing of the Norwegian version of the expanded prostate Cancer index composite 26-item version (EPIC-26). Scand J Urol, 50(4), 280–285. https://doi.org/10.3109/21681805.2016.1163617.
- 17.
Reeve, B. B., Chen, R. C., Moore, D. T., Deal, A. M., Usinger, D. S., Lyons, J. C., & Talcott, J. A. (2014). Impact of comorbidity on health-related quality of life after prostate cancer treatment: Combined analysis of two prospective cohort studies. BJU Int, 114(6b), E74–e81. https://doi.org/10.1111/bju.12723.
Acknowledgments
The authors would like to thank the patients who participated in the study, as well as the prostate cancer center staff who recruited patients and worked continuously on delivering and improving care. Funding was received from the Movember Foundation through Help for Prostate Cancer Patients, the collaborating partner of the Movember Foundation in Germany.
Funding
Funding was received from the Movember Foundation through Help for Prostate Cancer Patients, the collaborating partner of the Movember Foundation in Germany.
Author information
Affiliations
Contributions
C Kowalski: protocol/project development, data analysis, manuscript writing/editing. R Roth: data analysis, manuscript writing/editing. G Carl: protocol/project development, manuscript writing/editing. G Feick: protocol/project development, manuscript writing/editing. A Oesterle: data analysis, data collection and management. A Hinkel: data collection and management, manuscript writing/editing. T Steiner: data collection and management, manuscript writing/editing. M Brock: data collection and management, manuscript writing/editing. B Kaftan: data collection and management, manuscript writing/editing. R Borowitz: data collection and management, manuscript writing/editing. N Zantl: data collection and management, manuscript writing/editing. A Heidenreich: data collection and management, manuscript writing/editing. A Neisius: data collection and management, manuscript writing/editing. C Darr: data collection and management, manuscript writing/editing. C Bolenz: data collection and management, manuscript writing/editing. B Beyer: data collection and management, manuscript writing/editing. J Pfitzenmaier: data collection and management, manuscript writing/editing. B Brehmer: data collection and management, manuscript writing/editing. J Fichtner: data collection and management, manuscript writing/editing. B Haben: data collection and management, manuscript writing/editing. S Wesselmann: protocol/project development, manuscript writing/editing. S Dieng: protocol/project development, data collection and management, manuscript writing/editing. The author (s) read and approved the final manuscript.
Corresponding author
Ethics declarations
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee (include name of committee + reference number) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. The study protocol was approved by the ethics committee of the Medical Association of Berlin (Eth-12/16). Informed consent was obtained from all individual participants included in the study. The study was registered in the German Clinical Trials Register (ID: DRKS00010774).
Consent for publication
n. a.
Competing interests
CK, SD, AL and SW are employees of the two institutions in charge of the certification system. BK receives consulting and lecturing honoraria from various companies that do not involve any conflicts with the research presented here. A comprehensive list is available on request. RH, AHi, AHe, MB, TS, RB, NZ, CB, CD, AN, BBe, JP, BBr, BH, GC, GF, JF hereby declare that they have no potential conflicts of interest.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Additional file 1:
Table 1. Patient recruitment relative to center according to treatment (for patients recruited in 2017).
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Kowalski, C., Roth, R., Carl, G. et al. A multicenter paper-based and web-based system for collecting patient-reported outcome measures in patients undergoing local treatment for prostate cancer: first experiences. J Patient Rep Outcomes 4, 56 (2020). https://doi.org/10.1186/s41687-020-00224-7
Received:
Accepted:
Published:
Keywords
- Multidisciplinary studies
- Health services research
- Prostate cancer
- Patient-reported outcomes
- Certification
- Functional status
- Improvement intervention
- Implementation
