- Open Access
The Japanese version of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE): psychometric validation and discordance between clinician and patient assessments of adverse events
Journal of Patient-Reported Outcomes volume 2, Article number: 2 (2018)
The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed by the National Cancer Institute as an adverse event assessment system to evaluate patients’ symptoms, which tend to be underestimated in cancer clinical trials. The aim of this study was to assess the psychometric properties of the Japanese version of the PRO-CTCAE and the degree of adverse event assessment discordance between clinicians and patients.
A total of 187 cancer patients receiving systemic therapy were enrolled. Reproducibility, criterion validity, and responsiveness of the Japanese version of PROCTCAE were assessed. The EORTC QLQ-C30 was used as an external anchor. Discordance of assessment of adverse events between clinician and patients were also assessed using the CTCAE and PRO-CTCAE.
A total of 187 participants (187 for criterion validity, 80 for reproducibility, and 100 for responsiveness), were analyzed (Mage = 62.4 years). All patients responded to at least one symptom item (M = 16). The mean (SD) intra-class correlation coefficients of overall reproducibility for the Japanese PRO-CTCAE was 0.63 (0.02). The correlation coefficient for the corresponding items in the EORTC QLQ-C30 and the Japanese PRO-CTCAE was high (Pearson r = 0.56–0.76). The analysis of responsiveness revealed significant dose-response trends (Jonckheere-Terpstra test, ps < 0.001). Depending on the adverse events, a discrepancy was observed in evaluation between the clinician and patient.
These results revealed that there is underestimation in the assessment of adverse events in Japan, and that the Japanese version of the PRO-CTCAE had acceptable reliability and validity for common and clinically important symptoms.
The National Cancer Institute’s (NCI’s) Common Terminology Criteria for Adverse Events (CTCAE) is the current standard and predominant system for describing the severity of adverse events and is used worldwide, especially in cancer clinical trials [1, 2]. The Cancer Therapy Evaluation Program (CTEP) developed and released the original Common Toxicity Criteria in 1984, and the NCI’s CTCAE 4.0, distributed in 2009, is the latest version of the document. It is composed of 790 items and is harmonized with terminology used in the Medical Dictionary for Regulatory Activities (MedDRA). Although the CTCAE was developed by clinical experts, and has been the de facto standard method to report adverse events, its clinical validity has not yet been proven methodologically. Adverse events in the CTCAE, which can be classified into three general categories based on laboratory reports, clinical observation, and symptoms, are reported by a clinician; however, disagreement between clinicians and patients regarding symptom assessment has been revealed in research using a quality of life questionnaire (the European Organisation of Research and Treatment of Cancer Quality of Life Questionnaire Core 30, EORTC QLQ-C30), and in research on adverse event reporting using the modified CTCAE, which can use patients as respondents [3,4,5,6,7,8,9,10,11,12,13]. This evidence demonstrates that clinicians tend to underestimate the incidence and severity of symptoms in comparison with that reported by patients (i.e., patient-reported outcomes, PRO).
There is growing awareness of collecting symptom data using PRO; therefore, the NCI developed the “Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)” instrument to assess adverse symptomatic events that occur in clinical trials directly from patients’ responses. Of the 790 adverse events included in the CTCAE, 78 symptoms were identified as amenable to patient self-reporting, and the technical language was changed into plain expression . After evaluating content validity through a cognitive interview study, a multicenter study was conducted to examine the construct validity, reliability, and responsiveness of the instrument [15, 16]. This validation study included 940 patients with various types of cancer, out of which 522 (55.5%) had received chemotherapy in the 2 weeks preceding data collection, and 161 (17.1%) were a 2 to 4 on the Eastern Cooperative Oncology Group Performance Status (ECOG PS) scale. The results of this validation study showed favorable psychometric properties in diverse participants.
Various language versions of the PRO-CTCAE (e.g., German, Danish, and Spanish) have been developed [17,18,19]. In addition to these, a Japanese version of the PRO-CTCAE has also been developed and linguistically validated . The aim of the present study was twofold. First, we aimed to reveal discordance in symptom assessment between clinicians and Japanese patients with cancer. Second, we sought to examine the psychometric properties of the Japanese version of the PRO-CTCAE, including its construct validity, reliability, and responsiveness, and to confirm that the results were not significantly different from the results of the original validation study .
Patients initiating chemotherapy at four hospitals, namely Tokyo Medical University Hospital, Saitama Medical University Saitama Medical Center, Juntendo University Nerima Hospital, and Toshiba General Hospital, were invited to participate. Patients who were over 20 years of age, with any verified cancer, currently receiving systemic therapy for cancer, and with any score on the ECOG PS were eligible. All patients were required to possess sufficient Japanese language ability to understand and complete the questionnaire without assistance. Patients with cognitive impairment or any severe psychiatric disorder were excluded.
The Japanese version of the PRO-CTCAE. The original version of the PRO-CTCAE comprises 124 self-administered items, reflecting 78 symptomatic adverse events, and consists of five dimensions (presence, amount, frequency, severity, and interference with daily activities). Among these dimensions, one to three dimensions are assigned per symptom, which were selected based on attributes included in the original CTCAE items and the nature of each symptom. The recall period is the last 7 days. The psychometric properties were investigated by Dueck and colleagues, including assessment of construct validity, test-retest reliability, and responsiveness . The Japanese version of the PRO-CTCAE has been developed and linguistically validated . To limit burden on participants, only 39 items and 20 corresponding symptoms were used in the present study (see Appendix 1). These choices were based on research that has identified core symptoms that are common and clinically important to measure in clinical oncology trials, and were also used as “core symptomatic adverse events” in the development of the original version of the PRO-CTCAE [16, 21]. The following symptoms were selected: “anxiety;” “constipation;” “decreased appetite;” “dry mouth;” “fatigue, tiredness, or lack of energy;” “insomnia (including difficulty in falling asleep, staying asleep, or waking up early);” “loose or watery stools (diarrhea);” “mouth or throat sores;” “nausea;” “numbness or tingling in hands or feet;” “pain;” “sad or unhappy feelings;” “shortness of breath;” “vomiting;” “swelling in arms or legs;” “hair loss;” “headache;” “problems with concentration;” “problems with tasting food or drink;” and “appearance of a rash.” Assessment of adverse events related to these symptoms was based on their presence or absence, and three attributes: frequency, severity, and/or interference with daily life. The following were the response items for each attribute: Frequency: “Never/Rarely/Occasionally/Frequently/Almost constantly/Not applicable,” Severity: “None/Mild/Moderate/Severe/Very severe,” Interference with daily activities and amount: “Not at all A little bit/Somewhat/Quite a bit/Very much,” and Presence “Yes/No.”
As external information to assess the validity and responsiveness of the Japanese version of the PRO-CTCAE, the ECOS PS, Karnofsky Performance Status (KPS), CTCAE, EORTC QLQ-C30, and clinical global impression of change (GIC) were assessed. The KPS and ECOG PS are widely used to assess the functional status of cancer patients and the predictability of health outcomes [22,23,24,25,26,27]. Scores on the ECOG PS range from 0 to 5, representing “fully active, able to carry out all pre-disease performance without restriction” to “dead.” The Japanese version of the ECOG PS is a 5-item version developed by the Japan Cooperative Oncology Group (JCOG). Items are rated from 0 to 4, and exclude the rating of 5 (dead). The KPS consists of 11 categories, with scores ranging from 0 to 100, representing “dead” to “normal, no complaints; no evidence of disease.” In daily practice, the severity of adverse events is assessed by clinicians using the NCI-CTCAE version 4.0, distributed by JCOG, and the rating is documented in patients’ medical records. From these clinician-reported adverse events, we collected 20 adverse events corresponding to items on the Japanese version of the PRO-CTCAE used in this study. The EORTC QLQ-C30 is used to assess health related quality of life and is composed of five functional statuses, three symptoms, and six individual items. As with the performance status, some domains (e.g., physical function, pain, and appetite loss) of the EORTC QLQ-C30 are suggested to provide prognostic information . The Japanese version of the EORTC QLQ-C30 has been validated in patients with cancer . Each item is scored on a 4-point Likert-type scale, except for two items in the domain of global health status/quality of life (GHS/QOL), which uses a 7-point scale. All scores were linearly transformed to a 0 to 100 scale. A higher function score and lower symptom score represents better status. The GIC used in this study is a 7-point scale in which −3 = much worse, −2 = moderately worse, −1 = a little worse, 0 = almost the same, 1 = a little better, 2 = moderately better, and 3 = much better. Patients completed the GIC during Visit 2. The responses on the GIC were grouped as follows: items −3 to −1: worse, item 0: almost the same, and items 1 to 3: better.
Demographic information, including age, gender, type of cancer, and type of therapy, were obtained from medical records. Marital status, education level, employment status, and type of household were collected from the patient using a questionnaire.
Participants were registered in one of two cohorts, based on their chemotherapy schedule, to avoid an extra clinic visit. The first group, Cohort A, was asked to complete the questionnaire on consecutive days in order to assess the reproducibility of the Japanese version of the PRO-CTCAE. Participants assigned to the second group, Cohort B, were used to assess responsiveness between Visit 1 and Visit 2. The interval between these visits was defined as 7 days, based on the recall period of the PRO-CTCAE in this study; however, a delay of 3 days was acceptable for Visit 2. Additionally, their clinicians rated PS and CTCAE during Visit 1. For both cohorts, patients were asked to complete the Japanese version of the PRO-CTCAE and EORTC-QLQ-C30 on Visit 1. Furthermore, patients in Cohort B were asked to rate the GIC about the change in their quality of life, physical status, and emotional status after 1 week. To assess the test-retest reliability of the Japanese version of the PRO-CTCAE, 14 items were re-assessed on Day 2 in Cohort A. Data were collected using paper and pencil at two facilities, Juntendo University Nerima Hospital and Toshiba General Hospital, and using the electronic PRO (ePRO) platform at two facilities, Tokyo Medical University Hospital and Saitama Medical University Saitama Medical Center. Mode equivalence between paper and ePRO of the PRO-CTCAE has been shown to be good [30, 31]. The hospital pharmacist obtained patients’ informed consent and administered the questionnaire. The number of samples was not calculated statistically and was set to 100 for cohort A and 80 for cohort B, based on the feasibility of 1 year, which was the test period of this study.
Descriptive statistics were used to summarize baseline patient characteristics. Test-retest reliability was assessed using the intra-class correlation coefficient (ICC). The threshold value of the ICC is a matter of controversy. While some think it should be 0.7 or above, others consider such an absolute threshold to be too prescriptive . Given that this study includes patients undergoing chemotherapy, who are prone to change of condition, we adopted the threshold value defined by Cicchetti and colleagues . The threshold value of the ICC is considered poor when the ICC is less than 0.4, fair when it is 0.4 to 0.59, good when it is 0.6 to 0.74, and excellent when it is greater than or equal to 0.75. To assess construct validity, Pearson correlation coefficients were computed between each item on the Japanese version of the PRO-CTCAE and each item score on the EORTC QLQ-C30. In this construct validation, we checked that the PRO-CTCAE and EORTC correlated with scores on corresponding items and not with non-corresponding items. Responsiveness was assessed by comparing changes in the Japanese version of the PRO-CTCAE item scores, as a numerical value ranging from 1 to 4 or 5, from the first to second visit of Cohort B. Standardized response means (SRMs), which were calculated by dividing the mean score change by the standard deviation of the score change, were computed for each GIC category, and trends were investigated by the Jonckheere–Terpstra trend test. Regarding the discordance between the CTCAE (assessed by the clinician) and the Japanese version of the PRO-CTCAE (assessed by patients), the proportion of patients who reported adverse events on the Japanese version of the PRO-CTCAE to that of patients evaluated as having no adverse events by the CTCAE was examined. Statistical analyses were performed using JMP PRO (Version 13.0.0, SAS Institute Inc., Cary, NC, USA) and SPSS 24 (IBM SPSS Statistics 24, IBM Corporation, Somers, NY).
A total of 187 patients were recruited from March 2015 to March 2016. One patient in Cohort A and six patients in Cohort B dropped out before Visit 2, resulting in 80 patients included in analyses for Cohort A and 100 patients for Cohort B (Fig. 1). The mean (SD) age of the total sample was 62.4 (10.7) years [63.9 (10.6) years for Cohort A, and 61.2 (10.8) years for Cohort B; Table 1]. Breast and gastrointestinal cancer were common in both cohorts. More lung cancer patients were enrolled in Cohort A as compared to Cohort B, whereas more patients with head and neck cancer were enrolled in Cohort B. A majority of the patients received chemotherapy without molecular target medicine as curative treatment. Ninety percent or more of patients demonstrated ECOG PS scores of 0 or 1, and scores of 80 or more on the KPS. In all, 164 patients (87.7%) responded to the questionnaire via electronic tablet and 23 (12.3%) by paper; thus, the overall response rate was 100%. Further, all participants reported at least one symptom at Visit 1. Of the 39 items surveyed in this study, patients answered that they exhibited an average of 16 items (at least 1 item, maximum of 36 items per patient).
Discordance between clinicians and patients
Self-symptom assessment using the Japanese PRO-CTCAE in patients evaluated as non-graded by the clinician is shown in Fig. 2. Regarding frequency and severity, the proportion of patients who self-reported having no adverse events to that of patients rated by the clinician as having no adverse events was high for “vomiting” (87.5% for frequency, 90.2% for severity), “nausea” (74.6% for frequency, 79.3% for severity) and “swelling in the arms or legs” (67.8% for frequency, 72.3% for severity), but low for “anxiety” (34.3% for frequency, 38.7% for severity), “pain” (42.1% for frequency, 47.8% for severity) and “sad or unhappy feelings” (48.1% for frequency, 53.0% for severity).
The ICCs between responses on the Japanese version of the PRO-CTCAE items on Day 1 and 2 are shown in Table 2. The mean (95% CI) ICC for the whole scale (n = 28) was 0.63 (0.59–0.68). The mean ICCs for the attributes of frequency (n = 6), severity (n = 13), and interference (n = 9) were 0.67 (0.57–0.77), 0.62 (0.54–0.70), and 0.63 (0.56–0.71), respectively. Of the 28 items from the 14 symptoms evaluated, the ICC for 16 items (57.1%) exceeded 0.6, and only one item (vomiting, severity) had an ICC of less than 0.4.
The items with an absolute value of Pearson correlation coefficient of 0.4 or more between the function score on the EORTC QLQ-C30 and the items on the Japanese version of the PRO-CTCAE are shown in Table 3. The GHS/QOL correlated with “decreased appetite” (interference: Pearson r = −0.43, 95% CI: -0.54 to −0.31) and “fatigue, tiredness, or lack of energy” (severity: Pearson r = −0.45, 95% CI: -0.55 to −0.32; interference: Pearson r = −0.51, 95% CI: -0.61 to −0.40). There was a modest correlation between the physical functioning score and the Japanese version of the PRO-CTCAE items “fatigue, tiredness, or lack of energy” (severity: Pearson r = −0.43, 95% CI: -0.58 to −0.30; interference: Pearson r = −0.49, 95% CI: -0.59 to −0.38) and “shortness of breath” (severity: Pearson r = −0.52, 95% CI: -0.62 to −0.40; interference: Pearson r = −0.51, 95% CI: -0.60 to −0.39). Similarly, “anxiety,” “sad or unhappy feelings,” and “problems with concentration” were correlated with the emotional functioning score. There was a correlation between the cognitive functioning score and the “problem with concentration” score (severity: Pearson r = −0.52, 95% CI: -0.62 to −0.40; interference: Pearson r = −0.47, 95% CI: -0.57 to −0.35). Table 4 shows the correlations between the symptom scores on the EORTC QLQ-C30 and the Japanese version of the PRO-CTCAE. As indicated in bold in Table 4, the correlation coefficients for the corresponding items on the EORTC QLQL-C30 and the Japanese version of PRO-CTCAE were high, while those for the items that did not correspond tended to be low.
The SRMs of the Japanese version of the PRO-CTCAE are shown in Fig. 3 (see Appendix 2). Concerning the global health state, the means (95% CI) of the SRMs according to the three GIC categories were worse [−0.002 (0.058)], almost the same [−0.019 (0.020)], and better [−0.257 (0.022)], respectively. Regarding physical and emotional state, the means (95% CI) were as follows: physical state: worse, 0.257 (0.209–0.304); almost the same, 0.005 (−0.040–0.047); and better − 0.101 (−0.210–0.007); emotional state: worse, 0.353 (0.295–0.412); almost the same, 0.012 (−0.030–0.052); and better, 0.004 (−0.120–0.124); for the three GIC categories, respectively. In all states of the GIC, significant trends (namely dose dependent relationships) were observed (Jonckheere–Terpstra test, ps < 0.001).
This study aimed to investigate the discordance of adverse event assessment between clinicians and patients using the CTCAE and the Japanese version of the PRO-CTCAE, as well as to survey the psychometric properties of the Japanese version of the PRO-CTCAE. Results indicate that there is a discrepancy in evaluation between clinicians and patients, depending on the adverse events, and that the validity and reliability of the Japanese version of the PRO-CTCAE is acceptable.
First, our findings indicated that the CTCAE grades and the responses to the PRO-CTCAE did not identically correspond. Thus, we evaluated the discordance of adverse event assessment in this study, via patient self-assessment using the PRO-CTCAE, in patients assessed by medical staff as having no adverse events, via the CTCAE. Underestimation of adverse events by clinicians have been previously reported using the CTCAE and/or EORTC QLQ-C30 [3, 4, 7]. These reports have consistently shown an underestimation of fatigue, pain, and constipation by clinicians, a trend mirrored in the current study. Additionally, our results also showed that anxiety was underestimated. The Japanese version of the PRO-CTCAE was shown to have properties similar to trends demonstrated in previous studies utilizing this measure in other countries. In Japan, there is a dearth of research examining the underestimation of adverse events. Okamoto has proposed that patients believe that doctors do not actively engage in listening to their concerns, while doctors tend to believe that patients are not willing to voice their opinions . Within the unique context of Japanese physician-patient relationships, the Japanese version of the PRO-CTCAE is considered likely to affect the underestimation of adverse events, resulting in a more appropriate assessment. Finally, although this study revealed the existence of underestimation of adverse events in Japan, the sample utilized contained multiple different cancer types and treatment regimens. Further studies will be necessary to clarify discordance of particular treatments and/or cancer-type-specific adverse event assessments.
In addition, although almost all of the ICCs in evaluating the reproducibility of the Japanese version of the PRO-CTCAE were fair or good, the ICCs for “vomiting, severity (0.35)” and “decreased appetite, severity (0.48)” were relatively low. In almost all patients, these data evaluated the degree of agreement before and after the start of chemotherapy, and this timing is often defined as the acute phase in clinical trials to assess the effectiveness of antiemetic agents. A recent study in a Japanese population showed that the Complete Response (no vomiting/retching and no rescue medication) and Complete Control (no vomiting/retching, no rescue medication, and no more than mild nausea) of nausea and vomiting in the acute phase of highly emetogenic chemotherapy is approximately 75 to 90%, and the Total Control (no vomiting/retching, no rescue medication, and no nausea) is between 80 and 87% [35,36,37]. As can be seen from these data, it is well known that chemotherapy causes nausea and vomiting in the acute phase, much of which can be alleviated by antiemetic agents, although not completely controlled. The low ICCs found for some items may reflect symptomatic changes in the acute phase before and after chemotherapy.
Within the attributes of the same item, there was not much difference between ICC values. However, for “Numbness or tingling in hands or feet” there was a relatively large difference between severity and interference. A similar tendency exists in the original version . In many cases, chemotherapy-induced neuropathy becomes reversible to irreversible in a dose-dependent fashion. However, because the ICC for severity in “Numbness or tingling in hands or feet” was shown to be good in the current study, it is considered that the low value of the interference ICC is not due to a change in this symptom. Recently, a multidimensional scale for assessing chemotherapy-induced peripheral neuropathy (CIPN), called the EORTC QLQ CIPN20, has been developed . Wolf and colleagues conducted a study using this scale and argued that the evaluation of numbness of hands and feet should be strictly distinguished . They revealed that neuropathic symptoms, such as numbness or tingling, tend to appear more strongly in lower extremities than upper extremities. Therefore, it is considered that a change in the influence on daily life, especially in the feet, caused by hospitalization owing to the start of chemotherapy, is one of the reasons for the deviation in the ICC for “Numbness or tingling in hands or feet.” It should be noted when interpreting the change in this item however, that the data currently available are preliminary.
This study has several limitations. First, the psychometric properties of the Japanese version of the PRO-CTCAE revealed in this study were investigated in a subset of core items that were specifically selected as a result of previous research. Furthermore, although it is the same as the original validation study, test-retest reliability was examined with only 14 items on the Japanese PRO-CTCAE. Although it seems to be sufficient for interpretation, it should be noted that these findings are not the result of investigating all items of the Japanese version of the PRO-CTCAE. Secondly, we were unable to recruit patients with poor performance status. Although Dueck provided known group validity for these individuals during development of the original version , the relationship between this group and the Japanese version of the PRO-CTCAE was not clarified. Finally, the current study focused on underestimation of adverse events but did not assess overestimation. Future research should consider whether there is an overestimation of adverse events that leads to overtreatment.
This study revealed that the discordance between clinician and patient assessments of adverse events was similar to that of previous reports, and that the Japanese version of the PRO-CTCAE demonstrated acceptable reliability and validity for common and clinically important symptoms. It is expected that the Japanese version of the PRO-CTCAE will be applied to patient-centered evaluation of adverse events in future clinical trials in Japan.
Chemotherapy-induced peripheral neuropathy
Common Terminology Criteria for Adverse Events
Cancer Therapy Evaluation Program
- ECOG PS:
Eastern Cooperative Oncology Group Performance Status scale
- EORTC QLQ CIPN20:
European Organisation of Research and Treatment of Cancer Quality of Life Chemotherapy-Induced peripheral neuropathy
- EORTC QLQ-C30:
European Organisation of Research and Treatment of Cancer Quality of Life Questionnaire Core 30
Electronic patient-reported outcomes
Global Health Status/Quality of Life
Global impression of change
Intra-class correlation coefficient
Japan Cooperative Oncology Group
Karnofsky Performance Status
Medical Dictionary for Regulatory Activities
National Cancer Institute
Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events
Standardized response means
Trotti, A., Colevas, A. D., Setser, A., & Basch, E. (2007). Patient-reported outcomes and the evolution of adverse event reporting in oncology. Journal of Clinical Oncology, 25(32), 5121–5127.
Bruner, D. W. (2007). Should patient-reported outcomes be mandatory for toxicity reporting in cancer clinical trials? Journal of Clinical Oncology, 25(34), 5345–5347.
Basch, E., Jia, X., Heller, G., Barz, A., Sit, L., Fruscione, M., Appawu, M., Iasonos, A., Atkinson, T., Goldfarb, S., Culkin, A., Kris, M. G., & Schrag, D. (2009). Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. Journal of the National Cancer Institute, 101(23), 1624–1632.
Basch, E., Iasonos, A., McDonough, T., Barz, A., Culkin, A., Kris, M. G., Scher, H. I., & Schrag, D. (2006). Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. The Lancet Oncology, 7(11), 903–909.
Xiao, C., Polomano, R., & Bruner, D. W. (2013). Comparison between patient-reported and clinician-observed symptoms in oncology. Cancer Nursing, 36(6), E1–E16.
Fromme, E. K., Eilers, K. M., Mori, M., Hsieh, Y.-C., & Beer, T. M. (2004). How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. Journal of Clinical Oncology, 22(17), 3485–3490.
Laugsand, E. A., Sprangers, M. A., Bjordal, K., Skorpen, F., Kaasa, S., & Klepstad, P. (2010). Health care providers underestimate symptom intensities of cancer patients: a multicenter European study. Health and Quality of Life Outcomes, 8(1), 104.
Greimel, E. R., Bjelic-Radisic, V., Pfisterer, J., Hilpert, F., Daghofer, F., Pujade-Lauraine, E., Du Bois, A., & Arbeitsgemeinschaft Gynaekologische Onkologie Ovarian Cancer Study Group (AGO-OVAR), Groupe d’Investigateurs Nationaux pour les Etudes des Cancers de l’Ovaire (GINECO). (2011). Toxicity and quality of life outcomes in ovarian cancer patients participating in randomized controlled trials. Support Care Cancer, 19(9), 1421–1427.
Basch, E. (2010). The missing voice of patients in drug-safety reporting. The New England Journal of Medicine, 362(10), 865–869.
Cirillo, M., Venturini, M., Ciccarelli, L., Coati, F., Bortolami, O., & Verlato, G. (2009). Clinician versus nurse symptom reporting using the National Cancer Institute—Common Terminology Criteria for Adverse Events during chemotherapy: results of a comparison based on patient’s self-reported questionnaire. Annals of Oncology, 20(12), 1929–1935.
Novello, S., Capelletto, E., Cortinovis, D., Tiseo, M., Galetta, D., Valmadre, G., Casartelli, C., Rapetti, S. G., & Rossi, A. (2014). Italian multicenter survey to evaluate the opinion of patients and their reference clinicians on the “tolerance” to targeted therapies already available for non-small cell lung cancer treatment in daily clinical practice. Translational Lung Cancer Reseacrh, 3(3), 173–180.
Di Maio, M., Gallo, C., Leighl, N. B., Piccirillo, M. C., Daniele, G., Nuzzo, F., Gridelli, C., Gebbia, V., Ciardiello, F., & De Placido, S. (2015). Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. Journal of Clinical Oncology, 33(8), 910–915.
Montemurro, F., Mittica, G., Cagnazzo, C., Longo, V., Berchialla, P., Solinas, G., Culotta, P., Martinello, R., Foresto, M., & Gallizioli, S. (2016). Self-evaluation of adjuvant chemotherapy-related adverse effects by patients with breast cancer. JAMA Oncology, 2(4), 445–452.
Basch, E., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L. M., Dueck, A. C., Mendoza, T. R., Hay, J., Atkinson, T. M., Abernethy, A. P. B. D. W., Cleeland, C. S., Sloan, J. A., Chilukuri, R., Baumgartner, P., Denicoff, A., St. Germain, D., O’Mara, A. M., Chen, A., Kelaghan, J., Bennet, A. V., Sit, L., Rogak, L., Barz, A., Paul, D. B., & Schrag, D. (2014). Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Journal of the National Cancer Institute, 106(9), dju244.
Hay, J. L., Atkinson, T. M., Reeve, B. B., Mitchell, S. A., Mendoza, T. R., Willis, G., Minasian, L. M., Clauser, S. B., Denicoff, A., O’Mara, A., Chen, A., Bennet, A. V., Paul, D. B., Gagne, J., Rogak, L., Sit, L., Viswanath, V., Schrag, D., Basch, E., & NCI PRO-CTCA Study Group. (2014). Cognitive interviewing of the US National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Quality of Life Research, 23(1), 257–269.
Dueck, A. C., Mendoza, T. R., Mitchell, S. A., Reeve, B. B., Castro, K. M., Rogak, L. J., Atkinson, T. M., Bennett, A. V., Denicoff, A. M., O’Mara, A. M., Li, Y., Clauser, S. B., Bryant, D. M., Bearden, J. D., Gillis, T. A., Harness, J. K., Siegel, R. D., Paul, D. B., Cleeland, C. S., Schrag, D., Sloan, J. A., Abernethy, A. P., Bruner, D. W., Minasian, L. M., Basch, E., & National Cancer Institute PRO-CTCAE Study Group. (2015). Validity and reliability of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). JAMA Oncology, 1(8), 1051–1059.
Kirsch, M., Mitchell, S. A., Dobbels, F., Stussi, G., Basch, E., Halter, J. P., & De Geest, S. (2015). Linguistic and content validation of a German-language PRO-CTCAE-based patient-reported outcomes instrument to evaluate the late effect symptom experience after allogeneic hematopoietic stem cell transplantation. European Journal of Oncology Nursing, 19(1), 66–74.
Arnold, B., Mitchell, S. A., Lent, L., Mendoza, T. R., Rogak, L. J., Barragán, N. M., Willis, G., Medina, M., Lechner, S., Penedo, F. J., Harness, J. K., Basch, E. M., & PRO-CTCAE Spanish Translation and Linguistic Validation Study Group. (2016). Linguistic validation of the Spanish version of the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Support Care Cancer, 24(7), 2843–2851.
Bæksted, C., Nissen, A., Pappot, H., Bidstrup, P. E., Mitchell, S. A., Basch, E., Dalton, S. O., & Johansen, C. (2016). Danish Translation and Linguistic Validation of the US National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Journal of Pain and Symptom Management, 52(2), 292–297.
Miyaji, T., Iioka, Y., Kuroda, Y., Yamamoto, D., Iwase, S., Goto, Y., Tsuboi, M., Odagiri, H., Tsubota, Y., Kawaguchi, T., Sakata, N., Basch, E., Yamaguchi, T. Japanese translation and linguistic validation of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).
Reeve, B. B., Mitchell, S. A., Dueck, A. C., Basch, E., Cella, D., Reilly, C. M., Minasian, L. M., Denicoff, A. M., O’Mara, A. M., & Fisch, M. J. (2014). Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. Journal of the National Cancer Institute, 106(7), dju129.
Schag, C. C., Heinrich, R. L., & Ganz, P. (1984). Karnofsky performance status revisited: reliability, validity, and guidelines. Journal of Clinical Oncology, 2(3), 187–193.
Oken, M. M., Creech, R. H., Tormey, D. C., Horton, J., Davis, T. E., Mcfadden, E. T., & Carbone, P. P. (1982). Toxicity and response criteria of the Eastern Cooperative Oncology Group. American Journal of Clinical Oncology, 5(6), 649–656.
Gómez, H., Hidalgo, M., Casanova, L., Colomer, R., Pen, D., Otero, J., Rodriguez, W., Carracedo, C., Cortes-Funes, H., & Vallejos, C. (1998). Risk factors for treatment-related death in elderly patients with aggressive non-Hodgkin's lymphoma: results of a multivariate analysis. Journal of Clinical Oncology, 16(6), 2065–2069.
Lee, S. S., Ahn, J.-H., Kim, M. K., Sym, S. J., Gong, G., Ahn, S. D., Kim, S.-B., & Kim, W. K. (2008). Brain metastases in breast cancer: prognostic factors and management. Breast Cancer Research and Treatment, 111(3), 523–530.
Mani, S., Todd, M. B., Katz, K., & Poo, W.-J. (1995). Prognostic factors for survival in patients with metastatic renal cancer treated with biological response modifiers. The Journal of Urology, 154(1), 35–40.
Manola, J., Atkins, M., Ibrahim, J., & Kirkwood, J. (2000). Prognostic factors in metastatic melanoma: a pooled analysis of Eastern Cooperative Oncology Group trials. Journal of Clinical Oncology, 18(22), 3782–3793.
Quinten, C., Coens, C., Mauer, M., Comte, S., Sprangers, M. A., Cleeland, C., Osoba, D., Bjordal, K., Bottomley, A., & Clinical Groups, E. O. R. T. C. (2009). Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials. The Lancet Oncology, 10(9), 865–871.
Kobayashi, K., Takeda, F., Teramukai, S., Gotoh, I., Sakai, H., Yoneda, S., Noguchi, Y., Ogasawara, H., & Yoshida, K. (1998). A cross-validation of the European Organization for Research and Treatment of Cancer QLQ-C30 (EORTC QLQ-C30) for Japanese with lung cancer. European Journal of Cancer, 34(6), 810–815.
Bennett, A. V., Dueck, A. C., Mitchell, S. A., Mendoza, T. R., Reeve, B. B., Atkinson, T. M., Castro, K. M., Denicoff, A., Rogak, L. J., Harness, J. K., Bearden, J. D., Bryant, D., Siegel, R. D., Schrag, D., Basch, E., & National Cancer Institute PRO-CTCA Study Group. (2016). Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Health and Quality of Life Outcomes, 14(1), 24.
Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., Lenderking, W. R., Cella, D., & Basch, E. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value in Health, 12(4), 419–429.
Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., Schwartz, C., Revicki, D. A., Moinpour, C. M., & McLeod, L. D. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905.
Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.
Okamoto, S. (2007). Transformations in doctor-patient communication in Japan: the role of cultural factors. Patient Education and Counseling, 65(2), 153–155.
Suzuki, K., Yamanaka, T., Hashimoto, H., Shimada, Y., Arata, K., Matsui, R., Goto, K., Takiguchi, T., Ohyanagi, F., Kogure, Y., Nogami, N., Nakao, M., Takeda, K., Azuma, K., Nagase, S., Hayashi, T., Fujiwara, K., Shimada, T., Seki, N., & Yamamoto, N. (2016). Randomized, double-blind, phase III trial of palonosetron versus granisetron in the triplet regimen for preventing chemotherapy-induced nausea and vomiting after highly emetogenic chemotherapy: TRIPLE study. Annals of Oncology, 27(8), 1601–1606.
Komatsu, Y., Okita, K., Yuki, S., Furuhata, T., Fukushima, H., Masuko, H., Kawamoto, Y., Isobe, H., Miyagishima, T., Sasaki, K., Nakamura, M., Ohsaki, Y., Nakajima, J., Tateyama, M., Eto, K., Minami, S., Yokoyama, R., Iwanaga, I., Shibuya, H., Kudo, M., Oba, K., & Takahashi, Y. (2015). Open-label, randomized, comparative, phase III study on effects of reducing steroid use in combination with Palonosetron. Cancer Science, 106(7), 891–895.
Saito, M., Aogi, K., Sekine, I., Yoshizawa, H., Yanagita, Y., Sakai, H., Inoue, K., Kitagawa, C., Ogura, T., & Mitsuhashi, S. (2009). Palonosetron plus dexamethasone versus granisetron plus dexamethasone for prevention of nausea and vomiting during chemotherapy: a double-blind, double-dummy, randomised, comparative phase III trial. The Lancet Oncology, 10(2), 115–124.
Postma, T., Aaronson, N., Heimans, J., Muller, M., Hildebrand, J., Delattre, J.-Y., Hoang-Xuan, K., Lanteri-Minet, M., Grant, R., & Huddart, R. (2005). The development of an EORTC quality of life questionnaire to assess chemotherapy-induced peripheral neuropathy: the QLQ-CIPN20. European Journal of Cancer, 41(8), 1135–1139.
Wolf, S. L., Barton, D. L., Qin, R., Wos, E. J., Sloan, J. A., Liu, H., Aaronson, N. K., Satele, D. V., Mattar, B. I., Green, N. B., & Loprinzi, C. L. (2012). The relationship between numbness, tingling, and shooting/burning pain in patients with chemotherapy-induced peripheral neuropathy (CIPN) as measured by the EORTC QLQ-CIPN20 instrument, N06CA. Support Care Cancer, 20(3), 625–632.
The web-based electric data capture system, DEMAND™, was provided by Densuke Systems Cooperation Limited, Japan. The authors are grateful to the Japan Clinical Oncology Group, Yasushi Goto, Masahiro Tsuboi, Daigo Yamamoto, Hiroki Odagiri, Satoru Iwase, Yu Tsubota, and Sandra A. Mitchell for supporting this project. We thank Emi Fu, Hisako Ikezaki, Atsushi Okubo, KwiSeon Min, Megumi Nakamura, Tohru Kishino, Hiromi Shimada, Reiko Nunoya, Ayako Torii, Yuri Yamada, Takayuki Seki, Hideaki Ayuhara, Taiki Hirata, and Takao Akashi for their assistance in conducting the study. We would like to thank Editage (www.editage.com) for English language editing and Publication Support.
This work was supported by Japan Society for the Promotion of Science KAKENHI Grant Number 24590595, principal investigator Dr. Takuhiro Yamaguchi.
Ethics approval and consent to participate
The study protocol was approved by the institutional review board at each institution (Independent Ethics Committee of Tokyo Medical University, No.2758, 29 May 2014. Institutional Review Board of Saitama Medical Center, Saitama Medical University, No.1074, 27 November 2014. Ethics Committee of The Juntendo University Nerima Hospital, No. 14–31, 11 November 2014. Ethics Committee of Toshiba General Hospital, No. 14-A-14, 25 August 2014.) The study was conducted in accordance with the Japanese Ethical Guidelines for Epidemiological Research and was registered in the University Hospital Medical Information Network Clinical Trials Registry (UMIN000014277). All procedures performed were in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Consent for publication
Written informed consent was obtained from all individual participants included in this study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kawaguchi, T., Azuma, K., Sano, M. et al. The Japanese version of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE): psychometric validation and discordance between clinician and patient assessments of adverse events. J Patient Rep Outcomes 2, 2 (2018). https://doi.org/10.1186/s41687-017-0022-5
- Japanese Version
- Patient-Reported Outcomes Version
- Adverse Effects
- Common Terminology Criteria For Adverse Events (CTCAE)
- Intra-class Correlation Coefficient (ICC)