Skip to main content
Journal of Patient-Reported Outcomes
Download PDF
Download ePub

The Japanese version of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE): psychometric validation and discordance between clinician and patient assessments of adverse events

Journal of Patient-Reported Outcomes volume 2, Article number: 2 (2018) Cite this article

Abstract

Background

The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed by the National Cancer Institute as an adverse event assessment system to evaluate patients’ symptoms, which tend to be underestimated in cancer clinical trials. The aim of this study was to assess the psychometric properties of the Japanese version of the PRO-CTCAE and the degree of adverse event assessment discordance between clinicians and patients.

Methods

A total of 187 cancer patients receiving systemic therapy were enrolled. Reproducibility, criterion validity, and responsiveness of the Japanese version of PROCTCAE were assessed. The EORTC QLQ-C30 was used as an external anchor. Discordance of assessment of adverse events between clinician and patients were also assessed using the CTCAE and PRO-CTCAE.

Results

A total of 187 participants (187 for criterion validity, 80 for reproducibility, and 100 for responsiveness), were analyzed (Mage = 62.4 years). All patients responded to at least one symptom item (M = 16). The mean (SD) intra-class correlation coefficients of overall reproducibility for the Japanese PRO-CTCAE was 0.63 (0.02). The correlation coefficient for the corresponding items in the EORTC QLQ-C30 and the Japanese PRO-CTCAE was high (Pearson r = 0.56–0.76). The analysis of responsiveness revealed significant dose-response trends (Jonckheere-Terpstra test, ps < 0.001). Depending on the adverse events, a discrepancy was observed in evaluation between the clinician and patient.

Conclusions

These results revealed that there is underestimation in the assessment of adverse events in Japan, and that the Japanese version of the PRO-CTCAE had acceptable reliability and validity for common and clinically important symptoms.

Background

The National Cancer Institute’s (NCI’s) Common Terminology Criteria for Adverse Events (CTCAE) is the current standard and predominant system for describing the severity of adverse events and is used worldwide, especially in cancer clinical trials [1, 2]. The Cancer Therapy Evaluation Program (CTEP) developed and released the original Common Toxicity Criteria in 1984, and the NCI’s CTCAE 4.0, distributed in 2009, is the latest version of the document. It is composed of 790 items and is harmonized with terminology used in the Medical Dictionary for Regulatory Activities (MedDRA). Although the CTCAE was developed by clinical experts, and has been the de facto standard method to report adverse events, its clinical validity has not yet been proven methodologically. Adverse events in the CTCAE, which can be classified into three general categories based on laboratory reports, clinical observation, and symptoms, are reported by a clinician; however, disagreement between clinicians and patients regarding symptom assessment has been revealed in research using a quality of life questionnaire (the European Organisation of Research and Treatment of Cancer Quality of Life Questionnaire Core 30, EORTC QLQ-C30), and in research on adverse event reporting using the modified CTCAE, which can use patients as respondents [3,4,5,6,7,8,9,10,11,12,13]. This evidence demonstrates that clinicians tend to underestimate the incidence and severity of symptoms in comparison with that reported by patients (i.e., patient-reported outcomes, PRO).

There is growing awareness of collecting symptom data using PRO; therefore, the NCI developed the “Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)” instrument to assess adverse symptomatic events that occur in clinical trials directly from patients’ responses. Of the 790 adverse events included in the CTCAE, 78 symptoms were identified as amenable to patient self-reporting, and the technical language was changed into plain expression [14]. After evaluating content validity through a cognitive interview study, a multicenter study was conducted to examine the construct validity, reliability, and responsiveness of the instrument [15, 16]. This validation study included 940 patients with various types of cancer, out of which 522 (55.5%) had received chemotherapy in the 2 weeks preceding data collection, and 161 (17.1%) were a 2 to 4 on the Eastern Cooperative Oncology Group Performance Status (ECOG PS) scale. The results of this validation study showed favorable psychometric properties in diverse participants.

Various language versions of the PRO-CTCAE (e.g., German, Danish, and Spanish) have been developed [17,18,19]. In addition to these, a Japanese version of the PRO-CTCAE has also been developed and linguistically validated [20]. The aim of the present study was twofold. First, we aimed to reveal discordance in symptom assessment between clinicians and Japanese patients with cancer. Second, we sought to examine the psychometric properties of the Japanese version of the PRO-CTCAE, including its construct validity, reliability, and responsiveness, and to confirm that the results were not significantly different from the results of the original validation study [16].

Methods

Participants

Patients initiating chemotherapy at four hospitals, namely Tokyo Medical University Hospital, Saitama Medical University Saitama Medical Center, Juntendo University Nerima Hospital, and Toshiba General Hospital, were invited to participate. Patients who were over 20 years of age, with any verified cancer, currently receiving systemic therapy for cancer, and with any score on the ECOG PS were eligible. All patients were required to possess sufficient Japanese language ability to understand and complete the questionnaire without assistance. Patients with cognitive impairment or any severe psychiatric disorder were excluded.

Measurement

The Japanese version of the PRO-CTCAE. The original version of the PRO-CTCAE comprises 124 self-administered items, reflecting 78 symptomatic adverse events, and consists of five dimensions (presence, amount, frequency, severity, and interference with daily activities). Among these dimensions, one to three dimensions are assigned per symptom, which were selected based on attributes included in the original CTCAE items and the nature of each symptom. The recall period is the last 7 days. The psychometric properties were investigated by Dueck and colleagues, including assessment of construct validity, test-retest reliability, and responsiveness [16]. The Japanese version of the PRO-CTCAE has been developed and linguistically validated [20]. To limit burden on participants, only 39 items and 20 corresponding symptoms were used in the present study (see Appendix 1). These choices were based on research that has identified core symptoms that are common and clinically important to measure in clinical oncology trials, and were also used as “core symptomatic adverse events” in the development of the original version of the PRO-CTCAE [16, 21]. The following symptoms were selected: “anxiety;” “constipation;” “decreased appetite;” “dry mouth;” “fatigue, tiredness, or lack of energy;” “insomnia (including difficulty in falling asleep, staying asleep, or waking up early);” “loose or watery stools (diarrhea);” “mouth or throat sores;” “nausea;” “numbness or tingling in hands or feet;” “pain;” “sad or unhappy feelings;” “shortness of breath;” “vomiting;” “swelling in arms or legs;” “hair loss;” “headache;” “problems with concentration;” “problems with tasting food or drink;” and “appearance of a rash.” Assessment of adverse events related to these symptoms was based on their presence or absence, and three attributes: frequency, severity, and/or interference with daily life. The following were the response items for each attribute: Frequency: “Never/Rarely/Occasionally/Frequently/Almost constantly/Not applicable,” Severity: “None/Mild/Moderate/Severe/Very severe,” Interference with daily activities and amount: “Not at all A little bit/Somewhat/Quite a bit/Very much,” and Presence “Yes/No.”

Anchor

As external information to assess the validity and responsiveness of the Japanese version of the PRO-CTCAE, the ECOS PS, Karnofsky Performance Status (KPS), CTCAE, EORTC QLQ-C30, and clinical global impression of change (GIC) were assessed. The KPS and ECOG PS are widely used to assess the functional status of cancer patients and the predictability of health outcomes [22,23,24,25,26,27]. Scores on the ECOG PS range from 0 to 5, representing “fully active, able to carry out all pre-disease performance without restriction” to “dead.” The Japanese version of the ECOG PS is a 5-item version developed by the Japan Cooperative Oncology Group (JCOG). Items are rated from 0 to 4, and exclude the rating of 5 (dead). The KPS consists of 11 categories, with scores ranging from 0 to 100, representing “dead” to “normal, no complaints; no evidence of disease.” In daily practice, the severity of adverse events is assessed by clinicians using the NCI-CTCAE version 4.0, distributed by JCOG, and the rating is documented in patients’ medical records. From these clinician-reported adverse events, we collected 20 adverse events corresponding to items on the Japanese version of the PRO-CTCAE used in this study. The EORTC QLQ-C30 is used to assess health related quality of life and is composed of five functional statuses, three symptoms, and six individual items. As with the performance status, some domains (e.g., physical function, pain, and appetite loss) of the EORTC QLQ-C30 are suggested to provide prognostic information [28]. The Japanese version of the EORTC QLQ-C30 has been validated in patients with cancer [29]. Each item is scored on a 4-point Likert-type scale, except for two items in the domain of global health status/quality of life (GHS/QOL), which uses a 7-point scale. All scores were linearly transformed to a 0 to 100 scale. A higher function score and lower symptom score represents better status. The GIC used in this study is a 7-point scale in which −3 = much worse, −2 = moderately worse, −1 = a little worse, 0 = almost the same, 1 = a little better, 2 = moderately better, and 3 = much better. Patients completed the GIC during Visit 2. The responses on the GIC were grouped as follows: items −3 to −1: worse, item 0: almost the same, and items 1 to 3: better.

Demographics

Demographic information, including age, gender, type of cancer, and type of therapy, were obtained from medical records. Marital status, education level, employment status, and type of household were collected from the patient using a questionnaire.

Study design

Participants were registered in one of two cohorts, based on their chemotherapy schedule, to avoid an extra clinic visit. The first group, Cohort A, was asked to complete the questionnaire on consecutive days in order to assess the reproducibility of the Japanese version of the PRO-CTCAE. Participants assigned to the second group, Cohort B, were used to assess responsiveness between Visit 1 and Visit 2. The interval between these visits was defined as 7 days, based on the recall period of the PRO-CTCAE in this study; however, a delay of 3 days was acceptable for Visit 2. Additionally, their clinicians rated PS and CTCAE during Visit 1. For both cohorts, patients were asked to complete the Japanese version of the PRO-CTCAE and EORTC-QLQ-C30 on Visit 1. Furthermore, patients in Cohort B were asked to rate the GIC about the change in their quality of life, physical status, and emotional status after 1 week. To assess the test-retest reliability of the Japanese version of the PRO-CTCAE, 14 items were re-assessed on Day 2 in Cohort A. Data were collected using paper and pencil at two facilities, Juntendo University Nerima Hospital and Toshiba General Hospital, and using the electronic PRO (ePRO) platform at two facilities, Tokyo Medical University Hospital and Saitama Medical University Saitama Medical Center. Mode equivalence between paper and ePRO of the PRO-CTCAE has been shown to be good [30, 31]. The hospital pharmacist obtained patients’ informed consent and administered the questionnaire. The number of samples was not calculated statistically and was set to 100 for cohort A and 80 for cohort B, based on the feasibility of 1 year, which was the test period of this study.

Statistical analyses

Descriptive statistics were used to summarize baseline patient characteristics. Test-retest reliability was assessed using the intra-class correlation coefficient (ICC). The threshold value of the ICC is a matter of controversy. While some think it should be 0.7 or above, others consider such an absolute threshold to be too prescriptive [32]. Given that this study includes patients undergoing chemotherapy, who are prone to change of condition, we adopted the threshold value defined by Cicchetti and colleagues [33]. The threshold value of the ICC is considered poor when the ICC is less than 0.4, fair when it is 0.4 to 0.59, good when it is 0.6 to 0.74, and excellent when it is greater than or equal to 0.75. To assess construct validity, Pearson correlation coefficients were computed between each item on the Japanese version of the PRO-CTCAE and each item score on the EORTC QLQ-C30. In this construct validation, we checked that the PRO-CTCAE and EORTC correlated with scores on corresponding items and not with non-corresponding items. Responsiveness was assessed by comparing changes in the Japanese version of the PRO-CTCAE item scores, as a numerical value ranging from 1 to 4 or 5, from the first to second visit of Cohort B. Standardized response means (SRMs), which were calculated by dividing the mean score change by the standard deviation of the score change, were computed for each GIC category, and trends were investigated by the Jonckheere–Terpstra trend test. Regarding the discordance between the CTCAE (assessed by the clinician) and the Japanese version of the PRO-CTCAE (assessed by patients), the proportion of patients who reported adverse events on the Japanese version of the PRO-CTCAE to that of patients evaluated as having no adverse events by the CTCAE was examined. Statistical analyses were performed using JMP PRO (Version 13.0.0, SAS Institute Inc., Cary, NC, USA) and SPSS 24 (IBM SPSS Statistics 24, IBM Corporation, Somers, NY).

Results

Participants

A total of 187 patients were recruited from March 2015 to March 2016. One patient in Cohort A and six patients in Cohort B dropped out before Visit 2, resulting in 80 patients included in analyses for Cohort A and 100 patients for Cohort B (Fig. 1). The mean (SD) age of the total sample was 62.4 (10.7) years [63.9 (10.6) years for Cohort A, and 61.2 (10.8) years for Cohort B; Table 1]. Breast and gastrointestinal cancer were common in both cohorts. More lung cancer patients were enrolled in Cohort A as compared to Cohort B, whereas more patients with head and neck cancer were enrolled in Cohort B. A majority of the patients received chemotherapy without molecular target medicine as curative treatment. Ninety percent or more of patients demonstrated ECOG PS scores of 0 or 1, and scores of 80 or more on the KPS. In all, 164 patients (87.7%) responded to the questionnaire via electronic tablet and 23 (12.3%) by paper; thus, the overall response rate was 100%. Further, all participants reported at least one symptom at Visit 1. Of the 39 items surveyed in this study, patients answered that they exhibited an average of 16 items (at least 1 item, maximum of 36 items per patient).

Fig. 1
figure 1

Flow of experimental protocols

Full size image
Table 1 Participant characteristics
Full size table

Discordance between clinicians and patients

Self-symptom assessment using the Japanese PRO-CTCAE in patients evaluated as non-graded by the clinician is shown in Fig. 2. Regarding frequency and severity, the proportion of patients who self-reported having no adverse events to that of patients rated by the clinician as having no adverse events was high for “vomiting” (87.5% for frequency, 90.2% for severity), “nausea” (74.6% for frequency, 79.3% for severity) and “swelling in the arms or legs” (67.8% for frequency, 72.3% for severity), but low for “anxiety” (34.3% for frequency, 38.7% for severity), “pain” (42.1% for frequency, 47.8% for severity) and “sad or unhappy feelings” (48.1% for frequency, 53.0% for severity).

Fig. 2
figure 2

Response of PRO-CTCAE in patients assessed by the CTCAE as having no adverse events. (a: Frequency, b: Severity, c: Interference, d: Amount, and e: Present/absent.)

Full size image

Reliability

The ICCs between responses on the Japanese version of the PRO-CTCAE items on Day 1 and 2 are shown in Table 2. The mean (95% CI) ICC for the whole scale (n = 28) was 0.63 (0.59–0.68). The mean ICCs for the attributes of frequency (n = 6), severity (n = 13), and interference (n = 9) were 0.67 (0.57–0.77), 0.62 (0.54–0.70), and 0.63 (0.56–0.71), respectively. Of the 28 items from the 14 symptoms evaluated, the ICC for 16 items (57.1%) exceeded 0.6, and only one item (vomiting, severity) had an ICC of less than 0.4.

Table 2 Test-retest reliability of selected PRO-CTCAE items (n = 80)
Full size table

Validity

The items with an absolute value of Pearson correlation coefficient of 0.4 or more between the function score on the EORTC QLQ-C30 and the items on the Japanese version of the PRO-CTCAE are shown in Table 3. The GHS/QOL correlated with “decreased appetite” (interference: Pearson r = −0.43, 95% CI: -0.54 to −0.31) and “fatigue, tiredness, or lack of energy” (severity: Pearson r = −0.45, 95% CI: -0.55 to −0.32; interference: Pearson r = −0.51, 95% CI: -0.61 to −0.40). There was a modest correlation between the physical functioning score and the Japanese version of the PRO-CTCAE items “fatigue, tiredness, or lack of energy” (severity: Pearson r = −0.43, 95% CI: -0.58 to −0.30; interference: Pearson r = −0.49, 95% CI: -0.59 to −0.38) and “shortness of breath” (severity: Pearson r = −0.52, 95% CI: -0.62 to −0.40; interference: Pearson r = −0.51, 95% CI: -0.60 to −0.39). Similarly, “anxiety,” “sad or unhappy feelings,” and “problems with concentration” were correlated with the emotional functioning score. There was a correlation between the cognitive functioning score and the “problem with concentration” score (severity: Pearson r = −0.52, 95% CI: -0.62 to −0.40; interference: Pearson r = −0.47, 95% CI: -0.57 to −0.35). Table 4 shows the correlations between the symptom scores on the EORTC QLQ-C30 and the Japanese version of the PRO-CTCAE. As indicated in bold in Table 4, the correlation coefficients for the corresponding items on the EORTC QLQL-C30 and the Japanese version of PRO-CTCAE were high, while those for the items that did not correspond tended to be low.

Table 3 Pearson correlation coefficients between the function score on the EORTC QLQ-C30 and items of the PRO-CTCAE
Full size table
Table 4 Pearson correlation coefficients between symptom assessment by the PRO-CTCAE versus the EORTC QLQ-C30
Full size table

The SRMs of the Japanese version of the PRO-CTCAE are shown in Fig. 3  (see Appendix 2). Concerning the global health state, the means (95% CI) of the SRMs according to the three GIC categories were worse [−0.002 (0.058)], almost the same [−0.019 (0.020)], and better [−0.257 (0.022)], respectively. Regarding physical and emotional state, the means (95% CI) were as follows: physical state: worse, 0.257 (0.209–0.304); almost the same, 0.005 (−0.040–0.047); and better − 0.101 (−0.210–0.007); emotional state: worse, 0.353 (0.295–0.412); almost the same, 0.012 (−0.030–0.052); and better, 0.004 (−0.120–0.124); for the three GIC categories, respectively. In all states of the GIC, significant trends (namely dose dependent relationships) were observed (Jonckheere–Terpstra test, ps < 0.001).

Fig. 3
figure 3

Standardized response means of 39 PRO-CTCAE items by patient-reported global impression of change (Global health, physical, and emotioinal state). Significant trends were observed in each status. (Jonckheere–Terpstra test, ps < 0.001)

Full size image

Discussion

This study aimed to investigate the discordance of adverse event assessment between clinicians and patients using the CTCAE and the Japanese version of the PRO-CTCAE, as well as to survey the psychometric properties of the Japanese version of the PRO-CTCAE. Results indicate that there is a discrepancy in evaluation between clinicians and patients, depending on the adverse events, and that the validity and reliability of the Japanese version of the PRO-CTCAE is acceptable.

First, our findings indicated that the CTCAE grades and the responses to the PRO-CTCAE did not identically correspond. Thus, we evaluated the discordance of adverse event assessment in this study, via patient self-assessment using the PRO-CTCAE, in patients assessed by medical staff as having no adverse events, via the CTCAE. Underestimation of adverse events by clinicians have been previously reported using the CTCAE and/or EORTC QLQ-C30 [3, 4, 7]. These reports have consistently shown an underestimation of fatigue, pain, and constipation by clinicians, a trend mirrored in the current study. Additionally, our results also showed that anxiety was underestimated. The Japanese version of the PRO-CTCAE was shown to have properties similar to trends demonstrated in previous studies utilizing this measure in other countries. In Japan, there is a dearth of research examining the underestimation of adverse events. Okamoto has proposed that patients believe that doctors do not actively engage in listening to their concerns, while doctors tend to believe that patients are not willing to voice their opinions [34]. Within the unique context of Japanese physician-patient relationships, the Japanese version of the PRO-CTCAE is considered likely to affect the underestimation of adverse events, resulting in a more appropriate assessment. Finally, although this study revealed the existence of underestimation of adverse events in Japan, the sample utilized contained multiple different cancer types and treatment regimens. Further studies will be necessary to clarify discordance of particular treatments and/or cancer-type-specific adverse event assessments.

In addition, although almost all of the ICCs in evaluating the reproducibility of the Japanese version of the PRO-CTCAE were fair or good, the ICCs for “vomiting, severity (0.35)” and “decreased appetite, severity (0.48)” were relatively low. In almost all patients, these data evaluated the degree of agreement before and after the start of chemotherapy, and this timing is often defined as the acute phase in clinical trials to assess the effectiveness of antiemetic agents. A recent study in a Japanese population showed that the Complete Response (no vomiting/retching and no rescue medication) and Complete Control (no vomiting/retching, no rescue medication, and no more than mild nausea) of nausea and vomiting in the acute phase of highly emetogenic chemotherapy is approximately 75 to 90%, and the Total Control (no vomiting/retching, no rescue medication, and no nausea) is between 80 and 87% [35,36,37]. As can be seen from these data, it is well known that chemotherapy causes nausea and vomiting in the acute phase, much of which can be alleviated by antiemetic agents, although not completely controlled. The low ICCs found for some items may reflect symptomatic changes in the acute phase before and after chemotherapy.

Within the attributes of the same item, there was not much difference between ICC values. However, for “Numbness or tingling in hands or feet” there was a relatively large difference between severity and interference. A similar tendency exists in the original version [16]. In many cases, chemotherapy-induced neuropathy becomes reversible to irreversible in a dose-dependent fashion. However, because the ICC for severity in “Numbness or tingling in hands or feet” was shown to be good in the current study, it is considered that the low value of the interference ICC is not due to a change in this symptom. Recently, a multidimensional scale for assessing chemotherapy-induced peripheral neuropathy (CIPN), called the EORTC QLQ CIPN20, has been developed [38]. Wolf and colleagues conducted a study using this scale and argued that the evaluation of numbness of hands and feet should be strictly distinguished [39]. They revealed that neuropathic symptoms, such as numbness or tingling, tend to appear more strongly in lower extremities than upper extremities. Therefore, it is considered that a change in the influence on daily life, especially in the feet, caused by hospitalization owing to the start of chemotherapy, is one of the reasons for the deviation in the ICC for “Numbness or tingling in hands or feet.” It should be noted when interpreting the change in this item however, that the data currently available are preliminary.

This study has several limitations. First, the psychometric properties of the Japanese version of the PRO-CTCAE revealed in this study were investigated in a subset of core items that were specifically selected as a result of previous research. Furthermore, although it is the same as the original validation study, test-retest reliability was examined with only 14 items on the Japanese PRO-CTCAE. Although it seems to be sufficient for interpretation, it should be noted that these findings are not the result of investigating all items of the Japanese version of the PRO-CTCAE. Secondly, we were unable to recruit patients with poor performance status. Although Dueck provided known group validity for these individuals during development of the original version [16], the relationship between this group and the Japanese version of the PRO-CTCAE was not clarified. Finally, the current study focused on underestimation of adverse events but did not assess overestimation. Future research should consider whether there is an overestimation of adverse events that leads to overtreatment.

Conclusions

This study revealed that the discordance between clinician and patient assessments of adverse events was similar to that of previous reports, and that the Japanese version of the PRO-CTCAE demonstrated acceptable reliability and validity for common and clinically important symptoms. It is expected that the Japanese version of the PRO-CTCAE will be applied to patient-centered evaluation of adverse events in future clinical trials in Japan.

Abbreviations

CIPN:

Chemotherapy-induced peripheral neuropathy

CTCAE:

Common Terminology Criteria for Adverse Events

CTEP:

Cancer Therapy Evaluation Program

ECOG PS:

Eastern Cooperative Oncology Group Performance Status scale

EORTC QLQ CIPN20:

European Organisation of Research and Treatment of Cancer Quality of Life Chemotherapy-Induced peripheral neuropathy

EORTC QLQ-C30:

European Organisation of Research and Treatment of Cancer Quality of Life Questionnaire Core 30

ePRO:

Electronic patient-reported outcomes

GHS/QOL:

Global Health Status/Quality of Life

GIC:

Global impression of change

ICC:

Intra-class correlation coefficient

JCOG:

Japan Cooperative Oncology Group

KPS:

Karnofsky Performance Status

MedDRA:

Medical Dictionary for Regulatory Activities

NCI:

National Cancer Institute

PRO:

Patient-reported outcomes

PRO-CTCAE:

Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events

SRM:

Standardized response means

References

  1. Trotti, A., Colevas, A. D., Setser, A., & Basch, E. (2007). Patient-reported outcomes and the evolution of adverse event reporting in oncology. Journal of Clinical Oncology, 25(32), 5121–5127.

    Article  Google Scholar 

  2. Bruner, D. W. (2007). Should patient-reported outcomes be mandatory for toxicity reporting in cancer clinical trials? Journal of Clinical Oncology, 25(34), 5345–5347.

    Article  Google Scholar 

  3. Basch, E., Jia, X., Heller, G., Barz, A., Sit, L., Fruscione, M., Appawu, M., Iasonos, A., Atkinson, T., Goldfarb, S., Culkin, A., Kris, M. G., & Schrag, D. (2009). Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. Journal of the National Cancer Institute, 101(23), 1624–1632.

    Article  Google Scholar 

  4. Basch, E., Iasonos, A., McDonough, T., Barz, A., Culkin, A., Kris, M. G., Scher, H. I., & Schrag, D. (2006). Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. The Lancet Oncology, 7(11), 903–909.

    Article  Google Scholar 

  5. Xiao, C., Polomano, R., & Bruner, D. W. (2013). Comparison between patient-reported and clinician-observed symptoms in oncology. Cancer Nursing, 36(6), E1–E16.

    Article  Google Scholar 

  6. Fromme, E. K., Eilers, K. M., Mori, M., Hsieh, Y.-C., & Beer, T. M. (2004). How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the Quality-of-Life Questionnaire C30. Journal of Clinical Oncology, 22(17), 3485–3490.

    Article  Google Scholar 

  7. Laugsand, E. A., Sprangers, M. A., Bjordal, K., Skorpen, F., Kaasa, S., & Klepstad, P. (2010). Health care providers underestimate symptom intensities of cancer patients: a multicenter European study. Health and Quality of Life Outcomes, 8(1), 104.

    Article  Google Scholar 

  8. Greimel, E. R., Bjelic-Radisic, V., Pfisterer, J., Hilpert, F., Daghofer, F., Pujade-Lauraine, E., Du Bois, A., & Arbeitsgemeinschaft Gynaekologische Onkologie Ovarian Cancer Study Group (AGO-OVAR), Groupe d’Investigateurs Nationaux pour les Etudes des Cancers de l’Ovaire (GINECO). (2011). Toxicity and quality of life outcomes in ovarian cancer patients participating in randomized controlled trials. Support Care Cancer, 19(9), 1421–1427.

    Article  Google Scholar 

  9. Basch, E. (2010). The missing voice of patients in drug-safety reporting. The New England Journal of Medicine, 362(10), 865–869.

    Article  CAS  Google Scholar 

  10. Cirillo, M., Venturini, M., Ciccarelli, L., Coati, F., Bortolami, O., & Verlato, G. (2009). Clinician versus nurse symptom reporting using the National Cancer Institute—Common Terminology Criteria for Adverse Events during chemotherapy: results of a comparison based on patient’s self-reported questionnaire. Annals of Oncology, 20(12), 1929–1935.

    Article  CAS  Google Scholar 

  11. Novello, S., Capelletto, E., Cortinovis, D., Tiseo, M., Galetta, D., Valmadre, G., Casartelli, C., Rapetti, S. G., & Rossi, A. (2014). Italian multicenter survey to evaluate the opinion of patients and their reference clinicians on the “tolerance” to targeted therapies already available for non-small cell lung cancer treatment in daily clinical practice. Translational Lung Cancer Reseacrh, 3(3), 173–180.

    CAS  Google Scholar 

  12. Di Maio, M., Gallo, C., Leighl, N. B., Piccirillo, M. C., Daniele, G., Nuzzo, F., Gridelli, C., Gebbia, V., Ciardiello, F., & De Placido, S. (2015). Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. Journal of Clinical Oncology, 33(8), 910–915.

    Article  CAS  Google Scholar 

  13. Montemurro, F., Mittica, G., Cagnazzo, C., Longo, V., Berchialla, P., Solinas, G., Culotta, P., Martinello, R., Foresto, M., & Gallizioli, S. (2016). Self-evaluation of adjuvant chemotherapy-related adverse effects by patients with breast cancer. JAMA Oncology, 2(4), 445–452.

    Article  Google Scholar 

  14. Basch, E., Reeve, B. B., Mitchell, S. A., Clauser, S. B., Minasian, L. M., Dueck, A. C., Mendoza, T. R., Hay, J., Atkinson, T. M., Abernethy, A. P. B. D. W., Cleeland, C. S., Sloan, J. A., Chilukuri, R., Baumgartner, P., Denicoff, A., St. Germain, D., O’Mara, A. M., Chen, A., Kelaghan, J., Bennet, A. V., Sit, L., Rogak, L., Barz, A., Paul, D. B., & Schrag, D. (2014). Development of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Journal of the National Cancer Institute, 106(9), dju244.

    Article  Google Scholar 

  15. Hay, J. L., Atkinson, T. M., Reeve, B. B., Mitchell, S. A., Mendoza, T. R., Willis, G., Minasian, L. M., Clauser, S. B., Denicoff, A., O’Mara, A., Chen, A., Bennet, A. V., Paul, D. B., Gagne, J., Rogak, L., Sit, L., Viswanath, V., Schrag, D., Basch, E., & NCI PRO-CTCA Study Group. (2014). Cognitive interviewing of the US National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Quality of Life Research, 23(1), 257–269.

    Article  Google Scholar 

  16. Dueck, A. C., Mendoza, T. R., Mitchell, S. A., Reeve, B. B., Castro, K. M., Rogak, L. J., Atkinson, T. M., Bennett, A. V., Denicoff, A. M., O’Mara, A. M., Li, Y., Clauser, S. B., Bryant, D. M., Bearden, J. D., Gillis, T. A., Harness, J. K., Siegel, R. D., Paul, D. B., Cleeland, C. S., Schrag, D., Sloan, J. A., Abernethy, A. P., Bruner, D. W., Minasian, L. M., Basch, E., & National Cancer Institute PRO-CTCAE Study Group. (2015). Validity and reliability of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). JAMA Oncology, 1(8), 1051–1059.

    Article  Google Scholar 

  17. Kirsch, M., Mitchell, S. A., Dobbels, F., Stussi, G., Basch, E., Halter, J. P., & De Geest, S. (2015). Linguistic and content validation of a German-language PRO-CTCAE-based patient-reported outcomes instrument to evaluate the late effect symptom experience after allogeneic hematopoietic stem cell transplantation. European Journal of Oncology Nursing, 19(1), 66–74.

    Article  Google Scholar 

  18. Arnold, B., Mitchell, S. A., Lent, L., Mendoza, T. R., Rogak, L. J., Barragán, N. M., Willis, G., Medina, M., Lechner, S., Penedo, F. J., Harness, J. K., Basch, E. M., & PRO-CTCAE Spanish Translation and Linguistic Validation Study Group. (2016). Linguistic validation of the Spanish version of the National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Support Care Cancer, 24(7), 2843–2851.

    PubMed  Google Scholar 

  19. Bæksted, C., Nissen, A., Pappot, H., Bidstrup, P. E., Mitchell, S. A., Basch, E., Dalton, S. O., & Johansen, C. (2016). Danish Translation and Linguistic Validation of the US National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Journal of Pain and Symptom Management, 52(2), 292–297.

    Article  Google Scholar 

  20. Miyaji, T., Iioka, Y., Kuroda, Y., Yamamoto, D., Iwase, S., Goto, Y., Tsuboi, M., Odagiri, H., Tsubota, Y., Kawaguchi, T., Sakata, N., Basch, E., Yamaguchi, T. Japanese translation and linguistic validation of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).

  21. Reeve, B. B., Mitchell, S. A., Dueck, A. C., Basch, E., Cella, D., Reilly, C. M., Minasian, L. M., Denicoff, A. M., O’Mara, A. M., & Fisch, M. J. (2014). Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. Journal of the National Cancer Institute, 106(7), dju129.

    Article  Google Scholar 

  22. Schag, C. C., Heinrich, R. L., & Ganz, P. (1984). Karnofsky performance status revisited: reliability, validity, and guidelines. Journal of Clinical Oncology, 2(3), 187–193.

    Article  CAS  Google Scholar 

  23. Oken, M. M., Creech, R. H., Tormey, D. C., Horton, J., Davis, T. E., Mcfadden, E. T., & Carbone, P. P. (1982). Toxicity and response criteria of the Eastern Cooperative Oncology Group. American Journal of Clinical Oncology, 5(6), 649–656.

    Article  CAS  Google Scholar 

  24. Gómez, H., Hidalgo, M., Casanova, L., Colomer, R., Pen, D., Otero, J., Rodriguez, W., Carracedo, C., Cortes-Funes, H., & Vallejos, C. (1998). Risk factors for treatment-related death in elderly patients with aggressive non-Hodgkin's lymphoma: results of a multivariate analysis. Journal of Clinical Oncology, 16(6), 2065–2069.

    Article  Google Scholar 

  25. Lee, S. S., Ahn, J.-H., Kim, M. K., Sym, S. J., Gong, G., Ahn, S. D., Kim, S.-B., & Kim, W. K. (2008). Brain metastases in breast cancer: prognostic factors and management. Breast Cancer Research and Treatment, 111(3), 523–530.

    Article  Google Scholar 

  26. Mani, S., Todd, M. B., Katz, K., & Poo, W.-J. (1995). Prognostic factors for survival in patients with metastatic renal cancer treated with biological response modifiers. The Journal of Urology, 154(1), 35–40.

    Article  CAS  Google Scholar 

  27. Manola, J., Atkins, M., Ibrahim, J., & Kirkwood, J. (2000). Prognostic factors in metastatic melanoma: a pooled analysis of Eastern Cooperative Oncology Group trials. Journal of Clinical Oncology, 18(22), 3782–3793.

    Article  CAS  Google Scholar 

  28. Quinten, C., Coens, C., Mauer, M., Comte, S., Sprangers, M. A., Cleeland, C., Osoba, D., Bjordal, K., Bottomley, A., & Clinical Groups, E. O. R. T. C. (2009). Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials. The Lancet Oncology, 10(9), 865–871.

    Article  Google Scholar 

  29. Kobayashi, K., Takeda, F., Teramukai, S., Gotoh, I., Sakai, H., Yoneda, S., Noguchi, Y., Ogasawara, H., & Yoshida, K. (1998). A cross-validation of the European Organization for Research and Treatment of Cancer QLQ-C30 (EORTC QLQ-C30) for Japanese with lung cancer. European Journal of Cancer, 34(6), 810–815.

    Article  CAS  Google Scholar 

  30. Bennett, A. V., Dueck, A. C., Mitchell, S. A., Mendoza, T. R., Reeve, B. B., Atkinson, T. M., Castro, K. M., Denicoff, A., Rogak, L. J., Harness, J. K., Bearden, J. D., Bryant, D., Siegel, R. D., Schrag, D., Basch, E., & National Cancer Institute PRO-CTCA Study Group. (2016). Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Health and Quality of Life Outcomes, 14(1), 24.

    Article  Google Scholar 

  31. Coons, S. J., Gwaltney, C. J., Hays, R. D., Lundy, J. J., Sloan, J. A., Revicki, D. A., Lenderking, W. R., Cella, D., & Basch, E. (2009). Recommendations on evidence needed to support measurement equivalence between electronic and paper-based patient-reported outcome (PRO) measures: ISPOR ePRO Good Research Practices Task Force report. Value in Health, 12(4), 419–429.

    Article  Google Scholar 

  32. Reeve, B. B., Wyrwich, K. W., Wu, A. W., Velikova, G., Terwee, C. B., Snyder, C. F., Schwartz, C., Revicki, D. A., Moinpour, C. M., & McLeod, L. D. (2013). ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Quality of Life Research, 22(8), 1889–1905.

    Article  Google Scholar 

  33. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6(4), 284–290.

    Article  Google Scholar 

  34. Okamoto, S. (2007). Transformations in doctor-patient communication in Japan: the role of cultural factors. Patient Education and Counseling, 65(2), 153–155.

    Article  Google Scholar 

  35. Suzuki, K., Yamanaka, T., Hashimoto, H., Shimada, Y., Arata, K., Matsui, R., Goto, K., Takiguchi, T., Ohyanagi, F., Kogure, Y., Nogami, N., Nakao, M., Takeda, K., Azuma, K., Nagase, S., Hayashi, T., Fujiwara, K., Shimada, T., Seki, N., & Yamamoto, N. (2016). Randomized, double-blind, phase III trial of palonosetron versus granisetron in the triplet regimen for preventing chemotherapy-induced nausea and vomiting after highly emetogenic chemotherapy: TRIPLE study. Annals of Oncology, 27(8), 1601–1606.

    Article  CAS  Google Scholar 

  36. Komatsu, Y., Okita, K., Yuki, S., Furuhata, T., Fukushima, H., Masuko, H., Kawamoto, Y., Isobe, H., Miyagishima, T., Sasaki, K., Nakamura, M., Ohsaki, Y., Nakajima, J., Tateyama, M., Eto, K., Minami, S., Yokoyama, R., Iwanaga, I., Shibuya, H., Kudo, M., Oba, K., & Takahashi, Y. (2015). Open-label, randomized, comparative, phase III study on effects of reducing steroid use in combination with Palonosetron. Cancer Science, 106(7), 891–895.

    Article  CAS  Google Scholar 

  37. Saito, M., Aogi, K., Sekine, I., Yoshizawa, H., Yanagita, Y., Sakai, H., Inoue, K., Kitagawa, C., Ogura, T., & Mitsuhashi, S. (2009). Palonosetron plus dexamethasone versus granisetron plus dexamethasone for prevention of nausea and vomiting during chemotherapy: a double-blind, double-dummy, randomised, comparative phase III trial. The Lancet Oncology, 10(2), 115–124.

    Article  CAS  Google Scholar 

  38. Postma, T., Aaronson, N., Heimans, J., Muller, M., Hildebrand, J., Delattre, J.-Y., Hoang-Xuan, K., Lanteri-Minet, M., Grant, R., & Huddart, R. (2005). The development of an EORTC quality of life questionnaire to assess chemotherapy-induced peripheral neuropathy: the QLQ-CIPN20. European Journal of Cancer, 41(8), 1135–1139.

    Article  CAS  Google Scholar 

  39. Wolf, S. L., Barton, D. L., Qin, R., Wos, E. J., Sloan, J. A., Liu, H., Aaronson, N. K., Satele, D. V., Mattar, B. I., Green, N. B., & Loprinzi, C. L. (2012). The relationship between numbness, tingling, and shooting/burning pain in patients with chemotherapy-induced peripheral neuropathy (CIPN) as measured by the EORTC QLQ-CIPN20 instrument, N06CA. Support Care Cancer, 20(3), 625–632.

    Article  Google Scholar 

Download references

Acknowledgments

The web-based electric data capture system, DEMAND™, was provided by Densuke Systems Cooperation Limited, Japan. The authors are grateful to the Japan Clinical Oncology Group, Yasushi Goto, Masahiro Tsuboi, Daigo Yamamoto, Hiroki Odagiri, Satoru Iwase, Yu Tsubota, and Sandra A. Mitchell for supporting this project. We thank Emi Fu, Hisako Ikezaki, Atsushi Okubo, KwiSeon Min, Megumi Nakamura, Tohru Kishino, Hiromi Shimada, Reiko Nunoya, Ayako Torii, Yuri Yamada, Takayuki Seki, Hideaki Ayuhara, Taiki Hirata, and Takao Akashi for their assistance in conducting the study. We would like to thank Editage (www.editage.com) for English language editing and Publication Support.

Funding

This work was supported by Japan Society for the Promotion of Science KAKENHI Grant Number 24590595, principal investigator Dr. Takuhiro Yamaguchi.

Author information

Authors and Affiliations

  1. Department of Practical Pharmacy, School of Pharmacy, Tokyo University of Pharmacy and Life Sciences, 1432-1, Horinouchi, Hachioji, Tokyo, Japan

    Takashi Kawaguchi

  2. Department of Pharmacy, Tokyo Medical University Hospital, 6-7-1, Nishishinjuku, Shinjuku-ku, Tokyo, Japan

    Kanako Azuma, Yoko Sano & Tomohide Shimodaira

  3. Department of Pharmacy Services, Saitama Medical Center, Saitama Medical University, 1981, Kamoda, Kawagoe-city, Saitama, Japan

    Motohiko Sano

  4. Department of Pharmacy, Juntendo University Nerima Hospital, 3-1-10, Takanodai, Nerima-ku, Tokyo, Japan

    Soan Kim

  5. Department of Pharmacy, Toshiba General Hospital, 6-3-22, Higashioi, Shinagawa-ku, Tokyo, Japan

    Yosuke Kawahara

  6. Department of Digestive Tract and General Surgery, Saitama Medical Center, Saitama Medical University, 1981, Kamoda, Kawagoe-city, Saitama, Japan

    Keiichiro Ishibashi

  7. Department of Clinical Trial Data Management, Graduate School of Medicine, The University of Tokyo, 7-3-1, Hongo, Bunkyo-ku, Tokyo, Japan

    Tempei Miyaji

  8. Division of Health Care Research, QOL Research Group, Center for Public Health Sciences, National Cancer Center, 5-1-1 Tsukiji, Tokyo, Japan

    Tempei Miyaji & Takuhiro Yamaguchi

  9. Department of Medicine, University of North Carolina, Chapel Hill, NC, USA

    Ethan Basch

  10. Division of Biostatistics, Tohoku University Graduate School of Medicine, 1-1, Seiryo-machi, Aoba-ku, Sendai, Miyagi, Japan

    Takuhiro Yamaguchi

Authors
  1. Takashi Kawaguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kanako Azuma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Motohiko Sano
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Soan Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yosuke Kawahara
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yoko Sano
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tomohide Shimodaira
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Keiichiro Ishibashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Tempei Miyaji
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ethan Basch
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Takuhiro Yamaguchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

TK, KA, MS, SK, YK, TM, EB and TY were involved in the design of the study and development of protocol. KA, MS, SK, YK, YS, TS, and KI participated in the enrollment and data collection. TK, TM, and TY performed in the data analysis and participated in drafting the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Takashi Kawaguchi.

Ethics declarations

Ethics approval and consent to participate

The study protocol was approved by the institutional review board at each institution (Independent Ethics Committee of Tokyo Medical University, No.2758, 29 May 2014. Institutional Review Board of Saitama Medical Center, Saitama Medical University, No.1074, 27 November 2014. Ethics Committee of The Juntendo University Nerima Hospital, No. 14–31, 11 November 2014. Ethics Committee of Toshiba General Hospital, No. 14-A-14, 25 August 2014.) The study was conducted in accordance with the Japanese Ethical Guidelines for Epidemiological Research and was registered in the University Hospital Medical Information Network Clinical Trials Registry (UMIN000014277). All procedures performed were in accordance with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Consent for publication

Written informed consent was obtained from all individual participants included in this study.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix 1

Table 5 Japanese version of the PRO-CTCAE items used in the current study
Full size table

Appendix 2

Table 6 Standardized response mean in Cohort B (n = 100)
Full size table

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kawaguchi, T., Azuma, K., Sano, M. et al. The Japanese version of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE): psychometric validation and discordance between clinician and patient assessments of adverse events. J Patient Rep Outcomes 2, 2 (2018). https://doi.org/10.1186/s41687-017-0022-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-017-0022-5

Keywords