Japanese translation and linguistic validation of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE)
Journal of Patient-Reported Outcomes volume 1, Article number: 8 (2017)
The US National Cancer Institute (NCI) has developed the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) to capture patients’ self-reported symptomatic adverse events in cancer clinical trials. The aim of this study was to develop and linguistically validate a Japanese translation of PRO-CTCAE. Forward- and back-translations were produced, and an independent review was performed by the Japan Clinical Oncology Group (JCOG) Executive Committee and the US NCI. We then conducted cognitive interviews with 21 patients undergoing cancer treatment. Participants were asked to complete the PRO-CTCAE and were interviewed using semi-structured scripts and predetermined probes to investigate whether any items were difficult to understand or answer. The interviews were recorded and transcribed, and a thematic analysis was performed. The data were split into two categories: 1) remarks on the items and 2) remarks on the questionnaire in general.
Twenty-one cancer patients undergoing chemotherapy or hormone therapy were interviewed at the University of Tokyo Hospital and the Kansai Medical University Hirakata Hospital during 2011 and 2012. Thirty-three PRO-CTCAE items were evaluated as “difficult to understand,” and 65 items were evaluated as “difficult to answer” by at least one respondent. However, on further investigation, only 24 remarks were categorized as “comprehension difficulties” or “clarity” issues. Most of these remarks concerned patients’ difficulties with rating their experience of individual symptomatic events.
The study provides preliminary evidence supporting the linguistic validity of the Japanese version of PRO-CTCAE. Further cognitive interviewing is warranted for PRO-CTCAE items relating to sexuality and anxiety and for response options on severity attribute items.
The US National Cancer Institute (NCI) Common Toxicity Criteria for Adverse Events (CTCAE) is a longstanding grading system widely used for evaluating an array of adverse events in cancer treatment. There is a growing awareness of the importance of patients’ self-reporting of symptoms using patient-reported outcome (PRO) tools in addition to professional assessments [1,2,3]. In order to improve the accuracy and efficiency of collecting and grading adverse event data, the NCI has established the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) measurement system [4,5,6,7]. The PRO-CTCAE item library contains 124 items reflecting 78 symptomatic adverse events drawn from the CTCAE version 4.0. The PRO-CTCAE was developed to be used in conjunction with the CTCAE . PRO-CTCAE symptom attributes include frequency (F) (e.g., “In the last 7 days, how often did you have nausea?”), severity (S) (e.g., “In the last 7 days, what was the severity of your pain?”), interference with daily activities (I) (e.g., “How much did fatigue interfere with your usual or daily activities?”), and presence/absence/amount (P) (e.g., “In the last 7 days, did you have any bed sores?”). PRO-CTCAE scores range from 0 to 4, corresponding to the response choices. The recall period for each question is the previous 7 days . Content validation and a psychometric evaluation of the PRO-CTCAE measurement system have already been performed [6, 9] and their feasibility investigated in a multicenter oncology trial . In addition, cognitive interview-based validation was performed in an adolescents-with-cancer population  and a pediatric and prosy version of the PRO-CTCAE is also being developed . As part of the development of PRO-CTCAE, the NCI has collaborated on the development of versions of PRO-CTCAE in multiple other languages, such as Spanish , German , and Danish  language versions. A Japanese-language version was also created, in a joint endeavor by Tohoku University, The University of Tokyo, and the Japan Clinical Oncology Group (JCOG). The purpose of the Japanese translation is to encourage the use of PRO-CTCAE among Japanese-speaking cancer patients living in the USA, as well as those in Japan .
Estimated cancer incidence in Japan in 2016 was approximately one million, and estimated cancer deaths were 374,000. It is estimated that one in two Japanese will be diagnosed with cancer during their lifetime . Accessing subjective symptoms of Japanese cancer patients by self-report has come to be viewed as important as the objective clinical endpoints such as response rate, disease-free survival, and overall survival . The aim of this study was to translate the PRO-CTCAE into Japanese and linguistically validate the Japanese version.
The term “linguistic validation” often refers to the process of investigating the conceptual equivalence and content validity of translation of PRO instruments, including forward- and back-translation, review, harmonization, and cognitive debriefing [19, 20]; it does not include psychometric validation, which is nonetheless seen as an important second step for cross-cultural validation . This linguistic validation study is part of a cross-cultural validation; psychometric evaluation was conducted in another study with a different 180 participants (UMIN CTR: UMIN000015169). The results of psychometric validation are beyond the scope of the current report and will be presented separately.
The PRO-CTCAE questionnaire was translated into Japanese following the guidance of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) on the “Translation and Cultural Adaptation Process for PRO Measures” . Two forward-translations were independently produced by two native-Japanese-speaking language professionals, after which a single reconciled version was created by the project manager and these two translators. Two native English-speakers who are fluent in Japanese then produced independent back-translations. An independent review was performed by the JCOG Executive Committee, and the finalized Japanese translation was reviewed and approved by a bilingual medical oncologist at the US NCI.
Participants and setting
Patients were eligible for the study if they were 18 years of age or older; had been diagnosed with cancer; were receiving or had received chemotherapy, radiotherapy, or hormone therapy within the past 6 months at one of the study sites; and were able to read, write, and answer survey items and interview questions verbally in Japanese as their first language.
Patients were excluded if the physician determined that they had cognitive impairments or were incapable of participating in the study. Patients were recruited at the Kansai Medical University Hirakata Hospital and The University of Tokyo Hospital. Regarding the sample size determination, as a literature review of guidelines suggested that the required sample size of cognitive interviews for translation of HQOL questionnaire is around 5 to 15 , we aimed to include more than 16 patients while reflecting differences in cancer type, gender, regional characteristics, and educational background.
Cognitive interview goal and procedure
In total, 123 translated items were divided into two questionnaires (A and B) per symptom, to reduce participant burden. Although the PRO-CTCAE library consists of 124 items, the item “vaginal dryness” was not included in the cognitive interviews as it was a sexually sensitive question. As six items are gender-specific, two versions of each questionnaire were prepared. Furthermore, six symptoms (nine items) were shifted from questionnaire A to B when the cognitive interviews took place at the University of Tokyo Hospital to minimize the item number imbalance between questionnaires; a total of eight questionnaires were prepared for the cognitive interviews. Each questionnaire consists of about 61 items on average (50 items at minimum and 68 at maximum).
The cognitive interviews were conducted by master’s-level psychologists with field experience. The following study procedures were followed: 1) investigators at the site screened patients based on the eligibility criteria; 2) interviewers explained the details of the study to participants and obtained informed consent; 3) interviewers scheduled interviews; 4) on the day of the interview, interviewers reconfirmed participants’ willingness to participate; 5) participants were asked to fill out a gender-specific questionnaire; and 6) one-on-one semi-structured interviews were conducted.
Participants were asked to self-complete the PRO-CTCAE and make remarks on the items and response options they evaluated as “difficult to understand” or “difficult to answer” regarding: 1) comprehensibility; 2) clarity; 3) knowledge and recall (ease of memory retrieval); and 4) judgment (adequacy of response options). Participants were then interviewed using semi-structured scripts and predetermined probes to investigate any items that were difficult to comprehend if the response choices and recall period were clear and relevant to their experiences. Reluctance or hesitation to answer and spontaneous questions, responses, or facial expressions while answering were also observed by the interviewer. The interviews took place in a private room.
We conducted a qualitative-descriptive analysis using semi-structured cognitive interviews. The interviews were transcribed word-for-word, examined repeatedly, and categorized. Remarks were split into two categories: 1) remarks on the items and 2) remarks on the questionnaire in general. The remarks on the items were further categorized into those concerning 1) comprehension difficulties, 2) clarity, 3) knowledge and recall (ease of memory retrieval), and 4) ease of judgment and other spontaneous comments. The remarks on the questionnaire were also further categorized into those concerning 1) attributes, 2) ease of memory retrieval and judgment, 3) other remarks on the questionnaire, and 4) other remarks in general. As a single patient could make several remarks, the total number of remarks does not reflect the number of patients. We performed descriptive statistics to summarize the variables for the cognitive interviews.
Comparing the two independent forward-translations, 52 of 80 PRO-CTCAE symptom terms were fully agreed upon or conceptually equivalent. In the reconciliation process, the term “how often” in the frequency attribute items was altered so as not to be directly expressed, due to an element of the Japanese cultural background. In Japanese custom, asking about frequency of symptoms before asking about the presence of symptoms, that is, assuming their presence, is unnatural and impolite. The use of “how often” was flagged by the JCOG executive committee review, and so in the further revision process, we agreed to not translate “how often” directly in the question but instead to refer to “frequency” in the response options. For severity attribute items, translation of “its worst” became a discussion topic during the forward-translation reconciliation process. In Japanese, there are objective and subjective terms corresponding to the term “its worst”; for the test script, we changed “its worst” to “hidoi,” which is an objective term, and further solicited opinions on two options in the cognitive interviews. In the review by the bilingual medical oncologist from NCI, there were no major suggestions for change. One minor suggestion was made to add supplementary explanation to the “radiation skin reaction” item indicating that this question is intended for cancer patients receiving radiation therapy.
A total of 21 cancer patients (six males and 15 females) participated in interviews at either the University of Tokyo Hospital or the Kansai Medical University Hirakata Hospital between August 2011 and April 2012. Table 1 shows the clinical and demographic characteristics of the participants. The patients were 40 to 80 years old, with a mean of 64 years. Ten had been diagnosed with breast cancer, seven with lung cancer, three with pancreatic cancer, and one with esophageal cancer. All participants except one, who was receiving hormone therapy, were undergoing chemotherapy at the time of the interview. Nineteen participants had been receiving the cancer treatment less than 1 month, one participant for 1 to 3 months, and one participant for more than 12 months. The Eastern Cooperative Oncology Group (ECOG) performance status was 0 for 9 participants, 1 for 8 participants, 2 for 2 participants, and 3 for 2 participants. The participants took an average of 30 min to complete the questionnaires of approximately 61 items; skip patterns were not employed.
Table 2 lists the questionnaire items, showing the proportion of patients who evaluated each PRO-CTCAE item as “difficult to understand” and “difficult to answer” respectively. The patients self-completed 45 items without any difficulties; 33 items were evaluated as “difficult to understand,” and 65 as “difficult to answer.” Only one PRO-CTCAE item (anxiety severity) was found to be “difficult to understand” by more than 20% of the respondents. Eight items were evaluated as “difficult to answer” by more of 20% of the respondents; three of these were items related to sexual activities, such as “decreased sexual interest,” “unable to have an orgasm or climax,” and “took too long to have an orgasm or climax.”
Analyzing the 94 narrative comments provided on the items across the debriefing interviews, 11 remarks concerned comprehension difficulties, 13 remarks concerned clarity, 25 remarks concerned knowledge and recall (ease of memory retrieval), and 45 remarks concerned ease of judgment or spontaneous comments. These remarks thus applied to 52 out of the 123 items, with 71 items not remarked on. Patients found it difficult to interpret some symptoms, such as “body odor” (5 remarks) or “problems with memory” (2 remarks), because their effects are relatively subjective. Determining the severity of persistent symptoms such as “anxiety” (4 remarks) or “leg/arm swelling” (1 remark) was also problematic for small numbers of participants. Problems determining whether something like “aching joints” included symptoms such as stiff shoulders or aching muscles were reported by five patients.
The following remarks related to the symptom attribute (40 remarks) in the questionnaire were categorized: no evaluation criteria are given (5 remarks), need for benchmark (6 remarks), difficulty choosing only one response (4 remarks), distinction between response options is ambiguous (7 remarks), response options are inadequate (6 remarks), questions and response options need to be reformed (7 remarks), and inconsistency between questions and response options (4 remarks), and other (1 remarks). The following remarks related to ease of memory retrieval and judgment (26 remarks) were categorized: some symptoms are hard to notice (6 remarks), need for exemplars of symptoms (10 remarks), significance of the question should be clarified (3 remarks), problems interpreting “interference with daily activities” (4 remarks), and problems apprehending the severity of symptoms at their worst (3 remarks). Other remarks on the questionnaire include the phrase “in the last 7 days,” which complicated questions (5 remarks). Participants with constant subjective symptoms in particular seemed to find it challenging to confine the interpretation of their symptom experience to only the severity or interference experienced “in the last 7 days.” Regarding the two alternative wordings of “its worst,” the responses were varied among responders and symptoms. Following agreement on changes among the project team, ultimately we kept the initial wording, which is the objective term discussed above, in the final version.
This study was conducted following rigorous procedures for translation and linguistic validation . Although the methodologies used were consistent with other translations of PRO-CTCAE, our sample was relatively small compared to those of other linguistic validation studies [13,14,15]. However, the study population partially represents the clinical, socioeconomic, and geographic diversities in the general Japanese cancer population. Moreover, there were no missing data, and the feasibility of the measurements was confirmed.
In total, 33 items were evaluated as “difficult to understand” and 65 items were evaluated as “difficult to answer” by one or more participants when the participants completed the questionnaire independently. However, when we asked them about these items in the interviews, only 24 out of their 94 remarks were categorized as relating to “comprehension difficulties” or issues with “clarity.” For the qualitative analysis, we set the threshold for consideration of rephrasing and testing of alternative phrasing at more than 20%. Table 3 shows the remarks on the nine PRO-CTCAE items that were evaluated as “difficult to understand” or “difficult to answer” by more than 20% of the respondents. Although some remarks on “anxiety” were categorized as “clarity,” they were related to the apprehension of symptomatic toxicities in the context of attributes, such as “the severity of anxiety at its worst.” Anxiety items were also found to be challenging in Spanish PRO-CTCAE cognitive testing . Remarks on other items were mostly related to problems with interpreting the symptom itself and judging individual symptomatic events. Nevertheless, we concluded that there was a good general understanding of these items, so no amendments to the Japanese translated version were required after the cognitive interviews.
Items related to sexual activities, such as “decreased sexual interest,” “unable to have an orgasm or climax,” and “took too long to have an orgasm or climax,” were evaluated as “difficult to answer” by more than 40% of patients. Moreover, “unable to have an orgasm or climax” was remarked on only once, and “took too long to have an orgasm or climax” was not remarked on at all, even though a high proportion of patients evaluated them as “difficult to answer.” This may have been because Japanese patients tend to be hesitant about answering questions regarding sexual activities . The lack of remarks on these items could also have been influenced by gender non-concordance between study participants and interviewers.
As discussed above, the patients comprehended the questions and response items well in general. However, interpreting symptoms using given response options was confusing for some. For example, three patients had issues with the severity-related wording “at its worst.” Chronic symptoms seemed to be especially difficult to evaluate in this regard. There were opinions that the inclusion of “mild” as a response option was not consistent with asking about the severity of a symptom “at its worst.” Similar finding with severity items were reported in cognitive interviews in an adolescent population in USA . Considering remarks on the questionnaire such as “need for a benchmark” or “distinction among response options is ambiguous,” it is clear that some patients find it difficult to respond using abstract terminology instead of clear benchmarks, which may reflect the sociocultural matter of Japanese respondents’ limited experience with patient-reported outcome measurements in general rather than the Japanese translation. Cultural differences exist in how Likert-type scales are responded to, and Japanese people have frequently reported difficultly doing so . Also, 4 out of 6 remarks on “need for a benchmark” were related to psychological symptoms, including sleep, mood and memory, and sensory aspects, and patients might have felt more difficulty choosing options for these compared to other types of symptoms. Nevertheless, the cognitive interviews showed a good general understanding of most of the items.
These study results should be interpreted with several caveats in mind, including the small overall sample size, the small number of respondents interviewed about each item, and the possibility that the use of gender-discordant interviewers may have made some participants reticent to discuss difficulties with PRO-CTCAE items that reflect sexual functioning and genitourinary symptoms. Also, we could not include Japanese-speaking cancer patients living in the USA, who were part of the target population of this questionnaire in the study. These caveats notwithstanding, this study provides preliminary evidence supporting the linguistic validity of the Japanese language version of PRO-CTCAE. These results should be confirmed and elaborated on with further psychometric validation studies covering construct validity, test–retest reliability, and sensitivity to detect change; such studies are currently in progress as a part of the larger cross-cultural validation of which this study is a part. The translation certificate was issued by the NCI, and the first version of the Japanese translation is currently available through the PRO-CTCAE website .
This study revealed that translation was conducted using a rigorous process and the majority of the items in the Japanese language version of PRO-CTCAE are well understood by Japanese-speaking cancer patients. The results provide preliminary evidence supporting the linguistic and content validity of the Japanese language version of PRO-CTCAE. However, further cognitive interviews are needed for items related to sexuality and to anxiety severity, as well as response options for severity attribute items.
Basch, E., Iasonos, A., McDonough, T., et al. (2006). Patient versus clinician symptom reporting using the National Cancer Institute common terminology criteria for adverse events: Results of a questionnaire-based study. The Lancet Oncol, 7(11), 903–909.
Basch, E. (2010). The missing voice of patients in drug-safety reporting. N Engl J Med, 362(10), 865–869.
Basch, E., Abernethy, A. P., Mullins, C. D., et al. (2012). Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J Clin Oncol, 30(34), 4249–4255.
Bruner, D. W., Hanisch, L. J., Reeve, B. B., et al. (2011). Stakeholder perspectives on implementing the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Transl Behav Med, 1(1), 110–122.
Basch, E., Reeve. B. B., Mitchell, S. A., et al. (2014). Development of the National Cancer Institute's patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Natl Cancer Inst, 106(9).
Hay, J. L., Atkinson, T. M., Reeve, B. B., et al. (2014). Cognitive interviewing of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Qual Life Res, 23(1), 257–269.
US National Cancer Institute. Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™). http://outcomes.cancer.gov/tools/pro-ctcae.html. Accessed 30 Aug 2017.
Mendoza, T. R., Dueck, A. C., Bennett, A. V., et al. (2017). Evaluation of different recall periods for the US National Cancer Institute's PRO-CTCAE. Clin Trials, 14(3), 255–263.
Dueck, A. C., Mendoza, T. R., Mitchell, S. A., et al. (2015). Validity and reliability of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). JAMA Oncol, 1(8), 1051–1059.
Basch, E., Pugh, S. L., Dueck, A. C., et al. (2017). Feasibility of patient reporting of symptomatic adverse events via the patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE) in a Chemoradiotherapy cooperative group multicenter clinical trial. Int J Radiat Oncol Biol Phys, 98(2), 409–418.
Reeve, B. B., McFatrich, M., Pinheiro, L. C., et al. (2017). Cognitive interview-based validation of the patient-reported outcomes version of the common terminology criteria for adverse events in adolescents with cancer. J Pain Symptom Manag, 53(4), 759–766.
Reeve, B. B., McFatrich, M., Pinheiro, L. C., et al. (2017). Eliciting the child's voice in adverse event reporting in oncology trials: Cognitive interview findings from the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events initiative. Pediatr Blood Cancer, 64(3).
Arnold, B., Mitchell, S. A., Lent, L., et al. (2016). Linguistic validation of the Spanish version of the National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). Support Care Cancer, 24(7), 2843–2851.
Kirsch, M., Mitchell, S. A., Dobbels, F., et al. (2015). Linguistic and content validation of a German-language PRO-CTCAE-based patient-reported outcomes instrument to evaluate the late effect symptom experience after allogeneic hematopoietic stem cell transplantation. Euro J Oncol Nurs, 19(1), 66–74.
Baeksted, C., Nissen, A., Pappot, H., et al. (2016). Danish translation and linguistic validation of the US National Cancer Institute’s patient-reported outcomes version of the common terminology criteria for adverse events (PRO-CTCAE). J Pain Symptom Manag, 52(2), 292–297.
Yamaguchi, T., Iwase, S., Kuroda, Y., et al. (2012). Introduction and current status of the development program for a Japanese version of the NCI PRO-CTCAE. Annu Meet Jpn Soc Med Oncol, 10, 134–135.
Center for Cancer Control and Information Services NCC, Japan. Projected Cancer Statistics 2016. http://ganjoho.jp/en/public/statistics/short_pred.html. Accessed 30 Aug 2017.
Kurihara, M., Shimizu, H., Tsuboi, K., et al. (1999). Development of quality of life questionnaire in Japan: Quality of life assessment of cancer patients receiving chemotherapy. Psycho-Oncol, 8(4), 355–363.
Acquadro C, Conway K, Hareendran A, Aaronson N, ERIQA Group. (2008). Literature review of methods to translate health-related quality of life questionnaires for use in multinational clinical trials. Value Health, 11(3), 509–521.
Wild, D., Grove, A., Martin, M., et al. (2005). Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: Report of the ISPOR task force for translation and cultural adaptation. Value Health, 8(2), 94–104.
Acquadro, C., Conway, K., Girourdet, C., & Mear, I. (2004). Linguistic validation manual for patient-reported outcomes (PRO) instruments. Lyon: MAPI Research Trust.
Kawaguchi, T., Iwase, S., Koinuma, M., et al. (2012). Determinants affecting quality of life: Implications for pharmacist counseling for patients with breast cancer in Japan. Biol Pharmacol Bull, 35(1), 59–64.
Lee, J. W., Jones, P. S., Mineyama, Y., & Zhang, X. E. (2002). Cultural differences in responses to a Likert scale. Res Nurs Health, 25(4), 295–306.
We would like to thank the patients who participated in the study and their families. We also thank the JCOG Executive Committee and Data Center for reviewing the Japanese translation draft and providing clinical insights, and Dr. Sandra Mitchell from the US National Cancer Institute for her extensive review and feedback.
YK, MT, HO, SI, EB and TY participated in designing concepts and development of the protocol. YK, DY, SI, YG, YT and NS participated in the enrollment and data collection. TM, YI, TK and TY performed in the data analysis. TM, TK and TY participated in drafting the manuscript. All authors read and approved the final manuscript.
Ethics approval and consent to participate
The study was performed in accordance with the Helsinki Declaration and Japanese ethical guidelines for epidemiology research. The study protocol was approved by the institutional review boards of the Kansai Medical University Hirakata Hospital and the University of Tokyo Hospital. All patients gave written informed consent before participating. In order to safeguard the participants’ privacy, the interviews were performed one-on-one in private rooms.
This research was supported by Grants-in-Aid for Scientific Research from the Japanese Ministry of Education, Culture, Sports, Science, and Technology (24590595). The authors declare that they have no competing interests. Informed consent was obtained from all individuals involved in the study.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Miyaji, T., Iioka, Y., Kuroda, Y. et al. Japanese translation and linguistic validation of the US National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). J Patient Rep Outcomes 1, 8 (2017). https://doi.org/10.1186/s41687-017-0012-7