Translation and cross-cultural adaptation of the ICHOM standard set for stroke: the Dutch version
Journal of Patient-Reported Outcomes volume 7, Article number: 91 (2023)
The International Consortium for Health Outcomes Measurement (ICHOM) developed a standard set of patient-centered outcome measures for use in stroke patients. In addition to the Patient-Reported Outcomes Measurement Information System (PROMIS) Global Health, it is comprised of 25 questions that are not part of a specific questionnaire. This study aimed to translate these 25 single questions into Dutch.
Two native Dutch-speaking translators independently translated the original ICHOM questions into Dutch. A consensus translation was made by these translators and a third person. This translation was subsequently translated back to English independently by two native English-speaking translators. Afterwards a pre-final version was made by consensus of a committee. After field-testing among 30 stroke patients, a final version was made.
The forward and backward translations led to eight cross-cultural adaptations. Based on the interviews with stroke patients, 12 questions were changed to enhance comprehensibility leading to a final Dutch translation of the 25 single questions.
A Dutch translation of the 25 single questions of the ICHOM Standard Set for Stroke was developed. Now a complete ICHOM Standard Set for Stroke can be used in Dutch populations allowing comparison and improvement of stroke care.
Stroke is the second leading cause of death and of disability worldwide . From 1990 to 2017, the absolute number of people who suffered a new stroke has almost doubled . Stroke can lead to a multitude of impairments such as motor impairments , aphasia , cognitive problems , fatigue  and depression , resulting in a diminished Quality of Life (QoL) .
This wide range of sequelae is assessed with a large number of different disease-specific and general outcome measures in stroke health care and research. This lack of consensus of outcome measures limits benchmarking of the quality of stroke health care and interpretation of outcomes of stroke research .
Therefore, the International Consortium for Health Outcomes Measurement (ICHOM) developed a Standard Set for Stroke using a structured consensus-driven modified Delphi method with an international expert panel . The ICHOM Standard Set for Stroke consists of a combination of case mix variables, treatment variables and outcomes. These data are collected using administrative data, clinical data and Patient-Reported Outcome Measures (PROMs). The PROMs consist of 25 single questions and the Patient-Reported Outcomes Measurement Information System (PROMIS®) Global Health . Currently, the first studies reporting on the use of this standard set for stroke have been published [10,11,12]. In addition, a benchmarking platform was set up by ICHOM allowing facilities to add and compare the data from this Standard Set for Stroke (https://www.ichom.org/global-benchmarking-platform/).
To allow a valid interpretation and international comparison of the ICHOM Standard Set for Stroke, adequate translations of the PROMs of this set are needed. The PROMIS Global Health is already translated in several languages including Dutch. This Dutch translation was shown to have sufficient psychometric properties in the Dutch population . The aim of our study was to also translate the 25 single questions of this Standard Set for Stroke in Dutch so that a complete Standard Set for Stroke can be used in Dutch populations.
Materials and methods
This study is a translation and cross-cultural adaptation study including pilot testing among patients with stroke. The translation and pilot testing process were based on previously developed guidelines for cross-cultural translation of PROMs to ensure a comprehensible translation equivalent to the original questions [14,15,16]. For the pilot testing, patients who participated in the Stroke Cohort Outcomes of Rehabilitation (SCORE) 2.1 study (Netherlands Trial Register no. NL9509) were recruited. The study protocol was reviewed by the Medical Ethical Committee of the Leiden University Medical Center and was found not to fall within the scope of the Dutch Medical Research with Human Subjects Law (N21.070). All patients signed informed consent and an additional informed consent for the interview. The study was conducted in compliance with the Declaration of Helsinki . ICHOM was notified about this translation study and had no objection to the study.
The 25 questions of the ICHOM standard set for stroke
The 25 questions of the ICHOM standard set for stroke consist of 8 questions about ‘demographic factors’, 10 about ‘vascular and systemic’ history, 2 about ‘survival and disease control’ and 5 about ‘patient-reported health status’. ICHOM has assigned a variable ID to each question to facilitate benchmarking. The question concerning ethnicity should be formulated by each country and is not predefined as a single question. All other questions are multiple-choice questions.
Translation of the questions of the ICHOM standard set for stroke
First, the English questions were independently translated into Dutch by two native Dutch-speaking translators (DO, WvMP), who are fluent in English and experienced in the field of stroke research.
Second, a meeting was organized with these translators and a third native Dutch-speaking medical professional (GR). This medical professional was also involved in the development of the original questions. In this meeting the two forward translations were reconciled. Discrepancies were resolved or the better of the forward translations was chosen. In case of doubt, the ICHOM was contacted.
Thirdly, after this consensus Dutch translation was developed, the translation was translated back into English by two native English-speaking translators (FvM, EP), who were also fluent in the Dutch language. One of the native English-speaking translators was experienced in the field of PROMs and the other native English-speaking translator was not involved in research or PROMs. Both were blinded to the original English questions.
Fourth, a committee (comprised of the translation team, and two additional medical professionals) compared the two back-translations with the original 25 questions and provided a clarification for the differences. An alternate Dutch translation was made when the previous translation was not acceptable based on the back-translations. This resulted in the pre-final version.
Pilot testing of the translated questions of the ICHOM standard set for stroke
The pre-final version that resulted from above described translation process was tested for comprehensibility in a sample of 30 stroke patients . All patients were aged 18 or older and admitted to a rehabilitation center for inpatient stroke rehabilitation. Both patients with a first-ever stroke as with a recurrent stroke were included. Patients were excluded when they were unable to complete questionnaires in Dutch. Patients with aphasia or cognitive problems were only excluded if they were not able to answer the questions and participate in the interview. These patients completed the pre-final version of the 25 Dutch questions. Afterwards, they were interviewed by a research nurse to ensure that the substance of information of the translated question remained the same as the original English question and that the questions were comprehensible. After the interviews, the patient’ comments were analyzed and discussed, and when needed adjustments were made by the committee, resulting in the final translation.
The original questions and their variable IDs are shown in Table 1. After the two forward translations were completed, a consensus meeting with a Dutch-speaking medical professional was organized, resulting in the identification of eight cross-cultural issues. In the questions concerning living location (i.e. variable IDs LIVINGLOCPRE and LIVINGLOCPOST) specific health care facilities are mentioned that are not available in the Netherlands. Thus, we incorporated equivalent facilities in the Dutch health care system in these questions. As for LIVINGLOCPRE there was additional e-mail contact with the ICHOM, because ‘living’ in Dutch suggests a permanent stay and one of the original response options (i.e. option 4) is not a permanent living location in the Netherlands. Based on the advice from ICHOM, consensus was reached on adding another question (LIVINGLOCREHAB) when option 4 was chosen to identify the patients permanent living location pre-stroke. In addition, there was discussion about what is included within community care in option 1 and 2 of LIVINGLOCPRE and LIVINGLOCPOST. There is no direct Dutch translation for this term. ICHOM clarified this as being professional care, and the translation was adjusted accordingly.
For the translation of the questions concerning vascular and systemic medical history, the phrase or phrases for each medical condition were used that were thought to be most commonly used in the general Dutch population and therefore most comprehensible for Dutch stroke patients. For example, mini-stroke is not commonly used and was left out of the translation of question PRIORTIA, while we added the word ‘suikerziekte’ (‘sugar disease’) to diabetes in question DIAB. In addition, we chose to consistently use ‘a’ doctor, while in the original questions ‘your’ doctor and ‘a’ doctor are used inconsistently. In addition, a more free translation of the response option ‘999 = Unknown’, for the questions about medical history was chosen. This was done to give patients the option that they do not know the answer as an alternative for leaving the question unanswered.
After backward translation, a meeting with the committee took place. This led to the identification of an error in the two questions concerning ambulation (PRESTROKEAMB and POSTSTROKEAMB), which was corrected accordingly. A pre-final version was made. In Supplementary Tables 1, the translations resulting from each above mentioned step are described.
The pre-final version of the ICHOM questions of the Standard Set for Stroke was tested in 30 stroke patients for comprehensibility. The mean age of these patients was 60 years (range 38–83 years), 30% of them were females, 90% had an ischemic stroke. The interviews with these patients after completing the questions lead to changes in the wording of 12 questions.
The question LIVEALONEPOST should be completed by all patients, but most of our patients that were admitted to the rehabilitation center found it difficult to answer as they did not reside at home when filling in the questionnaire. To overcome this difficulty, option 999 was changed from ‘Unknown’ into ‘I am not living at home at the moment or unknown’. The questions concerning prestroke and poststroke functional status would be easier to answer when response options of these six questions would start with ‘yes’ and ‘no’ according to 6–8 patients depending on the question. This was done accordingly.
The Dutch equivalents for the terms coronary artery disease and atrial fibrillation were unknown to 40% and 30% of patients, respectively. We therefore added an explanation of these terms to the questions concerning these diseases (i.e. CAD and AFIB). Some patients reported that they had high cholesterol but that this was mentioned by a nurse or an assistant of their doctor and not by their doctor. Because this is common practice in the Netherlands, the question HYPERLIP was changed accordingly. The questions PRIORSTROKE and STROKERECUR were not always clear for patients and were clarified. During the discussion of the interviews of the patients, a better suited translation of the word ‘told’ in the questions about medical conditions was found and applied accordingly. The original questions and the final Dutch version of the 25 single questions in the ICHOM Standard Set for Stroke is shown in Table 1.
This cross-cultural translation study of 25 questions of the ICHOM Standard Set for Stroke into Dutch resulted in a comprehensible translation equivalent to the original questions. Equivalence is necessary to allow international comparison between national stroke populations and to allow pooling of data collected from stroke patients in different countries . Thereby, this translation contributes to the aims of the ICHOM to effectively improve stroke health care and usability of research.
The value of an international standard set has been shown previously with the Utstein template for patients with cardiac arrest . This template has been widely implemented and used for international comparison of cardiopulmonary resuscitation outcomes [20, 21]. Similar to the ICHOM Standard Set of Stroke, the Utstein template includes amongst others case mix variables and outcomes. However, unlike the ICHOM Standard Set for Stroke, it does not include PROMs.
As the authors of the ICHOM Standard Set for Stroke also state themselves, the utility of the Standard Set in clinical practice and research is still undetermined, because it is based on expert consensus rather than on high levels of evidence. The original single questions were not tested for comprehensibility in stroke patients nor is there a specific lay-out. In addition, to our knowledge no other translation of the complete Standard Set for Stroke exists, hampering harmonization and international comparison. A recent study demonstrated that both patients and medical staff found the Standard Set for Stroke relevant, but patients reported limited understanding of why the assessment was introduced and the medical staff found limited feasibility and sustainability of the set . Their acceptance of the implementation were low . Moreover, de Graaf et al.  found that factors not related to the stroke (such as coping style) influence the scores on PROMIS Global Health. This emphasized the need for further research.
Nevertheless, there is increasing evidence that the measurement properties of the PROMIS Global Health are favorable in hospital-based stroke populations and in general populations [23, 24]. In addition, in the Dutch population, the measurement properties of the Dutch-Flemish PROMIS Global Health were also favorable  and reference values are available .
Our study focused on comprehensibility, however further research is necessary to assess the other psychometric properties of the translated ICHOM Standard Set for Stroke. Although Flemish and Dutch are very much alike, there are also differences and it is unclear whether the translated questions are also the semantic equivalence in Flemish. This should be investigated further by native-speaking Flemish translators and in a Flemish population in Belgium.
In addition, for adequate international comparison Differential Item Functioning analysis by language is of value to determine whether Dutch- and English-speaking patients answer the ICHOM items differently. We added explanation of some of the medical terms, while this also might be necessary for the comprehensibility of the original questions. Next to the comprehensibility of the questions concerning vascular and systemic medical history, it is unclear how adequate patients answer these questions. Previous research  demonstrated that patients do not recall all medical information they have received, suggesting that these questions might be difficult to answer correctly. To overcome this problem, the developers of the original questions have noted that if the patient is unable to answer, this information can be extracted from the medical records. To determine whether and to what extent this is necessary, more experience with the completion of these questions in daily practice is needed.
As for other psychometric properties, test-retest reliability could be assessed. However, the questions are very concrete and therefore no problems regarding reliability are expected. In addition, there might still be bias left through cultural differences, although we have tried to avoid this.
In summary, currently no international standard set for measuring case mix variables and outcomes of stroke patients is used, hampering effective improvement of stroke health care and usability of research. In 2016, the ICHOM has proposed a Standard Set for Stroke for this purpose. The availability of suitable translations of both the single questions and the PROMIS Global Health of this Standard Set for Stroke in different languages will contribute to the international implementation and comparison of stroke outcomes. In this study we have provided a comprehensible translation of the single questions to complete the Dutch PROMs of the ICHOM Standard Set for Stroke.
Availability of data and materials
All data relevant to the study are included in the article or uploaded as supplementary information.
Krishnamurthi RV, Ikeda T, Feigin VL, Global (2020) Regional and Country-Specific Burden of Ischaemic Stroke, Intracerebral Haemorrhage and Subarachnoid Haemorrhage: a systematic analysis of the global burden of Disease Study 2017. Neuroepidemiology 54(2):171–179. https://doi.org/10.1159/000506396
Dimyan MA, Cohen LG (2011) Neuroplasticity in the context of motor rehabilitation after stroke. Nat Rev Neurol 7(2):76–85. https://doi.org/10.1038/nrneurol.2010.200
Berthier ML (2005) Poststroke aphasia: epidemiology, pathophysiology and treatment. Drugs Aging 22(2):163–182. https://doi.org/10.2165/00002512-200522020-00006
Makin SD, Turpin S, Dennis MS et al (2013) Cognitive impairment after lacunar stroke: systematic review and meta-analysis of incidence, prevalence and comparison with other stroke subtypes. J Neurol Neurosurg Psychiatry 84(8):893–900. https://doi.org/10.1136/jnnp-2012-303645
Loubinoux I, Kronenberg G, Endres M et al (2012) Post-stroke depression: mechanisms, translation and therapy. J Cell Mol Med 16(9):1961–1969. https://doi.org/10.1111/j.1582-4934.2012.01555.x
Cumming TB, Packer M, Kramer SF et al (2016) The prevalence of fatigue after stroke: a systematic review and meta-analysis. Int J Stroke 11(9):968–977. https://doi.org/10.1177/1747493016669861
Opara JA, Jaracz K (2010 Jul-Sep) Quality of life of post-stroke patients and their caregivers. J Med Life 3(3):216–220
Kelly-Hayes M (2004) Sep-Oct;19(5):301-7 Stroke outcome measures. J Cardiovasc Nurs. https://doi.org/10.1097/00005082-200409000-00005
Salinas J, Sprinkhuizen SM, Ackerson T et al (2016) An International Standard Set of patient-centered outcome measures after stroke. Stroke 47(1):180–186. https://doi.org/10.1161/STROKEAHA.115.010898
Rimmele DL, Lebherz L, Frese M et al (2019) Outcome evaluation by patient reported outcome measures in stroke clinical practice (EPOS) protocol for a prospective observation and implementation study. Neurol Res Pract 1:28. https://doi.org/10.1186/s42466-019-0034-0
Rimmele DL, Schrage T, Lebherz L et al (2021) Profiles of patients’ self-reported health after acute stroke. Neurol Res Pract 3(1):43. https://doi.org/10.1186/s42466-021-00146-9
Lebherz L, Fraune E, Thomalla G et al (2022) Implementability of collecting patient-reported outcome data in stroke unit care - a qualitative study. BMC Health Serv Res 22(1):346. https://doi.org/10.1186/s12913-022-07722-y
Pellicciari L, Chiarotto A, Giusti E et al (2021) Psychometric properties of the patient-reported outcomes measurement information system scale v1.2: global health (PROMIS-GH) in a dutch general population. Health Qual Life Outcomes 19(1):226. https://doi.org/10.1186/s12955-021-01855-0
Beaton DE, Bombardier C, Guillemin F et al (2000) Guidelines for the process of cross-cultural adaptation of self-report measures. Spine (Phila Pa 1976) 25(24):3186–3191
Tsang S, Royse CF, Terkawi AS (2017) Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth 11(Suppl 1):S80–S89. https://doi.org/10.4103/sja.SJA_203_17
Eremenco SL, Cella D, Arnold BJ (2005) A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof 28(2):212–232. https://doi.org/10.1177/0163278705275342
World Medical Association Declaration (2013) Of Helsinki: ethical principles for medical research involving human subjects. JAMA 310:2191–2194
Perneger TV, Courvoisier DS, Hudelson PM et al (2015) Sample size for pre-tests of questionnaires. Qual Life Res 24(1):147–151. https://doi.org/10.1007/s11136-014-0752-2
Perkins GD, Jacobs IG, Nadkarni VM et al (2015) Cardiac arrest and cardiopulmonary resuscitation outcome reports: update of the Utstein Resuscitation Registry Templates for Out-of-hospital cardiac arrest: a statement for healthcare professionals from a task force of the International Liaison Committee on Resuscitation (American Heart Association, European Resuscitation Council, Australian and New Zealand Council on Resuscitation, Heart and Stroke Foundation of Canada, InterAmerican Heart Foundation, Resuscitation Council of Southern Africa, Resuscitation Council of Asia); and the American Heart Association Emergency Cardiovascular Care Committee and the Council on Cardiopulmonary, critical care, Perioperative and Resuscitation. Circulation 132:1286–1300 Erratum in: Circulation. 2015;132(13):e168-9
Fredriksson M, Herlitz J, Nichol G (2003) Variation in outcome in studies of out-of-hospital cardiac arrest: a review of studies conforming to the Utstein guidelines. Am J Emerg Med 21(4):276–281. https://doi.org/10.1016/s0735-6757(03)00082-2
Dyson K, Brown SP, May S et al (2019) International variation in survival after out-of-hospital cardiac arrest: a validation study of the Utstein template. Resuscitation 138:168–181. https://doi.org/10.1016/j.resuscitation.2019.03.018
De Graaf JA, Post MWM, Visser-Meily JMA (2022) Ervaringen met de PROMIS-10 na CVA: promising in de spreekkamer? Ned. Tijdschrift voor Revalidatiegeneeskunde 3:40–42
Oosterveer DM, Arwert H, Terwee CB et al (2022) Measurement properties and interpretability of the PROMIS item banks in stroke patients: a systematic review. Qual Life Res 31(12):3305–3315. https://doi.org/10.1007/s11136-022-03149-4
Hays RD, Bjorner JB, Revicki DA et al (2009) Development of physical and mental health summary scores from the patient-reported outcomes measurement information system (PROMIS) global items. Qual Life Res 18(7):873–880. https://doi.org/10.1007/s11136-009-9496-9
Elsman EBM, Roorda LD, Crins MHP et al (2021) Dutch reference values for the patient-reported outcomes Measurement Information System Scale v1.2 - Global Health (PROMIS-GH). J Patient Rep Outcomes 5(1):38. https://doi.org/10.1186/s41687-021-00314-0
Kessels RP (2003) Patients’ memory for medical information. J R Soc Med 96(5):219–222. https://doi.org/10.1177/014107680309600504
We are grateful to Elizabeth Parker for her contribution as native English speaker in the translation process; to Asa Mennema and Manja Westerbeek, our research nurses for completing the interviews with the patients; and to Sara Sprinkhuizen, our contact of ICHOM, for her contributions to this translation and cross-cultural adaptation.
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
Ethics approval and consent to participate
The study protocol was reviewed by the Medical Ethical Committee of the Leiden University Medical Center and was found not to fall within the scope of the Dutch Medical Research with Human Subjects Law (N21.070). All patients signed informed consent and an additional informed consent for the interview.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Translation of the single questions of the ICHOM Standard Set for Stroke.
About this article
Cite this article
Oosterveer, D.M., van Meijeren-Pont, W., van Markus-Doornbosch, F. et al. Translation and cross-cultural adaptation of the ICHOM standard set for stroke: the Dutch version. J Patient Rep Outcomes 7, 91 (2023). https://doi.org/10.1186/s41687-023-00630-7
- Standard Set for Stroke