During stage 0, a targeted literature review was performed to identify relevant concepts and caregiver burden instruments. Based on these data and interviews conducted with health professionals and caregivers of patients with MLD [14], a preliminary instrument was developed for further testing in three stages (stages 1–3) of review by caregivers, resulting in revision of the instrument (Fig. 1). Stage 1 of the Caregiver Impact Questionnaire (CIQ) development involved interviews with caregivers of patients with MLD; stages 2 and 3 involved interviews with caregivers of patients with MPS II and MPS IIIA, respectively. For stages 1 and 2, data saturation of the concepts was required to justify item retention or addition. This method ensures that sufficient data are collected to support the findings: when no new information emerges in successive interviews, saturation has been reached [15].
Stages 1–3 of CIQ development were performed by researchers representing several companies, who were experienced in measurement development, and by experienced qualitative interviewers in conjunction with a sponsor drug development team (Shire, Lexington, MA, USA; a Takeda company). Stage 0 was performed by Evidera (Bethesda, MD, USA) and the sponsor. Stages 1 and 2 were conducted by Evidera and stage 3 by RTI Health Solutions (Ann Arbor, MI, USA).
Stage 0: development of the preliminary draft MLD-CIQ
A preliminary conceptual framework (Fig. 2a) was generated based on a literature review of 53 publications related to MLD, and interviews were conducted with expert clinicians and caregivers of patients with MLD [14]. In addition, a targeted literature search for existing caregiver burden instruments was performed to help inform item development. The search was pragmatic in nature, and used the MEDLINE database (National Library of Medicine, Bethesda, MD, USA) and the Patient-Reported Outcome and Quality of Life Instruments Database (Mapi Research Trust, Lyon, France) to identify existing instruments that measure caregiver strain and burden.
The 23 instruments identified were relatively diverse, having been developed for use with caregivers across a range of different conditions, including psychiatric illnesses, chronic physical impairment and diabetes. Overall, they covered a broad range of concepts relating to emotional, physical, social and economic impacts of caring. However, many had been generated before the availability of the Food and Drug Administration instrument development guidelines [16], and very few were developed with direct input from caregivers of pediatric patients. In addition, none had been specifically developed for caregivers of children with the motor and cognitive dysfunction associated with LSDs. As these existing instruments would require substantial revision to adequately address caregiver impact in the context of LSDs, the decision was made to proceed with the development of a new instrument. Items for the draft MLD-CIQ were developed within the conceptual framework (Fig. 2a), based on concepts identified from the literature review and interviews with clinicians and caregivers [14] and items identified in the existing caregiver burden instruments.
An outline of the preliminary draft MLD-CIQ is provided in Fig. 1.
Stages 1–3: testing and refining the instrument
Participants
Potential participants were initially approached by the MLD Foundation, the International MPS Network (in the USA, UK, Spain and Mexico) or the National MPS Society (in the USA) via email or printed invitations, and were recruited consecutively based on eligibility. To be included in the MLD and MPS II interviews, caregivers had to be at least 18 years old. In addition, criteria were imposed to limit the caregiver population to those who were caring for patients with the most severe and earliest presenting forms of these diseases, in order to capture the full extent of potential impacts on caregivers. For MLD, only caregivers of patients with late-infantile MLD (defined as symptom onset between 6 months and 4 years of age inclusive) or juvenile MLD (symptom onset at 5–18 years of age) were eligible. For MPS II, only caregivers of patients aged 3–18 years with a diagnosis of neuronopathic MPS II were eligible. For MPS IIIA, only primary caregivers of patients aged 1–10 years were eligible and, although no caregiver age requirement was specified, they were required to be a parent or family caregiver (i.e. not paid). For all three diseases, the patient’s physician-confirmed diagnosis was self-reported by the caregiver. In addition, for MLD and MPS II, patient stakeholders were engaged in the instrument development process. For MLD, these stakeholders were two patient caregiver advocates, who were identified through the MLD Foundation and reviewed the CIQ. For MPS II, the patient stakeholder was a caregiver consultant in MPS II, identified through the MPS Society, who reviewed the study protocol and interview guides.
Interview process
All study materials were approved by appropriate ethics committees. All interviews in the UK, USA and Canada were conducted in English, and five interviews with caregivers of patients with MPS II were conducted in Spanish (Spain and Mexico). The English versions of the CIQ and study documents were translated into Spanish by professional patient-reported outcome translators at FACITtrans (http://www.facit.org/TransHome) according to established standards [17].
Study documents were mailed to participants before the telephone interview, except for one MPS IIIA interview, which was conducted in person. Participants were told not to open the documents until asked to by the interviewer, to ensure that the instructions and purpose of the study could be fully explained first, and to maintain spontaneity in responses to questions. Interviews were audio-recorded with the caregivers’ permission and verbatim transcripts were produced. Before each interview, the interviewer introduced themselves as a researcher for a company conducting the study on behalf of Shire (a Takeda company), and explained the study aims and procedures in detail. Consent was obtained either during interview scheduling or immediately before the interview, once study eligibility had been confirmed. For telephone interviews, the interviewer discussed the consent form with participants and asked them to confirm their consent verbally. For the face-to-face interview, written consent was obtained in person before the interview.
Except for one face-to-face MPS IIIA interview, all interviews were conducted by telephone; it was recommended that participants find a quiet place for the call away from distractions. The interviewers followed a semi-structured interview guide, which included concept elicitation [18, 19] for MLD and MPS II and cognitive debriefing [20, 21] of the CIQ completed for all three diseases (see Additional file 1 for a summary of key themes of the interview guides). The cognitive debriefing conducted with MPS IIIA caregivers was performed to identify whether the items were relevant for this condition and understood by caregivers, and participants were also given an opportunity to describe any relevant concepts that were not already covered.
Each interview lasted approximately 90 min for MLD, and the caregiver-completed CIQs were returned via post to the study center. For MPS II, concept elicitation lasted 60–90 min and cognitive debriefing lasted 90–120 min. For MPS IIIA, cognitive debriefing lasted approximately 60 min.
Concept elicitation
Concept elicitation interviews [18, 19] began with a conversation about the caregiver’s experiences of taking care of the child with MLD or MPS II to elicit a description of the impacts of caregiving on their life. Caregivers were asked to describe specifically how caring for someone with MLD or MPS II impacts their relationships, family, social life, leisure activities, personal time, daily activities, physical and mental health, work productivity, and finances. Caregivers were probed on their experiences with caregiving at the domain level (e.g. how does providing care for your child affect your other family relationships?). The concepts elicited from the interviews were compared with the concepts in the draft CIQ instrument.
A saturation grid was used to capture the caregiver burden concepts experienced by participants, to identify if further interviews were required.
Cognitive debriefing
During cognitive debriefing [20, 21], the caregivers’ understanding of the instrument and the appropriateness of recall periods and response options were tested. Caregivers were asked about: (1) the questionnaire’s instructions; (2) the content coverage of the questionnaire to ensure items fully captured relevant concepts; (3) the clarity of the items within the scale; (4) how the participants interpreted the items; (5) ease of completion of the scale; (6) comprehensiveness and relevance of the measure; (7) the appropriateness of the format, response scales, and recall period. Participants engaged in a standard ‘think-aloud’ cognitive interview [20, 22], reading the instructions and questions aloud and verbalizing their thought process as they answered the questions and described the meaning of the questions in relation to their caregiver experience.
For MLD, the cognitive debriefing interviews were performed in three rounds, with changes made to the questionnaire after each round and the interview guide modified accordingly. During the revision process, a translatability and reading-level assessment was conducted to make the questionnaire more readily translatable into other languages. One round of cognitive debriefing was performed for MPS II. Two rounds of cognitive debriefing were performed for MPS IIIA, with revision of the instrument between rounds. A sociodemographic questionnaire was also completed by all caregivers.
Analysis
In stages 1 and 2 of CIQ development with MLD and MPS II caregivers, data from concept elicitation and cognitive debriefing interviews were collected and the wording revised until comprehension of items was reached. The resulting questionnaire was tested in stage 3. For stages 1 and 2, content analysis of the concept elicitation and cognitive debriefing interview data was performed using MS Excel, including documentation of saturation. For stage 3, notes from the first round of cognitive interviews were analyzed, and revisions to the draft CIQ were made based on caregiver feedback. After the last round of cognitive interviews in MPS IIIA, analysis was conducted by detailed evaluation of transcripts and interview notes to inform the final set of CIQ items, and a revised conceptual framework was generated (Fig. 2b).