The Cystic Fibrosis Impact Questionnaire: qualitative development and cognitive evaluation of a new patient-reported outcome instrument to assess the life impacts of cystic fibrosis

Abstract

Background

Patients with cystic fibrosis (CF) experience significant disease burden, including progressive pulmonary decline and reduced survival. This multicenter qualitative study was conducted to develop a new patient-reported outcome (PRO) measure to assess the impact of CF on patients’ quality of life: the Cystic Fibrosis Impact Questionnaire (CF-IQ).

Semi-structured qualitative concept elicitation (CE) interviews with patients and caregivers documented CF-related symptoms, impacts, and treatment experiences. Coded interview data were considered alongside existing PROs, published literature, and expert opinion to develop an initial scale. Three rounds of cognitive interviews evaluated respondent comprehension and facilitated refinement of the CF-IQ.

Results

Adult (N = 20) and pediatric (N = 22) patients with CF and their parents/caregivers (N = 22) completed CE interviews at 7 US clinics. The sample included patients aged 6–58 years, 57% females, and represented a broad range of disease severity (forced expiratory volume in 1 s range: 22%–127% predicted). Interviews identified 59 unique CF-related impact concepts in domains, including activity limitations (physical, social, leisure), functional limitations (school, work), vulnerability/lack of control, emotional impact, treatment burden, and future outlook. Concept saturation was achieved, and a draft questionnaire was developed. Findings from the cognitive interviews (n = 18) confirmed that instructions, items, and response scales were relevant and clear, and interpreted as intended by patients.

Conclusion

The CF-IQ is a 40-item novel PRO scale assessing a comprehensive set of patient-relevant concepts to characterize the multifaceted nature of CF. Qualitative interview data support the content validity of the CF-IQ, which is currently undergoing additional psychometric evaluation in patients with CF.

Background

Cystic fibrosis (CF) is a rare, autosomal recessive disease that affects an estimated 30,000 individuals in the US and more than 70,000 people worldwide; approximately 1000 new cases are diagnosed in the US each year [1]. CF is associated with serious, chronically debilitating morbidities and premature mortality. In 2017, the median age at death for patients with CF was 30.7 years, while the predicted median age of survival for patients born in 2017 was 46.2 years, reflecting increased expected longevity associated with recent advancements in CF care [2]. CF affects multiple organs (lungs, pancreas, sweat glands, and intestinal, biliary, and reproductive tracts) [3], and pulmonary disease is the primary cause of morbidity and mortality [4]. Patients with CF typically experience progressive loss of lung function, often resulting in respiratory failure and death [3, 5].

Patients with CF experience substantial decrements in health-related quality of life (HRQoL) arising from symptoms, functional limitations, high levels of treatment burden, and other complicating aspects of the condition [6], such as progressively declining lung function and repeated pulmonary exacerbations [7]. The treatment burden of CF is significant, with most patients spending more than 2 h per day on routine maintenance therapies, which involve taking multiple oral and inhaled medications, and chest physical therapy [8, 9].

Beyond the direct impact of symptoms and treatment burden on HRQoL, children and adolescents report feelings of vulnerability, loss of independence and opportunities, stigmatization, concern about getting sick, social isolation, lack of control, and resentment of long-term treatment. Children also report fear of injections and fatigue from prescribed physical exercise and describe their CF treatment as intensive and invasive [10]. Longitudinal studies in adults with CF have shown steady decreases in social functioning over time [6, 7]. Patients report interferences with activities of daily living, such as work and school, and treatment burden can further increase with added therapies, clinic visits, and hospitalizations resulting from acute infections and exacerbations [11].

Due to the substantial influence of CF on the quality of life of patients, rigorous assessment of these impacts is critical in understanding the potential benefits of new and emerging CF therapies. Because CF is a multisystemic disease, prior to the introduction of CF transmembrane conductance regulator (CFTR) modulators, therapy included symptomatic treatment and management of pulmonary exacerbations and other complications by controlling airway infections and inflammation, mobilizing secretions to reduce airway obstruction, and correcting gastrointestinal symptoms and nutritional deficits caused by pancreatic insufficiency [12,13,14,15]. The introduction of CFTR modulators allows treatment to advance from symptom reduction to targeting the underlying cause of the disease. With changes in the treatment landscape, there is an emerging place for patient-reported outcome (PRO) instruments for CF that capture the broad experience of the patients, which will be useful in understanding the totality of the benefits of the newer therapies.

The few CF-specific PRO instruments currently available focus largely on symptom assessment and were developed before the introduction of CFTR modulators as a therapeutic option, and prior to current PRO development guidelines emphasizing the importance of direct patient input to determine the concepts that are most relevant and important to the target patient population [16,17,18]. Three previously developed disease-specific PRO instruments for assessment of CF concepts are the Cystic Fibrosis Quality of Life (CFQoL) assessment [19], the Cystic Fibrosis Questionnaire-Revised (CFQ-R) [20,21,22], and the Questions on Life Satisfaction for Adolescents and Adults with Cystic Fibrosis (FLZ^M-CF) [23].

The CFQoL is an instrument that consists of 52 items across 9 domains: physical functioning, social functioning, treatment issues, chest symptoms, emotional responses, concerns for future, interpersonal relationships, body image, and career concerns [19]. The FLZ^M-CF is an 18-item CF-specific add-on module to the parent life satisfaction measure that includes items assessing satisfaction with symptoms (i.e. breathing difficulty/coughing, abdominal pain, digestive trouble, sleep difficulty) and other elements of overall disease burden (i.e. integration of therapy into daily routine, being needed by others, having your situation understood by others) [23]. Although the development of both instruments did include limited qualitative patient input, it is unclear from the available documentation whether the CFQoL or FLZ^M-CF were constructed consistently with current scientific standards for evidence of content validity in PRO development [16, 19].

The CFQ-R consists of 44 items over 9 HRQoL domains (physical, role/school, vitality, emotion, social, body image, eating, treatment burden, health perceptions) and 3 symptom scales for weight, respiratory function, and digestion [21]. The CFQ-R has demonstrated the ability to detect important differences in patient QoL in both the clinical trial and real-world settings [24,25,26] and is likely to remain the gold standard for clinical drug development. However, like the CFQoL and FLZ^M-CF, the CFQ-R was developed prior to the introduction of CFTR modulator-based therapies, and while this instument provides important information on quality of life, it may not be fully optimized to capture a patient’s full experience of life with CF and the impact of new therapies.

To ensure incorporation of the most relevant concepts that extend beyond the measurement of CF symptoms, the present study utilized qualitative concept elicitation and cognitive interview methods to assess the HRQoL impacts on patients with CF and to develop the Cystic Fibrosis Impact Questionnaire (CF-IQ). Our goal in developing the CF-IQ is to complement and build on the available PRO instruments by assessing a comprehensive set of patient-relevant and important concepts to characterize the multifaceted nature of disease burden in CF as experienced by patients within the modern therapeutic environment, including those receiving therapy with CFTR modulators.

Methods

Study design

PRO instrument development employed qualitative data collection and analysis techniques consistent with scientific best practices [16,17,18].

A review of the literature, including evaluation of existing PRO instruments used in CF clinical studies, was conducted to inform instrument development. The review identified 45 different PRO instruments used in published CF studies during the 15-year review period. Only four of the identified instruments were developed as CF-specific assessments, with three of those (the CFQoL, FLZ^M-CF, and CFQ-R, as detailed above) including coverage of concepts related to the impact of CF on daily life and treatment-related experiences, as opposed to assessment limited to CF symptoms.

The review was followed by qualitative interviews with clinicians and nurses providing care for patients with CF. The study team used input from CF specialists and instrument development experts, as well as the findings from the literature review and clinician interviews, to inform protocol design and to develop semi-structured interview guides for use in qualitative data collection with patients with CF and their caregivers.

Following development of interview guides, in-person qualitative concept elicitation interviews were conducted with pediatric, adolescent, and adult patients with CF to identify symptoms, symptom-related impacts, and treatment experiences relevant and important to patients. Primary caregivers of participating children aged 6–17 years were also interviewed. All interviews were audio-recorded and transcribed for concept coding. Figure 1 provides an overview of the key tasks undertaken in PRO development.

Fig. 1

Process flow chart for development of the Cystic Fibrosis Impact Questionnaire. CE concept elicitation

Study population

Eight CF treatment centers across the US recruited patients from their ongoing caseloads to participate in the qualitative interviews. Seven sites enrolled patients for the initial concept elicitation interviews and 4 sites (3 remaining from the concept elicitation phase plus 1 additional clinic) enrolled patients during the cognitive interview phase. A central institutional review board (IRB), Quorum Review IRB (Seattle, WA, USA), approved the study protocol and materials, as did local IRBs where required. All patients (or parents/legal guardians of patients aged < 18 years) provided written informed consent prior to initiation of any study-related procedures. No changes to the participants’ existing care were made, and no clinical interventions were initiated as part of the study protocol.

A purposive, non-probability sample of approximately 40–50 patients with CF was planned for the concept elicitation phase. Eligible patients had a physician-confirmed diagnosis of CF, defined as a sweat chloride value ≥60 mmol/L by quantitative pilocarpine iontophoresis, or 2 documented CF-causing mutations, and either chronic sinopulmonary disease or gastro-intestinal/nutritional abnormalities. Recruitment quotas within 5 age groups (6–8, 9–11, 12–17, 18–24, and ≥ 25 years) were used to capture age-related differences over the course of the disease, as well as in type and intensity of life activities. To reflect the range of comorbidities seen in patients with CF, additional recruitment targets were used to guide the inclusion of patients in 2 key clinical subgroups: 1) patients with comorbid CF-related diabetes were limited to ≤33% of the overall sample; and 2) patients with more severe CF presentations (defined as a history of ≥2 hospitalizations in the previous year for patients aged ≥18 years and ≥ 1 hospitalization in the prior year for patients aged < 18 years) were targeted to represent approximately 33% of the overall sample. Recruitment targets were set for approximately 50% of the sample to be composed of patients receiving CFTR modulator therapy at the time of interview.

During the enrollment process, patients completed a questionnaire to provide demographic characteristics (age, sex, ethnicity, race, educational attainment, employment status) and ratings of their overall health compared with others and the overall severity of their CF symptoms.

Qualitative concept elicitation interviews

Individual concept elicitation interviews were conducted to gather information about key symptoms and disease-related life impacts experienced by patients with CF, as well as functional changes in daily living following treatment. All interviews lasted approximately 60–90 min and used a semi-structured interview guide, starting with general, open-ended questions and a day reconstruction exercise, and continuing with a structured probing system to gather more detailed information about participants’ symptoms, disease impacts, and treatment-related experiences. The probe system was informed by clinician interviews and a systematic review of existing CF literature. Separate, age-appropriate interview guides were used with children (aged 6–11 years), adolescents/adult patients (aged ≥12 years), and the caregivers of participating patients aged < 18 years. Interviewers followed each institution’s infection control protocols (i.e. mask, gloves, gown procedures) during the patient interviews. Young children were interviewed with a parent or caregiver present, if desired, but they provided their own responses to interview questions. In most cases, the caregiver interviews were conducted separately, without the child present in the room at the time the caregiver provided his/her responses.

Analyses

All interviews were audio-recorded and transcribed. Transcript coding was conducted using ATLAS.ti software (ATLAS.ti Scientific Software Development GmbH, Berlin, Germany) [27], and coded expressions were grouped by similarity of content for analysis. Coded transcript data were evaluated for evidence of saturation of concept (defined as the point at which no new concept-level information was forthcoming from the study sample) to confirm the adequacy of the sample size and robustness of the set of concepts elicited. To detect the point of concept saturation, a summary saturation table was used to track the occurrence of novel impact concepts expressed within chronologically ordered sets of interviews. Consistency and reproducibility of concept coding were assessed via evaluation of inter-rater agreement (IRA).

Item generation

The development team met with CF clinical experts and clinical outcome assessment measurement experts to review results of the concept elicitation interviews and identify the concepts most appropriate to include in a CF-impact PRO. The hypothesized relationships between the selected concepts and domains of the draft instrument were modeled in a preliminary conceptual framework. For each selected concept, transcript data were used to develop draft PRO item text (i.e. question and scale) reflecting the choice of words and descriptions used by patients to articulate the concept.

Cognitive interviews

Following the development of the preliminary draft instrument, cognitive interviews were conducted to evaluate concept relevance and understandability, the structure of the draft items and response scales, and the appropriateness of the recall period. Interviews were conducted in 3 iterative waves; the development team reviewed the findings from each wave and implemented refinements to the draft instrument before each subsequent set of interviews. The preliminary conceptual framework for the draft instrument was adjusted based on refinements made during the cognitive interviews.

Cognitive interview participants were selected through the same recruitment process used for qualitative interviews. Cognitive interviews used a semi-structured interview guide, which included broad questions to assess the comprehensiveness and relevance of the concept coverage of the CF-IQ, as well as a “think-aloud” process to ensure the instructions, items, and response options were interpreted as intended for each item of the CF-IQ [18, 28]. When possible, questions were reframed in the terminology used by respondents to aid comprehension.

Results

Characteristics of study sample

A total of 42 qualitative patient interviews were conducted with English-speaking adult (N = 20) and child (N = 22) participants, as well as 22 adult caregivers (Table 1). The population was well distributed across the 5 quota-targeted age groups and differing levels of education. Twenty-four (57%) patients were female. Twelve of the 20 adult patients were employed either full-time or part-time, 3 were students, and 5 were not working outside of the home at the time of the interview. Seventeen (41%) of the concept elicitation interview participants (9 adult and 8 pediatric patients) were receiving treatment with a CFTR modulator, and 13 participants (7 children/adolescents and 6 adults; 30.9% of the overall sample of 42) had experienced ≥2 hospitalizations in the previous year.

Table 1 Participant demographic and clinical characteristics

Patients were mostly evenly distributed between fair, good, and very good overall health, with very few extreme-level ratings (poor or excellent) (Table 1). All 30 patients aged ≥12 years had symptoms associated with CF. Although there were some ratings of severe and very severe (5 of the 30 patients), most patients rated symptom severity as mild or moderate.

Qualitative results

Analysis of the concept elicitation interview transcripts resulted in more than 4000 coded expressions about CF symptom and impact experiences. The objective of the PRO development process was to construct an assessment of the impacts of CF (rather than its symptoms). Although the coded symptom concepts were used to illuminate the context of the patients’ downstream impact experiences, the subsequent steps focused on evaluating and selecting impact concepts for PRO measurement.

To support instrument development, coded expressions of CF impact were organized into 59 impact concepts for potential inclusion in the PRO scale. Four dual-coded transcripts were assessed for IRA, with 90.2%–95.7% agreement observed between 2 coders on concept codes assigned to text segments. With approximately 8227 words of narrative text per transcript and 4005 assignable codes, the results suggest high IRA.

Saturation of impact concepts was achieved within the first 35 of the 42 coded transcripts. No new concepts emerged in subsequent transcripts, suggesting that additional interviews were unlikely to result in additional concepts being identified, and that the sample of 42 interviews was adequate to achieve completeness of concepts from this study population (Table 2).

Table 2 Concept saturation: identification of novel CF impact concepts by transcript group

Item generation

Following the concept elicitation phase, the development team reviewed key findings from the qualitative data, literature review, and clinical expert input to identify relevant CF impact concepts targeted for inclusion in a PRO instrument. To determine the most patient-relevant and clinically meaningful concepts for selection, the following parameters were considered: frequency of expression, total number of expressions, spontaneous nature of expressed concepts, patient ratings of difficulty coping with each impact, and clinical expertise.

This process reduced 59 coded concepts to 42 targeted concepts for inclusion in the draft CF-IQ instrument. Concepts that exhibited low prevalence among interview participants, or those that were overly general or redundant were removed.

Cognitive interviews

A total of 18 patients participated in 3 waves of cognitive interviews (Table 1), during which patient feedback led to revisions to the wording and/or response options of several items, and the removal of 2 items. Cognitive interview data provided evidence that the revised instrument was comprehensive, relevant to the CF experience, readily understandable, and easy to complete. Patient input confirmed the appropriateness of the item structure, response scale, and recall period.

The Cystic Fibrosis Impact Questionnaire

The newly created CF-IQ is a 40-item PRO questionnaire that measures each impact concept (Fig. 2) using a 5- or 7-point verbal rating scale. Thirty-two items focus on severity or intensity, with a rating scale from “no [impact]” to “extreme [impact]”. The remaining 8 items focus on frequency or amount of time an impact was experienced, and employ a rating scale from “never” to “always”. Thirty-six CF-IQ items utilize a 7-day retrospective recall period to assess the impact of CF, whereas the remaining items assess a respondent’s “current” status. In cognitive interview testing, patients completed the instrument in an average of 5.7 min (median of 5.0 min).

Fig. 2

Conceptual framework for the Cystic Fibrosis Impact Questionnaire. CF cystic fibrosis

Consistent with current scientific best practices [17, 18], the selected recall period was based on qualitative concept elicitation data and confirmed as appropriate by patients during cognitive interviews. The current instrument is suggested for use in patients with CF aged ≥12 years. Although all patients, including those aged > 6 years, provided input on concepts, the readability (reading level) and terminology of CF-IQ has not yet been evaluated and adapted for this age group; thus, it is not advised for use with children aged 6–11 years. Following the qualitative instrument development steps described herein, efforts are underway to conduct a quantitative evaluation of the instrument’s psychometric and measurement properties.

Discussion

Evaluating the merits of new drug therapies and other interventions for CF requires comprehensive and rigorous measurement of patients’ experiences alongside clinical endpoints. Existing HRQoL assessments in CF include observations of signs and symptoms by clinicians, qualitative symptom scales devised by investigators, and CF symptom-reporting instruments. Available tools for CF are largely symptom-focused.

Recent therapeutic advancements in CF treatment have created a specific need for a sensitive PRO tool to quantify and assess broader changes in CF burden for use in natural history studies, observational studies, and in clinical practice. In response, a PRO instrument for assessment of CF-related life impacts, the CF-IQ was developed. This is the first PRO instrument designed to assess the day-to-day impact of CF, including patients receiving CFTR-modulator therapies. Developed in accordance with scientific best practice guidelines [16,17,18], the CF-IQ may both complement existing instruments and offer additional insights on the impact of CF among patients treated with CFTR modulators.

There are several key strengths of the development process for the CF-IQ. First, CF-IQ was developed through rigorous, qualitative methods leveraging direct input from patients and caregivers to support initial concept selection. With its development rooted in the collection of qualitative concept elicitation data to the point of concept saturation, the subsequent item generation process resulted in content that reflects patient-derived language to frame questions and provide response options that are understandable and meaningful to patients. Second, elements of the CF-IQ’s distinct measurement approach, such as use of a shorter (7-day) recall period and greater use of severity-focused items, arose directly from patient interview findings and may provide added sensitivity to the instrument’s measurement profile when contrasted with the longer (two-week) recall period and use of frequency-focused item construction of the other PRO instuments.

In addition, the sample of patients involved in the CF-IQ development process reflects experiences with currently available treatment modalities, including the availability of newer CFTR modulator-based therapies. Thus, the instrument includes item content tailored to reflect the experiences of current patients consistent with published literature, as well as expanded coverage of issues experienced by patients managing CF with expanded therapeutic options. The expectation of extended survival for patients with CF using new and emerging therapies has informed the development of the CF-IQ. Greater emphasis on the future and on patients’ ability to exert control over the direction of their lives were key themes that emerged in qualitative interviews. Elements focused on future outlook are included in multiple domains designed to measure the burden of CF in terms of missed opportunities for social, work, and leisure activities, rather than solely via physical limitations.

In the initial concept elicitation interviews, many patients reported relatively low levels of perceived symptom burden, but when asked to provide detailed descriptions of day-to-day life experiences, patients reported substantial impacts such as high treatment burden and substantial time spent every day to manage their disease; limitations in social, personal, and work or school life due to CF; effect on family relationships; constant concern and stress about catching infections or being hospitalized; and restraint of themselves from thinking about future goals. This may reflect that patients who are accustomed to living with CF may state that they are feeling well; however, a more detailed description of their lives may illustrate a deeper, more pronounced impact of CF. For example, patients in the present sample describe engaging in activities rather than avoiding them, but they experience limitations such as needing to rest, taking breaks, or requiring assistance from others while engaged in the activity. The CF-IQ has incorporated this pattern of patient experience, for example, within the “stamina” domain, whereby the number of resting periods required to complete tasks is accounted for directly in the item scoring. Similarly, the CF-IQ includes items assessing frequency with which patients need to cancel plans due to changes in condition and assistance required from others. Inclusion of these perspectives has informed the development of items and conceptual domains of the CF-IQ to capture the overall burden of CF. Although the CF-IQ includes some concepts assessed in other existing instruments, it also incorporates important additions of patient-perceived impacts and offers expanded coverage of concepts related to future outlook, sense of control in managing one’s condition, relationships with others, activity limitations, and the burden of treatment compared to those included in previously-developed CF-specific instruments.

It is important to note study limitations. Although the size of the concept elicitation sample (n = 42 patients) was consistent with commonly used qualitative methods and comparable with those employed in development of other de novo PRO instruments [29, 30], there is a possibility for selection bias due to the nature of non-random sampling procedures. In-person interviews may have biased the sample against sicker patients who may not have been healthy enough to participate, although recruitment quotas were used within the purposive sampling approach to ensure representation of key patient characteristics routinely present in CF clinical trial samples (e.g. range of age, sex, disease severity, treatment type). Patient recruitment was also primarily conducted at major academic medical centers. Some of the interviewed patients received their care in smaller community practice settings, but the nature of the recruitment approach may have biased the sample to reflect the experience of patients with CF in larger practice settings.

Conclusion

This qualitative study demonstrates initial content validity of the CF-IQ as a standardized PRO assessment tool for evaluating the impacts of CF, extending the scope of previous instruments to more sensitively quantify the burden of CF in patients treated with CFTR modulators and, thus, may offer a broader assessment of the efficacy of emerging therapies. Further evaluation of the instrument’s measurement properties is underway in current studies, which will be used to complete the psychometric validation of the instrument and may be used to reduce the number of items for some domains.

The CF-IQ can complement and extend information provided by symptom assessment, healthcare resource utilization, and existing PRO data to comprehensively quantify the burden of CF, and can be used in real-world studies to measure changes in the impact of CF.