To understand the importance of disease and treatment attributes from the perspective of members of the US general public, a descriptive qualitative study was conducted . One-on-one in-person qualitative interviews were conducted by three interviewers (SMS, MH and JB) in Seattle, San Francisco, and Dallas. The interview session also included a numeric ranking exercise to assess the importance of disease and treatment attributes relative to one another. The study conduct and reporting were guided by the COnsolidated criteria for REporting Qualitative research (COREQ) .
Identification of disease and treatment attributes
Disease and treatment attributes potentially important for decision-making were identified from several sources, described in more detail below: the publications of the ISPOR Special Task Force on Value Assessment [1, 11]; a targeted literature review; and discussions with a convenience sample of eight members of the US general public. The attributes considered by the ISPOR Special Task Force within any of the five US frameworks that reflected considerations commonly arising in decision-making dilemmas pertaining to rare disease treatments were tabulated [1, 11]. Any additional articles describing value frameworks published since 2018 were identified based on a targeted review undertaken using Pubmed, Google Scholar and an internet search; these publications were also reviewed for further potential attributes relevant to rare diseases [12, 13]. The list of potential attributes was supplemented by review of global health technology assessment agency submission requirements [14,15,16,17,18,19]. Feedback on the relevance of these attributes for healthcare decision-making was solicited from a convenience sample of eight members of the US general public; these individuals were also asked whether there were any additional attributes they felt would warrant consideration. This convenience sample was recruited from Seattle, Washington in October 2019; and discussions with this sample continued until no new attributes were identified based on feedback from two consecutive participants. The resulting set of attributes was used in discussions during the subsequent qualitative interviews (see below; Qualitative Interview Structure).
To reduce burden in the qualitative interviews, the number of attributes to consider was limited to ten per participant. This number was set based on feasibility feedback from pilot interviews, which were conducted with eight individuals to review the ordering, length, and comprehensibility of the interview materials (see below). As a result, from the full range of potential attributes, the list of attributes for discussion within the qualitative interviews was narrowed down to: disease rarity, age at disease onset, disease cause (genetic vs. acquired), availability of treatments, disease severity, life expectancy, mental health, impact on activities of daily living (ADL) and health-related quality-of-life (HRQoL) and caregiver burden.
The final set of attributes included was selected because it covered a wide breadth of aspects, and were well understood by members of the pilot interview sample (compared to potential attributes such as the value of hope or value of innovation, for example) . During the subsequent qualitative interviews, participants also had the opportunity to suggest additional attributes they thought important for discussion.
Qualitative interview participant recruitment
A purposive sample of adults (18 years of age or older) was continuously enrolled for qualitative interviews. Individuals were recruited to generally reflect the age and sex distribution of the US general population; and to include a mix of participants in terms of their familiarity with chronic and rare diseases, and whether they had children living at home. This was undertaken to account for the fact that particular attributes might be predictors of choice in different scenarios. For example, in situations where parents have young children living at home, the young age at onset attribute may resonate more acutely. In contrast, for someone with a more intimate experience of chronic diseases, the rare disease attribute may hold less weight. Therefore, a recruitment target was set of a minimum 8 participants with familiarity with rare diseases, and 8 participants with children living at home; such that approximately one quarter of the assumed minimum sample size would have experience with these factors, to ensure a diversity of experiences were represented. Level of familiarity with chronic and rare diseases was assessed by asking whether a participant or their family member had, or whether they considered themselves familiar with, any of the items on a list of health conditions that included ‘rare diseases’ as well as other more common conditions (e.g. diabetes, hypertension, dementia).
Consistent with sample size considerations for qualitative studies [20, 21], recruitment of a minimum of 30 participants was planned. Telephone recruitment and screening was carried out by a specialist healthcare market research provider using a panel of potential participants who had been assembled through social media and telephone recruitment. The invitation to participate was provided to panel members from each of the target cities by email, and interested participants directed to contact the recruiters for screening against recruitment criteria. Recruitment continued until information saturation was judged to be reached upon no new themes emerging within three consecutive rounds of interviews.
Prior to initiating data collection, approval from the IntegReview Independent Review Board (IRB) was obtained. Written informed consent was obtained from all participants prior to their interview. All participants were compensated $100 for their time.
A semi-structured interview guide and set of visualization props were developed. Questions for the interview guide were developed based on literature reviews of US value frameworks and the particular attributes of interest, feedback from a convenience sample of eight members of the general public, and iterative review by the study team. The interview guide consisted of a series of open-ended questions and prompts developed to understand participant impressions of the importance of the attributes of interest; note that participants were not asked to consider the importance of attributes in the context of financial resource restraints (i.e. they were not required to prioritize one attribute at the expense of another). The visualization props included disease-specific infographics created to highlight variability in levels of the attributes of interest, and were used to solicit participant feedback on these. One set of infographics described a series of health conditions that were either: (1) rare with pediatric onset and life-limiting (atypical teratoid/rhabdoid tumor [ATRT]); (2) rare with pediatric onset (inherited retinal dystrophy [IRD], type 1 diabetes [T1DM]); or (3) non-rare and affecting primarily older individuals (type 2 diabetes [T2DM], Alzheimer’s disease [AD]). Health conditions were selected by the study team to reflect varied ages of onsets, severities, types of clinical manifestations experienced, and frequency; these were representative of variability in levels of the attributes of interest. The content of the infographics was developed and validated through feedback from four clinician and two patient experts in disease areas of interest. Note, when participants reviewed these health conditions, they were not anonymized.
While the primary focus of the interviews was to understand individual preferences towards a broad set of disease and treatment attributes relevant to a wide variety of health conditions, during the latter part of the interview, opinions on the importance of attributes specifically characterizing life-limiting rare progressive diseases were sought. A separate infographic was therefore also developed for DMD, which was anonymized as ‘Disease X’ within the interview (Appendix Fig. 4); this infographic was discussed at the end of the interview, independently from the other health conditions infographics.
All interview materials were reviewed with the pilot test sample to confirm the ordering, duration, and comprehensibility of the interview materials.
Qualitative interview structure
Interviews were conducted in-person at dedicated private research interview facilities in Seattle, San Francisco, and Dallas between November 2019 to February 2020 (before the COVID-19 pandemic). These cities were selected to ensure some geographic variability in recruitment; participants from only three cities were included for feasibility. Each interview was conducted in English and lasted approximately 60 min. Interviews were conducted by three interviewers trained in qualitative methods; interviewers practiced together in pilot tests and met regularly to share insights and ensure approaches were standardized as the interviews progressed. No repeat interviews were conducted.
Following review of a brief preamble to introduce the interviewer and the motivation for the research, participants were asked to review the initial set of disease-specific infographics; comment on the attributes (or combinations of attributes) they, a priori, viewed as the most meaningful in terms of need for research and treatment; and provide the reason for their responses. Then putting aside the infographics, participants considered each individual attribute, ranking these on a scale of 1 (not important) to 10 (very important) in terms of their significance for prioritizing research and treatment. Each individual attribute was discussed in depth, with participants reporting on drivers for their choices, and their perceptions of relationships between attributes. Finally, participants ranked the anonymized Disease X profile (that described DMD) on a scale of 1 (not important) to 10 (very important) in terms of priority for research and treatment, and were asked to contrast their perceptions on the importance of research for Disease X with those of the attributes reflected within the initial health condition infographics they reviewed. This was to investigate whether in-depth consideration of the attributes might affect overall perceptions of the importance of research for a disease with these specific features. Within this part of the interview, participants also provided feedback on, and a numeric value for, what a ‘rare disease’ meant to them.
At the end of the interview, non-identifying demographic details (age, sex, highest level of education) were collected from all participants. Only the interviewer and the participant were present at the time of the interview; after obtaining participant permission, interviews were recorded and later transcribed. The participants were not contacted in follow-up after the completion of the interview so did not provide their feedback on the transcribed data or the study findings.
Transcripts were reviewed by two of the interviewers (SMS and JB), and the wider study team met regularly to discuss and validate emerging themes and interpretation of the data. Transcripts were independently coded by two study team members (SMS and LP) for analysis.
Thematic analysis was used to explore patterns in responses in accordance with the principles and guidelines described by Braun and Clarke [22, 23]. A theoretical approach to thematic analysis, in which codes were derived from the interview guide a priori and assigned using NVivo, allowed the research team to examine key aspects of the data in-depth. The analysis sought to identify themes regarding: (1) which attributes were important to participants; (2) rationales for attribute importance; and (3) broader reasons for importance that may span across individual attributes. Additional codes were inductively identified throughout the analysis.
Demographic and baseline characteristics of the sample were summarized using medians with ranges and numbers (n) with percentages, as appropriate.
Based on the initial set of disease-specific infographics, participant feedback on the reasons they thought the attributes reflected within the condition profile were important, was tabulated. The latter included reasons such as the impact of the pre-specified attributes, or other self-generated reasons for prioritization.
For each attribute, the mean, median, and range of rankings were calculated across participants, and their rationale for judging why attributes were important was summarized. The frequency of participants reporting interactions between attributes was estimated, as was the percentage classifying an attribute with a rank > 6 (i.e. they ranked an attribute as very [rank 7 or 8] or extremely [rank 9 or 10] important). Mean/median rankings, and the frequency of reporting of interactions between attributes, were summarized visually; relationships between attributes were categorized as moderately linked if 40–60% of the participants described an interaction, and highly linked if > 60% described an interaction. Themes that emerged in discussions of interactions across attributes were plotted and patterns reviewed.
For the Disease X (DMD) profile, the mean, median and range of rankings were estimated, the reasons for its importance were compiled, and the percentage of participants ranking the importance of the DMD profile as > 6 was tabulated. How perceptions on attributes reflected within the Disease X profile compared with those of the other profiles was summarized. Finally, feedback from participants on what prevalence estimates correspond to a disease being rare were summarized according to the thresholds of < 1:10,000 individuals and < 1:100,000 individuals.