Do patient-reported outcome measures measure up? A qualitative study to examine perceptions and experiences with heart failure proms among diverse, low-income patients

Abstract

Background

The Kansas City Cardiomyopathy Questionnaire (KCCQ) is a Patient-Reported Outcome Measure (PROM) used to evaluate the health status of patients with heart failure (HF) but has predominantly been tested in settings serving predominately white, male, and economically well-resourced populations. We sought to examine the acceptability of the shorter version of the KCCQ (KCCQ-12) among racially and ethnically diverse patients receiving care in an urban, safety-net setting.

Methods

We conducted cognitive interviews with a diverse population of patients with heart failure in a safety net system to assess their perceptions of the KCCQ-12. We conducted a thematic analysis of the qualitative data then mapped themes to the Capability, Opportunity, Motivation Model of Behavior framework.

Results

We interviewed 18 patients with heart failure and found that patients broadly endorsed the concepts of the KCCQ-12 with minor suggestions to improve the instrument’s content and appearance. Although patients accepted the KCCQ-12, we found that the instrument did not adequately measure aspects of health care and quality of life that patients identified as being important components of managing their heart failure. Patient-important factors of heart failure management coalesced into three main themes: social support, health care environment, and mental health.

Conclusions

Patients from this diverse, low-income, majority non-white population experience unique challenges and circumstances that impact their ability to manage disease. In this study, patients were receptive to the KCCQ-12 as a tool but perceived that it did not adequately capture key health components such as mental health and social relationships that deeply impact their ability to manage HF. Further study on the incorporation of social determinants of health into PROMs could make them more useful tools in evaluating and managing HF in diverse, underserved populations.

Background

Heart failure (HF) is a common, progressive, fatal disease with a prevalence expected to increase by 40% over the next 10–15 years [1]. In the United States, HF incidence, morbidity, and mortality are worse for African American, Hispanic, and other minority populations as compared to Caucasians [2, 3]. Patient-reported outcome measures (PROMs) are important tools to evaluate HF treatments and are associated with clinical HF outcomes [4]. Routine use of PROMs can significantly benefit the provider’s ability to provide personalized, tailored patient care [5]. The Kansas City Cardiomyopathy Questionnaire (KCCQ) is a 23-item questionnaire qualified as a medical device [6] and drug development tool by the FDA [7]. The KCCQ includes items about daily functioning, symptoms, and quality of life (QoL) in patients with HF. Clinical trials often employ the validated KCCQ as a clinical endpoint, due to its ability to predict hospitalization and death [8]. The shorter version of the KCCQ—the KCCQ-12—retains the psychometric properties of the full KCCQ, but is more feasible to implement as it minimizes the response burden patients experience [9]. The formative studies used to develop and validate the KCCQ and KCCQ-12 drew from predominantly White and male populations, similar to many clinical trials and observational studies in HF [8, 10]. We lack a complete understanding of how patients of different racial/ethnic backgrounds respond to items (questions) within the KCCQ [11]. In addition, providers, payers, and policymakers increasingly recognize social determinants of health (SDOH) as direct contributors to HF morbidity and mortality [11, 12]. Despite this, PROMs like the KCCQ and KCCQ-12 do not include information about SDOH and its impact on HF symptoms or self-management.

The goal of this project is to address this health equity gap through the examination of how a racially/ethnically diverse, low-income sample of patients with HF perceive the KCCQ-12. This qualitative inquiry explores how this PROM resonates with diverse patients, including how lived experiences directly impact HF self-management and influence their symptom reporting.

Methods

Study setting and patients

This study enrolled patients receiving outpatient primary or cardiology care at an academic, urban, safety-net system that serves one in eight San Franciscans regardless of immigration or insurance status. The San Francisco Health Network’s (SFHN) Cardiology Clinic at Zuckerberg San Francisco General Hospital (ZSFG) delivers over 4,000 annual visits. Patients have primarily Medicaid or Medicare or are uninsured. Patients were eligible if they met all of the following criteria: (1) spoke English; (2) were between the ages 24–85; (3) received care at primary care or Cardiology clinics at the ZSFG location; (4) diagnosed with HF; and (5) did not have a significant cognitive or visual impairment. We invited eligible individuals to participate in person at ZSFG or by mail, email, or phone. All study visits beginning April 2020 were virtual given the COVID-19 pandemic. Patients provided consent and self-reported demographics. Those who completed an interview with our study team received $25 in cash or gift card for their time. The UCSF Institutional Review Board (IRB) (#18-26769) and the FDA IRB (2019-CDRH-019) approved this study.

Interviews

To characterize and understand how a diverse sample of HF patients interpreted items and responses within the KCCQ, we presented the KCCQ-12 [9], an abbreviated version of the KCCQ questionnaire. Compared to the KCCQ, the KCCQ-12 omits the stability, symptom burden, and symptom efficacy domains plus items from the physical limitation, quality of life, and social limitations domains. The KCCQ-12 remains a valid and reliable tool to measure a patient’s HF-related health status and how it impacts QoL, physical, and social activities [9]. The KCCQ-12 was thus used to be better integrated into clinic workflow. All 18 patients received the KCCQ-12 on paper, and of those patients, six (33%) also received the KCCQ-12 tablet-adapted version. For exploratory purposes, six (33%) also received an adapted version with a lower reading level on paper (see Table 4 for Flesh Kincaid (FK) score). The KCCQ-12 and reading level-adapted KCCQ-12 were printed on 8.5 × 11-inch standard white paper and used 10-pt Arial black font. The study team asked patients for feedback on both content and layout of the KCCQ-12 (e.g., how the KCCQ-12 was presented, readability and font size).

Trained interviewers (KO, MS, SL) conducted cognitive interviews using a semi-structured interview guide (Additional file 1). Cognitive interviewing is a qualitative research method that is used to understand whether questionnaires and survey questions work as intended [13]. There are two approaches that can be used to conduct a cognitive interview—think-aloud and verbal probing. Think-aloud is a technique in which participants are instructed to say anything that comes to mind as they go through the survey. Verbal probing is another technique in which the interviewer follows up with another question to elicit a more detailed response from the participant [13, 14]. Our team used both the think-aloud and verbal probing methods concurrently [13]. The interviews took between 60 and 90 min. Study staff audio-recorded interviews with participant permission. The interviews were professionally transcribed for analysis.

Analysis

According to best practices in qualitative research, we conducted data analysis in parallel with data collection, using the method of constant comparisons. We reached thematic saturation—the point at which no new themes emerged from the interviews—after analyzing 10 interviews and completed data analysis for the remaining two interviews. For the reading level-adapted KCCQ-12, we reached thematic saturation after four of the six interviews.

We utilized Dedoose qualitative analysis software (Los Angeles, CA) to code transcripts [15]. Using an inductive and deductive approach, we coded qualitative interviews to reduce code bias. Two study coders (KO, MS) collaboratively analyzed transcripts to lower variability between interviewers and to support consistent data cleaning.

For the deductive coding, we applied the behavioral Capability, Opportunity, Motivation model of Behavior (COM-B) model [16] to explore the open-ended participant comments about the behaviors of heart failure symptom management/self-management. Michie et al. describes the capability to entail both the patient’s psychological and physical capacity, plus having the necessary knowledge and skills to engage. Opportunity involves those external factors that impact behavior. Finally, motivation includes those factors that prompt and guide behavior, including conscious and unconscious processes. We developed a list of thematic codes based on the interview guide domains and topics then transcripts were re-reviewed along with notes and memos to determine broader thematic codes. Through this process, we developed sub-themes to capture participant experiences.

Results

In total, we conducted 18 qualitative interviews with heart failure patients from April 2019 to September 2020. The mean age was 52 years, 33% were women, and 78% were non-White. Complete baseline demographics are in Table 1.

Table 1 Cognitive interview participant characteristics

Study patients shared a range of experiences managing their heart failure condition as well as a variety of feedback on the KCCQ-12 [original (n = 12) and tablet (n = 6) and reading level-adapted (n = 6) versions]. Table 2 summarizes feedback on the questionnaire including suggested changes.

Table 2 KCCQ-12 feedback

Applying the COM-B framework [16] to the interviews, we identified several key issues for the self-management of heart failure (Table 3). In the capability domain, both physical issues related to medication adherence and psychological issues related to both general HF disease management and their experience with health care providers affected self-management. Patients were concerned about their ability to practice HF self-management behaviors such as taking medications, exercising, and maintaining a balanced diet. They reported unclear knowledge of key behaviors and actions related to HF self-management, which correspond with their ability to understand health information. In the motivation domain, themes related to beliefs about their own identity and role as a patient with HF, emotions, and attitudes toward heart disease, and their goals for HF management emerged. Items in the opportunity domain intersected with the motivation domain, namely their social support and physical environment such as living environment. Both strongly impacted their ability and desire to manage their HF. Environment—home, neighborhood, work, and even clinic—repeatedly emerged as a major SDOH that touched every aspect of their HF management.

Table 3 Mapping quality of life and HF management to the COM-B framework

We calculated the FK reading level of the KCCQ-12 and it was higher than the recommended 8^th-grade reading level for patient-reported outcome measures. Therefore, we adapted it to recommended reading levels and explored patient perceptions of the literacy-appropriate version (Table 4). While some patients found them to be “the same” (ptid29, ptid395), others preferred the simplified version. None of the interviewees expressed a preference for the original KCCQ-12 version on paper.

Table 4 Survey content

Discussion

This prospective qualitative thematic analysis of patients’ perceptions of the KCCQ-12 in an integrated, urban safety-net health system demonstrated that while items on the KCCQ-12 were relevant to their symptoms, they did not adequately capture all the factors that affect daily life with HF, such as diet, sexual activity, or key SDOH such as mental health and social relationships (Additional file 2). However, for patients to successfully manage their heart disease, all three components of the COM-B system (capability, opportunity, motivation) are important. Importantly, HF patients participating in these interviews did not draw categorical distinctions between symptoms, such as the extent of their shortness of breath, and their self-management, such as their dietary restrictions. Opportunity domains, such as one’s built environment and social factors like the quality of patient-provider communication, may influence a patient’s opportunity to engage in self-management behaviors. Mental health, resiliency, and social support are patient-important factors that were not captured on the KCCQ-12 but may influence a patient’s motivation to manage their heart disease. This is particularly relevant in this diverse, publicly insured population who face increased medical and social complexity in everyday life [12]. Inclusion of a broader range of factors into HF PROMs may increase their relevance for diverse populations.

In our exploratory analysis, patients preferred an adapted version of the KCCQ-12, which reflected a lower literacy level, consistent with prior studies comparing higher vs. lower literacy patient-facing materials [17]. Modifying the KCCQ-12 literacy level for diverse and low-income populations may improve its performance in predicting HF outcomes. Before the literacy-adapted version of the KCCQ-12 can be used, further testing in diverse populations with HF will need to be performed.

The KCCQ-12, a novel and well-validated health-related quality of life assessment tool for patients with HF, is comprised of distinct domains: physical limitations, symptoms (frequency, severity, and change over time), self-efficacy and knowledge, social interference, and quality of life [8, 18]. The subjective perception of dyspnea varies greatly and impacts the quality of life [19], and the KCCQ-12 captures this. The KCCQ-12 does not, however, capture patients’ SDOH, which has also been shown to impact risk [20]. It has been shown that correlating physical and psychological symptom profiles are predictive of survival [21]. The multiple factors that impact these, from mental health to housing to food insecurity, are acknowledged but seldom incorporated into patient assessments and shared-decision making [11]. The current study further supports the imperative to prioritize and directly assess the broader SDOHs that impact patient experience, disease management, and ultimately survival.

Social determinants of health deeply impact patient experience and outcomes and are disproportionately present in patients with HF as compared to the general population. For example, depression is more common in patients with HF and portends a worse prognosis [22, 23]. Similar trends are seen with other SDOHs such as socioeconomic position [24], food insecurity [25], and unemployment [26]. Non-White patients tend to experience these SDOHs more commonly than white patients, are underrepresented in clinical trials, and have poorer outcomes [2, 3, 10, 27]. This study of almost exclusively non-White patients brings to the surface these multiple factors that impact the burden of living with heart failure, from polypharmacy to dietary recommendations to social support to sex. Assessing and understanding how an individual patient navigates HF management with the competing demands of life, health, and work is paramount to providing thoughtful, personalized, and appropriate care.

Using the COM-B framework [16], we identified several key issues for the self-management of heart failure congruent with prior research [28, 29]. The KCCQ-12 most heavily captures capability, particularly physical issues, and patients’ feedback in this domain primarily focused on its psychological components. Most notably, patients in this study articulated the underrepresentation of the opportunity and motivation domains. They do not separate their HF symptoms from either external factors or conscious and unconscious processes that prompt and guide behavior. Patient perspectives on the overlap between the opportunity and motivation domains highlight the critical influence that SDOH has on HF self-management. Attributes in the opportunity and motivation domains weigh heavily on those facing medical and social complexity and have not been fully explored to date. This finding highlights a challenge in the use of PROMs across settings. Well-validated tools such as the PHQ-9 unfortunately do not fully capture those additional social determinants of health beyond mental health that impact their heart failure experience. Thus, inclusion of such an assessment in conjunction with the KCCQ-12 still leaves gaps in SDOH assessment. For medical product performance, PROMs need only include the target symptoms for the device or drug. However, for clinical care, patients experience their illness in the context of their social vulnerabilities. PROMs developed for regulatory purposes may be insufficient to inform clinical decision-making without consideration of SDOH.

Several limitations exist in the current study. We conducted interviews in a single safety-net health system hospital limiting external generalizability. However, this study touches on key demographic groups not typically represented in clinical studies. Only English-speaking patients were included, which unfortunately excluded non-English speaking patients in the SFHN, most commonly Spanish and Cantonese. This is an area of ongoing study. We employed the KCCQ-12 rather than the original 23-item KCCQ, which does have an item on depression and an item on sexual function. Based on our patients’ feedback, though, inclusion of these items would not have sufficiently addressed those SDOH deficiencies described above. Finally, the COVID-19 pandemic necessitated that the final 6 months of the study be conducted remotely via telephone or video conferencing. Given the socioeconomic makeup of our patient population, this likely impacted who was able to participate. Shelter-in-place requirements increased patient isolation and impacted patients’ feedback of the KCCQ-12. COVID-19 has exaggerated health disparities [30,31,32], and the impact of PROM and SDOH can be further explored.

Conclusion

Patients from a diverse, low-income, majority non-white population in a safety net health care system experience unique challenges and circumstances that impact their ability to manage disease. It is important that future studies continue to enroll such patients, as SDOH disproportionately affects them negatively and impacts public health outcomes. In this study, patients were receptive to the KCCQ-12 as a tool but perceived that it did not adequately capture their illness experience because it did not assess key SDOH such as mental health and social relationships that deeply impact their ability to manage HF. Further study on the intersection of PROMs and SDOH reporting could investigate how to best use standardized patient reporting in evaluating and managing HF in diverse, underserved populations.