This study is part of a larger Patient-Centered Outcomes Research Institute® (PCORI)-funded methodology project focused on comparing individual interviews with GCM brainstorming for their relative comprehensiveness and resource intensiveness (detailed methods published elsewhere) [7]. In the current study, we compared the outcomes elicited in a single GCM brainstorming session to those elicited across two and three sessions in order to establish when and whether concept saturation had occurred. All phases of the study were performed in collaboration with our Patient and Key Stakeholder Advisory Board (PAKSAB). The study received institutional IRB expedited approval.
Participants & study setting
Participants were a convenience sample of 52 adults from a large Philadelphia health system. Potentially eligible participants were identified from the health system’s electronic medical record if they had an active diagnosis of type 1 or type 2 diabetes mellitus (DM); a primary care visit, emergency department (ED) visit, or hospital admission in the last 3 months; and moderately- to poorly-controlled DM. Moderate-to-poor control defined as at least two hemoglobin A1c measurements greater than 7.5 for the primary care setting, presentation to the ED with a DM-related problem, or admission to the hospital for a DM-related problem. We excluded patients if they had significant complications related to DM, were undergoing medical clearance, were in police custody or incarcerated, were non English-speaking, or had other major communication barriers.
Eligible individuals were contacted by phone and invited to participate for the upcoming GCM session, with targeted enrollment for a final show-rate of approximately 20 participants per session, consistent with sample sizes recommended in standard GCM guidance [11]. No-shows account for differences in the sample sizes at each session.
Data collection
We performed in-person group brainstorming with three separate groups of participants (A, B, and C). Each brainstorming session lasted approximately 90 min. Written consent and a self-reported demographics survey were obtained from each participant. Participants were compensated $125 to complete all GCM activities.
Participants responded to a prompt developed by the research team and PAKSAB: “You are here as a person with diabetes; when people with diabetes seek care, what are they hoping to improve or make happen?” Participants were given notecards to record ideas before sharing them out loud. During brainstorming, all new ideas shared by the group were added to a list on a document projected at the front of the room. Brainstorming ended when participants had no new ideas to share in the group. Notecards were collected and unique written ideas were added to the list. Participants subsequently completed GCM sorting and rating; however, only the brainstorming data was analyzed to assess concept saturation.
Data analysis
The research team and three PAKSAB members compiled a “master list” of patient-important outcomes (PIOs) by combining the lists of individual brainstormed ideas from the three groups, and then combining similarly themed ideas into PIO’s. For example, “get information on weight loss,” “understand how to control weight,” and “understand how to handle weight gain” were all merged into the PIO “understand how to control weight”. To minimize subjectivity in combining ideas, our entire PAKSAB made final decisions on merging ideas into PIOs, with disagreements reconciled by vote.
We compared the list of PIOs generated at each individual session and those obtained in each combination of 2 sessions to the master list to determine the proportion of final PIOs identified in each individual session or combination of sessions. We visualized unique and overlapping PIOs across all three sessions in a Venn diagram.
Participant demographics
Participant demographics did not differ significantly on any measured variables across the three groups so we report the aggregated demographics. Participants had a mean age of 55.6 years (SD = 15, range 23–95), half (50%) were male, and they were predominately non-Hispanic (94%) African Americans (81%). Most (71%) had a high-school diploma or less, and more than half (56%) reported a household income of less than $50 K/year. Most (71%) reported a history of DM greater than 5 years, and all participants reported at least one other chronic health condition. They had a mean HbA1c of 9.2 (SD = 2.6), consistent with our recruitment strategy of moderately- to poorly-controlled DM patients. There were 24 participants in sessions A, and 14 participants each in the subsequent two sessions.
Results: Outcomes elicitation
Participants generated 85 ideas in session A, 63 in session B, and 47 in session C. The per-person outcome generation in each session was 3.5, 4.8, and 3.3 respectively. After similar ideas were grouped into PIOs, the three sessions generated 38 unique PIOs. As displayed in Fig. 1, session A generated 87% of all PIOs (33/38). All combinations of 2 sessions produced over 90% of all PIOs. Sixteen PIOs were consistent across all 3 sessions. Session B added 3 unique PIOs not present in session A, and session C added 2 PIOs not present in either A or B.