Skip to main content

Psychometric evaluation of the DAILY EATS questionnaire in individuals living with obesity

Abstract

Background

Physiological and behavioral factors including hunger, satiety, food intake, and cravings are health determinants contributing to obesity. Patient-reported outcome (PRO) measures focused on eating-related factors provide insight into the relationships between food choice and quantity, weight change, and weight-loss treatment for individuals living with obesity. The DAILY EATS is a novel 5-item, patient-reported measure evaluating key eating-related factors (Worst and Average Hunger, Appetite, Cravings, and Satiety).

Methods

Psychometric analyses, consistent with regulatory standards, were conducted to evaluate the DAILY EATS using data from two randomized trials that included individuals with severe obesity without diabetes (NCT03486392) and with severe obesity and type 2 diabetes (NCT03586830). Additional measures included Patient Global Impression of Status (PGIS) and Patient Global Impression of Change items, Impact of Weight on Quality of Life-Lite, Ease of Weight Management, and Patient-Reported Outcomes Measurement Information System Physical Function Short Form 8b and 10a. The reliability, validity, and responsiveness of the DAILY EATS were assessed, and a scoring algorithm and thresholds to interpret meaningful score changes were developed.

Results

Item-level analyses of the DAILY EATS supported computation of an Eating Drivers Index (EDI), comprising the related items Worst Hunger, Appetite, and Cravings. Internal consistency (Cronbach’s coefficient alphas ≥0.80) and test-retest reliability (coefficients > 0.7) of the EDI were robust. Construct validity correlation patterns with other PRO measures were as hypothesized, with moderate to strong significant correlations between the EDI and PGIS-Hunger (0.30 ≤ r ≤ 0.68), PGIS-Cravings (0.33 ≤ r ≤ 0.77) and PGIS-Appetite (0.52 ≤ r ≤ 0.77). Anchor- and distribution-based analyses support reductions ranging from 1.6 to 2.1 as responder thresholds for the EDI, representing meaningful within-person improvement.

Conclusions

The DAILY EATS individual items and the composite EDI are reliable, sensitive, and valid in evaluating the concepts of hunger, appetite, and cravings for use in individuals with severe obesity with or without type 2 diabetes.

Background

A recent initiative to develop a patient-centered disease-illness model for obesity identified physiological and behavioral factors, including hunger, satiety, food intake, and cravings, as health determinants contributing to obesity [1]. Specifically, potential eating-related barriers to weight loss included difficulties in controlling hunger and appetite and the lack of the sensation of fullness after eating a meal. Individuals living with obesity may be able to lose weight or maintain a healthier weight if they have more control over these eating-related factors, thus allowing for more appropriate meal portion-sizes and fewer cravings, particularly for foods high in calories. For individuals living with obesity, with or without concomitant type 2 diabetes mellitus (T2DM), weight loss may relieve physical, social, emotional, and functional impacts associated with obesity [2, 3].

While select weight-loss medications target hormones that control hunger and satiety, patients’ interpretations of such concepts and their role in chronic weight management are not well understood. Patient-reported outcome (PRO) measures focused on factors related to eating may help facilitate better understanding of the relationships between eating-related factors, weight change, and weight-loss treatment.

The DAILY EATS: Measuring Daily Eating Factors questionnaire is a novel 5-item, patient-reported measure developed to provide information about the key factors associated with eating and includes assessments of hunger (2 items), appetite, cravings for unhealthy food, and satiety after meals. An 11-point numerical rating scale (0–10) is used for each item, with a higher value indicating more hunger, bigger appetite, stronger cravings, or greater satiety. Selection of potential concepts for the DAILY EATS was informed by the results of previously conducted obesity-related research, input of clinical and PRO experts, and qualitative research that included concept elicitation interviews with 35 overweight or obese individuals, either with or without T2DM [4]. The results of the qualitative research informed the development of a conceptual model of the hypothesized relationships among the eating-related factors, as well as the impacts of these factors on food quantity and choice, identified as important to patients during the concept elicitation interviews (see Fig. 2 in Appendix).

The pilot version of the DAILY EATS, initially referred to as the Eating-Related Concepts Questionnaire, was subsequently debriefed in three rounds of interviews and refined between each round, as needed, based on participant feedback. The DAILY EATS is designed to be completed as a daily diary (24-h recall period) at the same time each day, preferably in the evening. The daily responses are used to compute item-level weekly averages.

The objectives of this research were to conduct a psychometric evaluation of the DAILY EATS, assessing its reliability, validity, and responsiveness, as well as assess structure to develop optimal composite scores and an interpretation guideline. The psychometric evaluation was conducted by using data from two phase 2 clinical trials of a novel weight-loss medication: one conducted in individuals with severe obesity (body mass index [BMI], 35–50 kg/m2) and without diabetes (Study 1; NCT03486392) and a separate study conducted in individuals with severe obesity and T2DM (Study 2; NCT03586830). Development and psychometric evaluation of the DAILY EATS were conducted in a manner consistent with the review criteria described in the United States Food and Drug Administration’s Patient-Reported Outcome Guidance [5]. Additional details about the psychometric evaluation are summarized in the Appendix.

Methods

Study measures

Instruments used in the psychometric analysis included the DAILY EATS diary; Patient Global Impression of Severity (PGIS) items related to hunger, appetite, cravings, satiety, and physical functioning; Patient Global Impression of Change (PGIC) items related to hunger, cravings, and physical functioning; the Impact of Weight on Quality of Life-Lite (IWQOL-Lite) measure; the single-item Ease of Weight Management (EWM) measure; and the Patient-Reported Outcomes Measurement Information System Physical Function Short Form (PROMIS PF SF) 8b and 10a measures. All instruments were administered on paper in Study 1 and Study 2. Higher scores for the DAILY EATS, PGIS items, and PGIC items are indicative of higher levels of hunger, appetite, cravings, and other eating-related behaviors, while higher scores for the IWQOL-Lite, EWM and PROMIS PF SF indicate better health-related quality of life, greater ease of weight loss, and physical functioning, respectively. Details related to the recall period and time points for the key measures, including the DAILY EATS, used in the psychometric evaluation are provided in Table 7 in Appendix.

Study design and population

Data from the two studies were used separately to evaluate the psychometric properties of the DAILY EATS in individuals living with severe obesity. Study 1 was a randomized, phase 2b, double-blind, placebo-controlled and open-label active-controlled, parallel-group, multicenter, dose-ranging study to evaluate the safety and efficacy of a novel weight-loss medication in individuals with severe obesity without diabetes across 26 weeks of treatment. Study 2 was a randomized, phase 2b, double-blind, placebo-controlled, parallel-group, multicenter, dose-ranging study to evaluate the safety and efficacy of the same novel weight-loss medication in individuals with severe obesity with T2DM across 12 weeks of treatment. All psychometric analyses were conducted without reference to treatment group (i.e., data were pooled across treatment arms into a study-related analysis population). For each study, analyses were conducted using all patients in the modified intent-to-treat clinical analysis data set who completed at least one DAILY EATS item at least 1 day at baseline and also at least 1 day in a follow-up week. Both studies complied with the Declaration of Helsinki and were approved by the relevant investigational review boards or ethics committees for the respective study sites.

Descriptive statistics, missing data, and DAILY EATS structure

The study populations and descriptive statistics for the supporting measures were summarized descriptively. Weekly average and change-score standard descriptive statistics were reported. Floor or ceiling effects for DAILY EATS items were defined as more than 18% of patients (approximately twice the expected probability for each of the 11 categories in a uniform distribution) selecting an extreme response category (e.g., 0 [Not hungry at all], 10 [Extremely hungry]).

The impact of missing DAILY EATS data was evaluated at the daily level of baseline to inform scoring rules for weekly averages using a missing data simulation: different subsets of daily responses to each item were deleted to assess the stability of the resulting distribution.

The pattern of inter-item correlations was evaluated at baseline and end of treatment (EOT: Week 26 in Study 1, Week 12 in Study 2) to inform potential DAILY EATS composite scores, such that moderate correlations (r ≥ 0.30) supported composite formation and strong correlations r > 0.80 indicated potential redundancy. Exploratory factor analysis (EFA) was conducted using an inter-item Pearson correlation matrix based on the weekly scores at baseline and maximum likelihood estimation with robust standard errors. The size of the eigenvalues [6] and the scree plot [7] guided the decision regarding dimensionality.

Each psychometric property was evaluated for the DAILY EATS weekly items and, after reviewing the item-level and the scoring analyses, a single composite of three DAILY EATS items, the Eating Drivers Index (EDI), was developed for scoring purposes. The EDI is scored as the average of the weekly scores for Worst Hunger (Item 2), Appetite (Item 3), and Cravings (Item 4). Early qualitative research [4] and the item-level quantitative analyses support the relevance of all five DAILY EATS items. Future studies may consider reporting both the EDI and the five individual items scores. For the purposes of this manuscript, the remaining properties focus primarily on the EDI composite and the three component DAILY EATS items of Worst Hunger, Appetite, and Cravings.

Reliability

Internal consistency reliability analyses evaluated the degree to which items were associated with one another. Cronbach’s coefficient alpha [8] was computed at baseline and EOT. The approximate range of optimal alphas suggested by Streiner and Norman [9] is between 0.70 and 0.90, indicating a set of items that is strongly related and capable of supporting a unidimensional scoring structure but not redundant.

The test-retest reliability of the DAILY EATS weekly item scores was assessed by computing intraclass correlation coefficients (ICCs) among patients considered to be stable based on an external criterion over the test-retest period. A two-way mixed-effects analysis of variance (ANOVA) with absolute agreement for single measures was used to compute test-retest reliability ICCs [10, 11]. Study 1 data used Week 15 (test) and Week 26/EOT (retest) for a subgroup with no corresponding PGIS change. Study 2 data used baseline (test) and Week 12/EOT (retest) for a subgroup with no corresponding PGIS change.

Validity

Construct validity

Construct validity describes the relationships among multiple indicators of a construct and the degree to which they follow predictable patterns. Cross-sectional correlations were computed between weekly DAILY EATS item and EDI composite scores and supporting measures (i.e., PGIS item scores, EWM, BMI, IWQOL-Lite domain and total scores, and PROMIS PF SF 8a and 10b total scores) at baseline and EOT. The magnitude and direction of the resulting correlation coefficients were compared with respect to specific a priori hypotheses and to Cohen’s guideline [12] for interpreting correlation coefficients: absolute values of correlations of 0.50 or greater are considered strong, correlations that fall between 0.30 and 0.49 are moderate, and those that fall between 0.10 and 0.29 are small. Moderate to strong correlations were hypothesized for the weekly DAILY EATS item scores and the EDI composite with corresponding PGIS items (e.g., between DAILY EATS Worst Hunger items and PGIS-Hunger item), whereas smaller correlations were hypothesized between DAILY EATS items and the EDI composite with the PGIS-Physical Functioning (PGIS-PF) item. Trivial (|r| <  0.1) to small correlations were hypothesized for DAILY EATS item scores and the EDI composite with physical function scores based on the PROMIS PF SF 8b and SF 10a.

Known-groups validity

Known-groups analyses comparing subgroups of interest were conducted to evaluate the discriminating ability of the DAILY EATS weekly item and EDI composite scores at baseline and EOT. Analyses of variance, with the use of overall F test and pairwise comparisons based on a priori hypotheses, were conducted to examine mean differences in weekly DAILY EATS item and EDI composite scores between patients classified into subgroups based on the corresponding PGIS items. It was hypothesized that individual DAILY EATS item and EDI composite scores would differentiate between patients who report low levels of eating-related issues versus those who report higher levels on the corresponding PGIS items. It also was hypothesized that patients who reported little to no difficulty with their weight management on the EWM would have lower DAILY EATS item and EDI composite scores, on average, than those patients who report higher levels of difficulty in managing weight.

Responsiveness

The DAILY EATS’ responsiveness—or its ability to detect change when change is expected—was evaluated using multiple methods: by computing correlations of change from baseline to EOT in the weekly DAILY EATS item and EDI composite scores and the supporting outcome measures, ANOVA, and effect-size estimates of change. Specifically, longitudinal correlations were computed between changes in weekly DAILY EATS item and EDI composite scores and changes in the supporting measures (i.e., corresponding PGIC items, weight change percentage, and changes in corresponding PGIS items, EWM, BMI, IWQOL-Lite, and PROMIS PF SF 8b and SF 10a) at the EOT. For the ANOVAs (using overall F test, pairwise comparisons, and effect sizes), it was hypothesized that patients who had improved scores on the corresponding PGIS (or PGIC) would have larger changes indicative of improvement than would patients who have remained the same or worsened on these assessments. For correlational (Pearson) analyses, the following correlations were hypothesized: (1) Moderate to strong correlations between changes in weekly DAILY EATS item and EDI composite scores and changes in the corresponding PGIS items; smaller correlations between changes in weekly DAILY EATS item and EDI composite scores and the change in PGIS-PF; (2) Moderate to strong correlations between changes in DAILY EATS Worst Hunger and Appetite items and PGIC-Hunger; moderate to strong correlations between the change in the weekly DAILY EATS Craving item and PGIC-Craving; smaller correlations between the changes in weekly DAILY EATS item and EDI composite scores and PGIC-Physical Functioning (PGIC-PF); (3) Small correlations between changes in weekly DAILY EATS item and EDI composite scores and changes in the physical function scores from the IWQOL-Lite and the PROMIS PF SF 8b and 10a; and (4) Small to moderate correlations between changes in weekly DAILY EATS item and EDI composite scores and the weight change percentage. Effect sizes of approximately 0.20 were interpreted to represent small effects, those of approximately 0.50 represented moderate effects, and those greater than approximately 0.80 represented large effects [13].

Interpretation of change

To identify patients who experienced a meaningful change, a threshold or responder definition was estimated for three weekly DAILY EATS items (Worst Hunger, Appetite and Cravings) and the EDI composite score. Both anchor-based and distribution-based methods were used to estimate thresholds defining meaningful within-person change, or responder definitions, of the weekly DAILY EATS item and EDI composite scores in individuals with severe obesity without diabetes (Study 1) and with T2DM (Study 2). An anchor-based approach is the primary method recommended in the PRO guidance [5] to define this threshold. Prior to applying anchor-based methods, the appropriateness of the anchor measures was assessed by reviewing responsiveness correlations. A commonly applied criterion for identifying an appropriate anchor measure was used: the magnitude of the correlation of change was required to be at least 0.371, based on achieving a large effect size using Cohen’s rule of thumb [14,15,16,17]. In addition, the size and direction of the mean and median change in the weekly DAILY EATS item and EDI composite scores by the change in the corresponding anchor measures were reviewed to confirm that greater improvement or worsening in the weekly DAILY EATS item and EDI composite scores was achieved by patients who showed greater levels of improvement or worsening on the change in the anchor measures.

A 1-point improvement on the related PGIS was selected a priori as the primary anchor. Distribution-based estimates were also conducted to provide additional information and to serve as secondary threshold estimates. Finally, to support the anchor-based methods, cumulative distribution function (CDF) and probability density function (PDF) plots were developed.

Results

Sample characteristics

Table 1 presents key baseline characteristics of the 99 patients from Study 1 (individuals with severe obesity without diabetes) and the 146 patients from Study 2 (individuals with severe obesity with T2DM) in the psychometric analysis sample. Patients without diabetes and those with diabetes had an average BMI of 40.9 and 40.3, respectively, and were aged, on average, 48.2 years and 56.4 years at time of study entry. Both samples contained a higher proportion of female patients (71.7%, 58.9%) than male patients (28.3%, 41.1%), and patients were predominantly white (79.8%, 69.2%) and of non-Hispanic or Latino ethnicity (79.8%, 74.4%).

Table 1 Patient Characteristics at Baseline (Psychometric Analysis Sample)

Descriptive statistics of the supporting PRO measures used in the psychometric evaluation were reviewed (data not shown). The dominant baseline responses were “Moderate” on the PGIS-Hunger, the PGIS-Cravings, and the PGIS-Appetite; this supports patients acknowledge concerns in the key eating behavior concepts assessed on the DAILY EATS. Notably, most patients reported being “Completely satisfied” on the PGIS-Satiety in both studies, suggesting that patients were eating to being comfortably full. The baseline scores of the PRO measures addressing physical functioning and health-related quality of life tended to correspond to a better status in the sample without diabetes (Study 1) than the scores in the sample with T2DM (Study 2).

The trends in the responses of PGIS-Hunger, PGIS-Cravings, and PGIS-Appetite showed improvement from “Moderate” to “Mild” by EOT. In addition, by EOT patients on average showed some overall improvement on all the supporting measures in both studies.

Descriptive statistics, missing data, and DAILY EATS structure

An examination of the item response distributions during the baseline weeks in Study 1 and Study 2 indicated little evidence of ceiling effects and no evidence of floor effects. Over the baseline week, the highest percentage of patients who reported a daily score of 10 on any day was from DAILY EATS Satiety (Item 5) in both studies (15.3% in Study 1; 19.9% in Study 2) (data not shown). The baseline weekly averages were indicative of moderate severities on eating-related concepts and ranged from 5.9 (Average Hunger [Item 1]) to 7.1 (Satiety [Item 5]) in Study 1 and from 5.0 (Average Hunger [Item 1]) to 6.9 (Satiety [Item 5]) in Study 2 (Table 8 in Appendix).

The average weekly change from baseline to EOT was an improvement (a decline for Items 1–4 and an increase for Item 5) of approximately − 1.1 points across the items in both studies; the change in Cravings (Item 3) was the largest at − 1.6 (Study 1), and the change in Satiety (Item 5) was the smallest at 0.3 and 0.0 points (Table 8 in Appendix).

Across evaluated time points and studies, more than 98% of patients completed all five items of the DAILY EATS for at least 6 days, indicating very good assessment compliance. No problematic completion differences were observed across the items. Missing simulation analyses in both studies showed that the 95% CIs of the SD of each item-level weekly score from partially complete data were still within the ±0.5 limits of the SD from complete data, despite the random loss of up to 6 daily responses. These results support the proposed missing rule for weekly scoring (requirement of at least 4 days of data per week).

Satiety (Item 5) scores performed differently than the other DAILY EATS items (i.e., low EFA loadings and weak inter-item correlation (Table 9 in Appendix) and Table 2) when evaluating the DAILY EATS structure. Further, because Average and Worst Hunger item scores were found to be potentially redundant (i.e., a high degree of collinearity due to overlapping content area), Worst Hunger (Item 2) was retained for further consideration instead of Average Hunger (Item 1). Subsequently, analyses of the DAILY EATS structure supported the computation of a three-item DAILY EATS composite, the EDI, as the average of the weekly scores for Worst Hunger (Item 2), Appetite (Item 3), and Cravings (Item 4) for both populations. It is recommended that Average Hunger (Item 1) and Satiety (Item 5) should be reported separately.

Table 2 Reliability and Structure of the Weekly DAILY EATS item and EDI Composite Scores

Reliability

Item-level test-retest reliability coefficients were above 0.7, except for Appetite (Item 3) in Study 2 and Satiety (Item 5) in both studies (Table 2). The smaller magnitude of the ICC for Satiety (Item 5) was expected since responses in both samples were high throughout the treatment period, reducing the scores’ variability across participants (hence the ICC) at each time point. Internal consistency reliability for the EDI was strong across studies and time points (all Cronbach’s coefficient alpha ≥0.80), providing evidence to support the relationships among the items to justify reporting a composite score. Test-retest reliability coefficients were greater than the 0.7 threshold for both studies for the EDI, indicating stability in the EDI scores.

Validity

Construct validity

Correlations were computed between the weekly DAILY EATS items and EDI composite scores and supportive measures at EOT (Table 3). Correlation patterns observed were generally as hypothesized. Specifically, strong positive correlations were observed between the DAILY EATS items for Worst Hunger, Appetite, and Cravings and their corresponding PGIS items. Correlations between pairs of DAILY EATS items and PGIS referring to similar content were typically the largest observed. Also as expected, the correlations between the three DAILY EATS item scores and PGIS-PF were considerably lower than the correlations between the item scores and the eating-related PGIS items. EDI composite scores consistently showed moderate to strong correlations with PGIS-Hunger (0.30 ≤ r ≤ 0.68), PGIS-Cravings (0.33 ≤ r ≤ 0.77), and PGIS-Appetite (0.52 ≤ r ≤ 0.77) across time points and studies. These correlations were significant and much stronger than the correlations between the EDI composite and any other supporting measure (e.g., PGIS-Satiety, PGIS-PF). Small to moderate negative correlations were observed between the three DAILY EATS item and EDI composite scores and all the IWQOL-Lite scores. Trivial to small correlations were observed between the three DAILY EATS item and EDI composite scores and PROMIS PF SF scores. As expected, the correlations with BMI in Study 1 were trivial (near 0) and the correlations with BMI in Study 2 were small.

Table 3 Construct Validity Correlations for DAILY EATS Scores

Known-groups validity

Known-groups ANOVAs were conducted to evaluate the discriminating ability of the three weekly DAILY EATS item and EDI composite scores at baseline and Week 26/EOT or Week 12/EOT. As hypothesized and shown in Table 4 for EOT, patients reporting “No …” or “Mild …” (e.g., hunger, appetite, cravings) on the corresponding PGIS (i.e., PGIS-Hunger, PGIS-Appetite, PGIS-Cravings) had lower (less severe) weekly DAILY EATS item and EDI composite scores on average than those with “Moderate …” or “Severe …” responses (P <  0.0001). As expected, the mean weekly DAILY EATS item and EDI composite scores increased (increased hunger, appetite, or cravings) as the PGIS level increased (increased hunger, appetite, or cravings), providing strong support for the discriminating ability of the individual DAILY EATS items and the EDI composite. Results were strongest for Cravings (Item 4) and the EDI composite, with the overall and all pairwise comparisons statistically significant at the P <  0.05 level.

Table 4 Known-Groups Results for the Weekly DAILY EATS item and the EDI Composite Scores

Responsiveness

The correlation coefficients for change from baseline to EOT scores between the three weekly DAILY EATS items and the EDI composite with a subset of the supporting measures are shown in Table 5. Table 10 in Appendix, provides the correlations for all supporting measures. Correlations between change in the weekly DAILY EATS item and EDI composite scores and change scores in the PGIS and PGIC measures, which assessed similar constructs, tended to be moderate to strong (|r| ≥ 0.30), as expected. Notably, Worst Hunger, Appetite, Cravings, and the EDI composite were more strongly correlated with change in eating-related PGIS and PGIC items than in other measures. Finally, correlations between the three DAILY EATS items and EDI composite and the IWQOL-Lite, PROMIS PF SF (8b and 10a), BMI, and weight change percentage tended to be small to moderate, as expected. Results for the additional longitudinal responsiveness evaluations are included within the (Table 11 in Appendix).

Table 5 DAILY EATS Ability to Detect Change Results

Interpretation of change

Anchor-based and distribution-based methods were used to estimate thresholds defining meaningful within-person change, or responder definitions, of the three DAILY EATS item and the EDI composite in individuals with severe obesity without diabetes (Study 1) and with T2DM (Study 2) after confirming the appropriateness of the candidate anchor measures. Table 6 displays the responder definition estimates characterizing improvement based on change in the corresponding PGIS and PGIC items, as well as the half–standard deviation and standard error of the measurement (SEM) estimates. Due to the small sample sizes in the 1-point deterioration PGIS subgroups and the “Moderately worse/Moderately hungrier/Moderately stronger cravings” PGIC subgroups, the estimation of thresholds identifying deterioration are not recommended using the current data. A larger sample is recommended to further investigate deterioration. Tables 12 and 13 in Appendix show the complete set of results.

Table 6 Range of Potential Meaningful Within-Person Change Thresholds Characterizing Improvement on DAILY EATS item and EDI Scores (Responder Definitions)

The range of responder definitions, based on a 1-point improvement in the PGIS, the primary anchor, were higher than the range of estimates based on PGIC and the distribution-based methods. The thresholds estimated using anchor-based methods with Study 1 data tended to be slightly larger than the anchor-based thresholds estimated with Study 2 data. However, the SEM–based estimates were larger in Study 2 due to the lower ICCs (resulting from the test-retest evaluation timespan). Furthermore, all estimates were closer in magnitude across studies than within study using different methods (e.g., PGIS based, PGIC based, distribution based).

The CDF and PDF plots were reviewed to provide visual support of the primary anchor measures. For example, a greater proportion of patients with improvement in PGIS-Hunger also achieved improvement in the EDI composite from baseline to EOT across a range of possible response thresholds, as shown in the CDF curves for Studies 1 and 2 (Fig. 1a-b). The 1-point improvement (cyan blue) curve is clearly distinct from the no change (green) curves in each curve, providing support for the use of the 1-point improvement in PGIS as the primary anchor. In addition, PGIS-Hunger was adequately associated with the EDI composite change scores within each level of change in PGIS, as shown in the PDF plots for Studies 1 and 2 (Fig. 1c-d).

Fig. 1
figure 1

Changes From Baseline to EOT in EDI Composite by PGIS-Hunger. a. Empirical CDF Plot, Study 1a . b. Empirical CDF Plot, Study 2a. c. PDF Plot, Study 1b. d. PDF Plot, Study 2b. CDF = cumulative distribution function; EDI = Eating Drivers Index; EOT = end of treatment; ERCQ = Eating-Related Concepts Questionnaire; PDF = probability density function; PGIS = Patient Global Impression of Status; T2DM = type 2 diabetes mellitus. a Bands around selected lines denote 95% confidence limits with colors specified by the legend for 1-point improvement and no-change subgroups. b The kernel densities are estimated via normal weight functions with the standardized bandwidth = 0.79 × interquartile range × n − 1 ÷ 5. Solid vertical lines denote means of different curves, and dashed vertical lines denote the corresponding 95% confidence limits for 1-point improvement and no-change subgroups. The confidence limits outside the change range are not shown

Discussion

The purpose of this analysis was to evaluate the DAILY EATS measurement properties using data from two studies in severely obese adult patients with and without T2DM.

Descriptive statistics for the DAILY EATS item scores suggested adequate item performance with no limiting distributional anomalies or response biases in the daily and weekly average scores at baseline or EOT. Furthermore, the change in scores across time points was indicative of improvement during the study period in Average Hunger, Worst Hunger, Appetite, and Cravings items. In comparison, patients reported fairly high Satiety scores at baseline, suggesting they experienced a great deal of satisfaction with being “comfortably full” prior to treatment, and the scores over time provided evidence of maintenance of satisfaction.

A review of the structure of the DAILY EATS informed the preliminary scoring decisions. Average Hunger, Worst Hunger, Appetite, and Cravings items were strongly correlated, and the results suggest that these items can support the formation of a composite score. Scores for Average and Worst Hunger exhibited a high degree of collinearity, which may be viewed as redundant (overlapping content area); thus, the Worst Hunger item was retained in the composite instead of Average Hunger. The resulting composite, the EDI, is an average of the three item weekly scores. The item-level results also indicated that Satiety scores performed differently than the other DAILY EATS items (i.e., low loadings and weak inter-item correlation). These results corroborate the findings from the qualitative work with obese patients that the concept of satiety is distinct from the other eating-related factors. Given the importance of the concept of satiety, it is recommended that the DAILY EATS questionnaire retain the Satiety item and report it in addition to the other item and EDI composite scores. Due to the small sample sizes in the PGIS and PGIC subgroups for satiety, the estimation of thresholds identifying deterioration could not be evaluated using the current data. A larger sample is recommended to further investigate deterioration.

Overall, Average Hunger, Worst Hunger, Appetite, Cravings, and the EDI composite weekly and change scores demonstrated acceptable measurement properties. Internal consistency evidence was strong and supported the EDI composite. Test-retest reliability estimates were well above the recommended 0.70 threshold when using Study 1 data; ICCs based on Study 2 were not as strong, potentially owing to differences in the studies’ respective test-retest evaluation time points. Study 1 used a span of 9 weeks, and both time points were within the treatment period; Study 2 used a span of 14 weeks, in which the first time period occurred within the pretreatment phase and the second time period occurred within the treatment phase. The remaining properties focused on the EDI composite.

For construct validity, the patterns of correlations with other PRO measures were as hypothesized and consistent across the two studies, thus supporting the weekly DAILY EATS item scores and EDI composite scores and the constructs measured. Mean weekly item and composite scores also differed as anticipated and significantly across known groups based on the PGIS, providing evidence for the scores discriminating between meaningful groups. Lastly, the weekly item and composite scores demonstrated responsiveness based on the moderate to strong correlations of change observed with the related PGIS and PGIC measures, the moderate to large effect-size estimates of change, and the moderate to large magnitudes of change observed across levels of change in the PGIS and between PGIC improvement classification groups.

Finally, results of the anchor-based analyses using the PGIS provided evidence that changes ranging from − 1.5 (mean) to − 2.1 (mean or median) for the EDI composite were appropriate for identifying meaningful within-person improvement. Estimates based on the PGIC, a supportive anchor, tended to be lower in magnitude than the PGIS-based estimates for the items and EDI composite, and the distribution-based estimates were lower than the anchor-based values.

Along with the existing qualitative evidence supporting the measure’s content validity in these patient populations [4], the quantitative results provide further evidence that the DAILY EATS item and EDI composite scores are well-defined, reliable, sensitive, and valid for use in individuals with severe obesity with or without T2DM.

Conclusions

The five-item DAILY EATS and its EDI composite exhibit content validity and good psychometric properties for assessing key factors related to eating. The DAILY EATS item and EDI composite scores shows similar performance among individuals with severe obesity alone and individuals with severe obesity and T2DM, providing a fit-for-purpose measure of eating-related behaviors. The proposed scoring algorithm and thresholds for meaningful change are recommended for both populations.

Availability of data and materials

Not applicable.

Abbreviations

ANOVA:

Analysis of variance

BMI:

Body mass index

CDF:

Cumulative distribution function

EDI:

Eating Drivers Index

EOT:

End of treatment

EWM:

Ease of Weight Management

ICC:

Intraclass correlation coefficient

IWQOL-Lite:

31-item Impact of Weight on Quality of Life-Lite

PDF:

Probability density function

PGIC:

Patient Global Impression of Change

PGIC-PF:

PGIC-Physical Functioning

PGIS:

Patient Global Impression of Severity

PGIS-PF:

PGIS-Physical Functioning

PRO:

Patient-reported outcome

PROMIS PF SF:

Patient-Reported Outcomes Measurement Information System Physical Function Short Form

SD:

Standard deviation

SEM:

Standard error of measurement

T2DM:

Type 2 diabetes mellitus

References

  1. Fastenau, J., Kolotkin RL Fujioka, K., Alba, M., Canovatchel, W., & Traina, S. (2019). A call to action to inform patient-centred approaches to obesity management: Development of a disease-illness model. Clinical Obesity, 9(3), e12309.

    Article  Google Scholar 

  2. Ul-Haq, Z., Mackay, D. F., Fenwick, E., & Pell, J. P. (2013). Meta-analysis of the association between body mass index and health-related quality of life among adults, assessed by the SF-36. Obesity, 21, E322–E327.

    Article  Google Scholar 

  3. Warkentin, L. M., Das, D., Majumdar, S. R., Johnson, J. A., & Padwal, R. S. (2013). The effect of weight loss on health-related quality of life: Systematic review and meta-analysis of randomized trials. Obesity Reviews, 15, 169–182.

    Article  Google Scholar 

  4. DiBenedetti, D., Harris, N., Rozjabek, H. M., & Fastenau, J. (2019). Assessing factors influencing eating in overweight and obese individuals. New Orleans: Poster presented at the Endocrine Society Meeting 2019.

    Book  Google Scholar 

  5. Food and Drug Administration (FDA) (2009). Guidance for industry patient-reported outcome measures: Use in medical product development to support labeling claims Available at: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM193282.pdf. Accessed 21 July 2017.

    Google Scholar 

  6. Kaiser, H. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.

    Article  Google Scholar 

  7. Cattell, R. B. (1966). The scree test for the number of factors. Multivariate Behavioral Research, 1, 245–276.

    Article  CAS  Google Scholar 

  8. Cronbach, L. (1951). Coefficient alpha and the internal structure of tests. Psychometrika., 16, 294–334.

    Google Scholar 

  9. Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use, (2nd ed., ). New York: Oxford University Press.

    Google Scholar 

  10. McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.

    Article  Google Scholar 

  11. Qin, S., Nelson, L., McLeod, L., Eremenco, S., & Coons, S. J. (2019). Assessing test-retest reliability of patient-reported outcome measures using intraclass correlation coefficients: Recommendations for selecting and documenting the analytic formula. Quality of Life Research, 28(4), 1029–1033. https://doi.org/10.1007/s11136-018-2076-0.

    Article  PubMed  Google Scholar 

  12. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.

    Article  CAS  Google Scholar 

  13. Cohen, J. (1988). Statistical power analysis for the behavioral sciences, (2nd ed., ). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  14. Hays, R., & Revicki, D. A. (2005). Reliability and validity (including responsiveness). In P. Fayers, & R. Hays (Eds.), Assessing quality of life in clinical trials, (pp. 25–39). New York: Oxford University Press.

    Google Scholar 

  15. Hays, R. D., Farivar, S. S., & Liu, H. (2005). Approaches and recommendations for estimating minimally important differences for health-related quality of life measures. Journal of Chronic Obstructive Pulmonary Disease, 2, 63–67.

    Article  Google Scholar 

  16. Revicki, D., Hays, R., Cella, D., & Sloan, J. (2008). Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. Journal of Clinical Epidemiology, 61(2), 102–109.

    Article  Google Scholar 

  17. Fayers, P. M., & Hays, D. R. (2014). Don’t middle your MIDs: Regression to the mean shrinks estimates of minimally important differences. Quality of Life Research, 23(1), 1–4.

    Article  Google Scholar 

Download references

Acknowledgements

Kate Lothman of RTI Health Solutions provided medical writing services, which were funded by Janssen Pharmaceutical Companies.

Funding

This study was performed under a research contract between RTI Health Solutions and Janssen and was funded by Janssen. Authors affiliated with Janssen participated in design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

JF and HR initiated the analysis, secured funding, contributed to the post hoc analysis study design, provided input on the analyses, interpreted the data, participated in drafting the manuscript, and approved the final version of the manuscript for publication. LN, SQ, and LM contributed to the post hoc analysis study design, oversaw and participated in the analyses, interpreted the data, participated in drafting the manuscript, and approved the final version of the manuscript for publication. NH and JM participated in the analyses, interpreted the data, participated in drafting the manuscript, and approved the final version of the manuscript for publication.

Corresponding author

Correspondence to John Fastenau.

Ethics declarations

Ethics approval and consent to participate

The studies presented here complied with the Declaration of Helsinki and were approved by the relevant investigational review boards or ethics committees for the respective study sites.

Consent for publication

Not applicable.

Competing interests

NH, JM, LM, LN, and SQ are salaried employees of RTI Health Solutions. JF and HR are salaried employees of Janssen.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

Conceptual Model of Eating Related Factors

Fig. 2
figure 2

Conceptual Model

Measures Relevant to the Psychometric Evaluation

Table 7 Measures Relevant to the Psychometric Evaluation

Descriptive Statistics for Supportive Measures

Table 8 DAILY EATS Weekly Scores Descriptive Statistics

DAILY EATS Inter-Item Correlations

Table 9 Inter-item Correlations at Baseline and EOT, Study 1 and Study 2

Responsiveness

Table 10 DAILY EATS Worst Hunger, Appetite, Cravings, and EDI Ability to Detect Change Results: Correlational Analysis of Change

Longitudinal Responsiveness

Patterns of mean change in the weekly DAILY EATS item scores across the levels of the change in the corresponding PGIS and PGIC provide supportive evidence for the responsiveness of the EDI composite score. As an item-level example, the largest average weekly Worst Hunger (Item 2) score change reflecting improvement (negative change) was in the improved PGIS-Hunger subgroup (least square mean, − 2.1 and − 1.8) and this value was significantly larger than the mean change in the stable subgroup (least square mean, − 0.7 and − 0.4; P = 0.0072). In addition to the mean differences, the effect-size estimates of change for the EDI composite based on the SD at baseline, the SD of change, and between the PGIS subgroups are moderate to large, at least 0.50 or greater.

Table 11 DAILY EATS Ability to Detect Change: Longitudinal Construct Validity

Interpretation of Change: Changes From Baseline to End of Treatment in DAILY EATS Items and Anchor Measures

Table 12 Interpretation of Change: Study 1, Severely Obese Without T2DM
Table 13 Interpretation of Change: Study 2, Severely Obese With T2DM

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fastenau, J., Rozjabek, H., Qin, S. et al. Psychometric evaluation of the DAILY EATS questionnaire in individuals living with obesity. J Patient Rep Outcomes 4, 99 (2020). https://doi.org/10.1186/s41687-020-00259-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41687-020-00259-w

Keywords