Psychometric validation of the work productivity and activity impairment questionnaire in ulcerative colitis: results from a systematic literature review

Ulcerative colitis (UC) is an inflammatory disease of the colon that is characterized by intermittent periods of disease flaring and remission and affects 900,000 patients in the United States alone [1, 2]. Patients with UC experience recurring clinical signs and symptoms, including rectal bleeding, abdominal pain, frequent diarrhea, and an urgent need to defecate. These symptoms are typically assessed in clinical trials using a disease activity index, such as the Mayo score [3] or the Simple Clinical Colitis Activity Index (SCCAI), [4] among others.

What these disease activity measures often fail to capture, however, is the impact of these clinical signs and symptoms on the everyday functioning of patients with UC. One aspect of functioning likely to be affected by recurrent symptoms is work-related outcomes (WRO), such as absenteeism and impairment in productivity at work (i.e., presenteeism). Several studies have used individual or group interviews of patients with inflammatory bowel disease (IBD; inclusive of both UC and Crohn’s disease) to elicit patients’ input on the effect of the disease on patients’ everyday lives, including work experiences [5–10]. Patients in these studies discussed missing work because of disease-related pain, fatigue, or other symptoms [5, 6, 8–10].

Further, in all of these studies, patients described the negative impact on their work performance due to these symptoms, including the difficulty of accessing a toilet and avoiding meetings or interactions with colleagues to hide their symptoms and frequency of toilet use. The increase of absenteeism and presenteeism has been shown to limit employment opportunities for patients with UC; several studies have reported rates of unemployment and use of disability benefits for patients with IBD that were typically two-to-three times higher than those for matched general population controls [11–14]. Further examination of unemployment and disability rates found that these differences disappear when only including IBD patients who are asymptomatic or in remission [15, 16].

Collectively, these results indicate that patients with active UC have impaired WRO and that inducing remission may improve these outcomes. Thus, it is important that clinical trials capture the degree to which the studied treatment may be able to improve WRO. Since objective data on WRO (e.g., absentee information from employment databases) are difficult to obtain in a clinical trial setting, self-reported measures are typically used to assess the impact of disease and treatment on absenteeism and presenteeism. Several patient-reported outcomes measures (PROMs) have been developed for this purpose; among the most frequently used is the Work Productivity and Activity Impairment questionnaire (WPAI) [17].

The WPAI measures the impact of health problems on absenteeism, presenteeism, overall work performance, and non-work activities. The WPAI has been shown to be reliable, valid, and responsive when used with patients across several disease areas, including those with gastrointestinal conditions (e.g., irritable bowel syndrome, [18, 19] gastroesophageal reflux disease [20], and Crohn’s disease [21, 22]). However, there has not (to our knowledge) yet been an examination of the measurement properties of the WPAI when used with samples of patients with UC; as such, we believe that this article is the first to do so.

The objective of this paper is to report results from the first systematic literature review on the measurement properties of the WPAI when used with UC patients. Studies included in this review were identified within both the published and unpublished (or “gray”) literature. Evidence was synthesized across identified studies for the purpose of examining evidence for the instrument’s reliability, construct validity, ability to detect change, and responsiveness to treatment for the UC patient population.

This first review (to our knowledge) of the measurement properties of the WPAI when used with UC patients shows the instrument to be reliable, valid, able to detect change, and responsive to treatment when used to assess WRO in this patient population. Results from reviewed studies found evidence that WPAI domain scores were reproducible after 12 months in patients who showed no change in underlying disease condition, and showed change in the expected direction among patients whose disease status improved (e.g., achieved remission) or worsened (e.g., relapsed) over time.

Two kinds of evidence supported the validity of the WPAI. First, convergent validity was supported by findings of higher scores on WPAI domains, particularly presenteeism, OWI, and activity impairment, being associated with lower scores on measures of HRQoL and with higher scores on indices of disease activity – that is, worse scores on WPAI domains were associated with worse scores on health status and disease activity, as expected. Second, known-groups validity was supported by findings that WPAI scores were substantially higher for patients with worse disease activity and more severe symptoms, again as expected. Divergent validity was not examined in this review, due to a lack of relevant data from identified sources.

Ability to detect change was supported by evidence of decreases in WPAI scores for patients with improved disease status (i.e., patients in active disease who achieved remission) and by increases in WPAI scores for patients with worsened disease status (i.e., patients in remission who relapsed to active disease). Responsiveness to treatment was evidenced by substantial decreases in scores for patients who received effective treatments. The magnitude of differences in scores over time and by condition exceeded established CIC thresholds, indicating that changes and differences were clinically meaningful.

Measurement properties were weaker for the WPAI absenteeism domain relative to the other three domains. This is likely related to the highly skewed distribution observed for the absenteeism domain in most studies, with the majority of responses indicating zero days absent in the past seven days due to health problems. For example, in Yarlas et al. (2015a) [36], a response of zero was observed in 73% of subjects with active UC (as compared to approximately 25% in the other three domains), meaning that positive change (reduction) in the absenteeism domain was only possible for one-quarter of subjects. This limits the magnitude of improvements that can be observed as a function of treatment or accompanying change in disease status, and the restricted range also can lead to underestimation of correlations with other variables [48].

The magnitudes of association for WPAI domains with criterion HRQoL measures varied across the domains of those measures. Specifically, all WPAI domains showed stronger associations with the SIBDQ Bowel symptoms and Social function domains than with the Systemic symptoms and Emotional function domains. Most WPAI domains also correlated more strongly with the SF-12v2 Bodily pain, Role physical, and Role emotional domains than other domains of the SF-12v2. These patterns of associations are logical from both a clinical and content perspective. From a clinical perspective, bowel symptoms, such as stool urgency and abdominal pain, would lead to impairment in work and activity impairment more than mental health or perception of general health. From a content standpoint, the SIBDQ Social function domain and the SF-12v2 Role physical and Role emotional domains assess constructs involving the impact of health on work and other activities, and thus would be expected to strongly correlate with WPAI domains. So, while divergent validity was not explicitly examined in this review, the patterns of magnitudes of convergence with HRQoL domains are consistent with content overlap.

The majority of studies included in this review reported administering the UC-specific version of the WPAI (i.e., WPAI-UC), with only three studies [29, 30, 38] reporting use of the general health version (i.e., WPAI-GH). One would expect more precision in findings from studies using the WPAI-UC than the WPAI-GH, since the former is more specific to UC activity, but in fact this was not observed in our review, as findings were comparable across both types of instruments. For studies using the WPAI-GH, this may be indicative of the fact that UC accounted for the majority of health problems in these patients, or that within the context of these studies, where the focus was on patients’ UC, patients responses were driven by their perceptions of their UC-specific health even without explicit instruction. However, to maximize the sensitivity of the instrument, we recommend that the disease-specific version be used as a trial endpoint.

There are some limitations in the current review and gaps in the extant literature that require discussion. First, the evidence base was rather small. Evidence for test-retest reliability, ability to detect change, and convergence with HRQoL measures were each based on findings from a single study, while evidence for convergence with measures of disease activity and responsiveness to treatment were based on only three studies each.

The evidence base was limited due the lack of potentially relevant data. We conducted a systematic, comprehensive search of both the published and unpublished literature. However, our search of the unpublished ‘gray’ literature was focused on presentations and posters at conferences deemed relevant to the topic and accessible through the Embase database (as well as the ISPOR conference). Further, as discussed earlier, there were two dozen conference presentations that we were unable to retrieve, either due to the authors’ refusal to provide or our inability to get a response from authors even after repeated contact attempts. Assuming that the availability of presentations was unrelated to the findings reported (which was supported by the titles and abstracts of unavailable presentations), then the findings presented here should be unbiased and generalizable.

Another point for consideration is the quality of the studies included in this review. While all 13 identified studies were judged as acceptable for inclusion during our quality assessment process, most of these studies, or at least their description of them, had weaknesses identified during this process. Most of the posters reported only cursory descriptions of the study sample, limiting the ability to generalize their findings to a larger patient population. None of the studies provided any justification for sample size; however, since we do not consider statistical significance of findings in our review, but rather effect sizes or magnitudes of differences/change, this limitation is not too relevant to our findings.

All but one of the 13 studies failed to make statistical adjustments or use another strategy to identify or control for confounding factors in their statistical models, which was an item on both the JBI and NIH checklists. This failure to control for possible confounding factors means that the effect sizes and magnitude of differences and change in our assessments of validity, ability to detect change, and responsiveness may be biased. Findings from this review of evidence in the literature is bound by the quality of that evidence. While no study was judged to be of poor enough quality that required exclusion from the review, all reviewed studies were flawed in some way, and so the possibility of bias or limits to the generalizability of our findings cannot be fully dismissed.

We examined the responsiveness of the WPAI to effective treatment in observational studies, but did not identify publications of randomized-controlled trials (RCTs) with UC patients in which the WPAI was included as an endpoint. This limited our ability to examine its responsiveness (within-patient change) and sensitivity to detect treatment differences (between-group difference) in RCTs. We are aware of recent, unpublished RCTs with UC patients in which the WPAI was administered, and hope that results from these studies will be made available soon so that the instrument can be evaluated for this purpose.

Another important gap in the existing literature is that no identified studies have solely examined the measurement properties of the WPAI. The review here relied on use of the WPAI data in studies that had other research purposes. A study dedicated to examining the psychometric profile of the WPAI in a large UC sample would provide a much needed base of information regarding these properties.

This review found that the WPAI has demonstrated good measurement properties in studies of UC patients. Findings from these studies included evidence that WPAI domains have adequate test-retest reliability; convergent validity with measures of HRQoL and UC disease activity; discriminant validity in predicting classification of patients by UC disease activity or other health-related outcomes; ability to detect changes in disease activity; and evidence of responsiveness to effective treatment. At the same time, because of the limited evidence base in the published and unpublished literature, these findings should be considered encouraging though preliminary until more evidence emerges.