- Open Access
Measuring respiratory symptoms in moderate/severe asthma: evaluation of a respiratory symptom tool, the E-RS®: COPD in asthma populations
Journal of Patient-Reported Outcomes volume 5, Article number: 104 (2021)
Symptom constructs included in the Evaluating Respiratory Symptoms in Chronic Obstructive Pulmonary Disease (E-RS®: COPD) tool may be relevant to patients with asthma. The purpose of this study was to evaluate content validity and psychometric performance of the E-RS: COPD in moderate/severe asthma patients.
Content validity of the E-RS: COPD was evaluated in patients with moderate/severe asthma using concept elicitation and cognitive debriefing interviews. Secondary analyses using data from two clinical trials in patients with moderate/severe asthma evaluated the factor structure of the E-RS: COPD plus two supplementary items (wheeze; shortness of breath with strenuous physical activity) and assessed psychometric properties of the tool, which will be referred to as E-RS®: Asthma when used in asthma populations.
Qualitative interviews (N = 25) achieved concept saturation for asthma respiratory symptoms. Concepts in the E-RS: COPD were relevant to patients and instructions were understood. Most patients (19/25; 76%) reported experiencing all concepts in the E-RS: COPD; no patients indicated missing symptoms. Secondary analyses of clinical trial data supported the original factor structure (RS-Total and three symptom-specific subscales). The two supplemental items did not fit with this factor structure and were not retained. RS-Total and subscale score reliability was high (internal consistency [α] > 0.70). Validity was demonstrated through significant (P < 0.0001) relationships with the St George’s Respiratory Questionnaire (SGRQ) and Asthma Symptom Severity scale. E-RS: Asthma was responsive to change when evaluated using SGRQ, Patient Global Impression of Change and Asthma Quality of Life Questionnaire as anchors (P < 0.0001). Clinically meaningful change thresholds were also identified (RS-Total: − 2.0 units).
The E-RS: Asthma is reliable and responsive for evaluating respiratory symptoms in patients with moderate/severe asthma.
Asthma is a chronic inflammatory airway disease characterized by respiratory symptoms such as wheeze, shortness of breath (SOB), chest tightness, and cough , which impair health-related quality of life, often in proportion to disease severity [2,3,4]. Understanding overall symptom burden in patients with asthma is a critical step in assessing the impact of treatment-related changes in both clinical trials and clinical practice . However, respiratory symptoms are often only assessed periodically using diaries in individual studies, making between-study comparisons difficult. This variation in methods used for symptom assessment indicates a need for improved patient-reported measures of symptomatic experience in asthma. Furthermore, asthma symptoms vary daily, indicating that a standardized daily diary measure is appropriate.
The Evaluating Respiratory Symptoms in Chronic Obstructive Pulmonary Disease (E-RS®: COPD) measure is an 11-item patient-reported diary derived from the 14-item The EXAcerbations of Chronic Pulmonary Disease Tool (EXACT®) . The EXACT was developed to assess the frequency, severity, and duration of COPD exacerbations from the patient perspective. The E-RS: COPD assesses the cardinal respiratory symptoms of COPD as a total score through three symptom-specific domains: cough and sputum, chest symptoms, and breathlessness .
Literature reviews [7, 8] and qualitative research  indicate that respiratory symptom constructs in the E-RS: COPD may also be relevant to patients with moderate/severe asthma, as both are diseases of airflow obstruction and share common symptoms, such as SOB and cough [1, 10]. However, it is unclear whether the E-RS: COPD can adequately detect symptoms across the full range of asthma severity. Furthermore, no research has assessed content validity or psychometric performance of the E-RS: COPD in asthma populations.
The current analysis comprised a two-phase approach utilizing both qualitative and quantitative data. First, qualitative interviews were conducted to determine the suitability and content validity of the E-RS: COPD in a moderate/severe asthma population. Second, data from two clinical studies in moderate/severe asthma patients [11, 12] were used to evaluate the factor structure of the E-RS: COPD in asthma populations (hereafter referred to as the E-RS®: Asthma when used in the context of asthma patient populations). Subsequently, psychometric properties (ie, reliability, validity, and responsiveness) of the E-RS: Asthma were assessed in this patient population. A responder threshold for clinically important change in this asthma population was also established.
Phase 1: qualitative study
A cross-sectional, qualitative study (GSK: 206605; ClinicalTrials.gov: NCT03344406) involving one-on-one semi-structured concept elicitation and cognitive debriefing telephone interviews was conducted between December 2017 and September 2018 with patients attending five US asthma and allergy clinics. Quorum Review Institutional Review Board (IRB) approval was obtained prior to initiation of the study (Quorum Review File # 32810). Interviews were conducted in English or Spanish language, lasted 60–90 min and were recorded and translated into English (if required). Eligibility criteria are summarized in Supplementary Table 1. Briefly, adults aged ≥18 years with moderate/severe asthma with a history of airflow obstruction and evidence of bronchodilator reversibility were eligible for inclusion. Current smokers and patients with a diagnosis of COPD or other clinically important lung conditions were excluded.
Interviews followed a semi-structured interview guide. All participants provided written informed consent prior to the interview. Interview questions were used to elicit asthma symptoms from the patient perspective first, and subsequently participants completed the E-RS: COPD questions in a paper format. Participants provided further feedback during cognitive debriefing to assess comprehension, relevance, and completeness of the tool.
Interviews were transcribed and anonymized prior to descriptive analysis using ATLAS.ti version 7.0 or higher (ATLAS.ti Scientific Software Development GmbH, Berlin, Germany). A coding dictionary was developed iteratively based on concept themes related to participants’ symptom experience. Words and phrases reported by interview participants were selected using the coding dictionary and grouped into key themes, attributes, concepts, and relationships. Saturation was defined as the point at which no major new themes, descriptions of a concept, or terms were introduced as subsequent interviews were conducted . Concepts were subsequently thematically mapped to assess content coverage. Feedback on the E-RS: COPD was obtained using qualitative codes to capture overall feedback, item comprehension, recall period, and response options. Results are reported in accordance with the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist .
Phase 2: quantitative psychometric evaluation
Utilizing data from two randomized controlled trials (RCTs) (GSK: 205832; ClinicalTrials.gov: NCT03012061 ; GSK: 205715; ClinicalTrials.gov: NCT02924688 ), a quantitative psychometric evaluation was performed to first evaluate the factor structure of the E-RS: COPD with the intent to use this tool in a moderate/severe asthma population (E-RS: Asthma). Subsequently, the scoring algorithm and psychometric properties, including reliability, construct validity, and responsiveness were evaluated. IRB approval was obtained prior to initiation of patient recruitment or administration of measures as part of each clinical trial (205832 and 205715). No additional ethics committee or IRB approval were required for this secondary data analysis.
GSK 205832 was a Phase IIb, randomized, double-blind, placebo-controlled, three-arm, parallel-group study conducted between January 2017 and May 2018. The study evaluated the efficacy, safety, and tolerability of two doses of umeclidinium bromide (UMEC) administered once daily (QD) via the ELLIPTA dry powder inhaler (DPI) over 24 weeks in patients with moderate uncontrolled asthma receiving fluticasone furoate (FF) 100 mcg QD. GSK 205715 was a Phase IIIa, randomized, double-blind, active-controlled, six-arm, parallel-group study conducted between December 2016 and August 2018. This study compared the efficacy, safety and tolerability of four fixed-dose triple combinations of FF/UMEC/vilanterol (VI) with two fixed-dose dual combinations of FF/VI, administered QD via the ELLIPTA DPI for 24–52 weeks in patients with moderate/severe asthma uncontrolled on inhaled corticosteroids (ICS) ± a long-acting β2-agonist (LABA) therapy with the primary outcome of the study (clinic trough forced expiratory volume in 1 s [FEV1]) completed at Week 24.
Detailed eligibility criteria for both trials are summarized in Supplementary Tables 2 and 3, and have been presented elsewhere [11, 12]. Briefly, 205832 enrolled a moderate asthma population prescribed ICS ± a LABA or a long-acting muscarinic antagonist (LAMA), whereas 205715 included a broader population of patients with moderate/severe disease uncontrolled on ICS/LABA. There were also differences in asthma control at baseline. Thus, 205832 required patients to have Asthma Control Questionnaire (ACQ)-6 score > 0.75 at screening (partially or inadequately controlled asthma), whereas 205715 restricted entry to those only with inadequately controlled asthma symptoms (ACQ-6 score ≥ 1.5). Both studies required evidence of reversibility (post-bronchodilator increase in FEV1 of ≥12% and ≥ 200 mL following salbutamol inhalation), and evidence of airflow obstruction at screening, although this differed between studies at screening (e.g., pre-bronchodilator AM FEV1 < 85% and ≤ 90% predicted in 205715 in 205832, respectively). In 205715, patients were required to meet additional entry criteria at the end of the 3-week run-in period before entering the 2-week stabilization period and at the end of the stabilization period prior to randomization (Supplementary Table 3). In contrast, 205832 had a shorter 2-week run-in phase with patients required to meet additional criteria only prior to randomization (Supplementary Table 2).
Timing of collection of individual patient-reported outcomes (PROs) in both RCTs is summarized in Supplementary Table 4. All PROs used in 205832 and 205715 were administered on an electronic diary which was also programmed to allow collection of periodic PRO assessments during patient visits to study sites. Electronic administration of PROs, such as Asthma Quality of Life Questionnaire (AQLQ) and Asthma Control Questionnaire (ACQ) has demonstrated high levels of agreement with paper versions . In addition, the US Food and Drug Administration (FDA) indicates that the St George’s Respiratory Questionnaire (SGRQ) can be administered electronically . To avoid missing data, patients were required to provide a response before they could move to the next question. Once data were submitted, patients were unable to view their previous responses.
The E-RS: COPD consists of 11 items that measure respiratory symptoms recalled by patients over the previous 24 h, rather than a change in symptoms over this timeframe, capturing information related to breathlessness, cough, sputum production, chest congestion, and chest tightness. Daily recording provides an assessment of underlying day-to-day variability of symptoms. Items 1–8 are scored on a 5-point scale of not at all to extreme, and items 9–11 are scored on a 6-point scale of not at all to too breathless to do these. Thus, the RS-Total score has a range of 0–40, comprising three subscales: RS-Breathlessness (sum of 5 items, range 0–17); RS-Cough and Sputum (sum of 3 items, range 0–11); and RS-Chest Symptoms (sum of 3 items, range 0–12) [6, 17]. For both the 205715 and 205832 studies, the E-RS was administered electronically with exactly the same appearance and format as the paper version administered in Phase 1.
Alongside the E-RS: COPD, two supplemental items were included that were rooted in patient feedback from previously reported qualitative work . First, a wheeze item was included, in which participants were asked “Did you wheeze today?” with response options of not at all, rarely, occasionally, frequently, and almost constantly. Second, a SOB with strenuous physical activity item was included with the question “Were you short of breath today when performing strenuous activities such as climbing stairs, running, or participating in sports activity?” with response options of not at all, slightly, moderately, severely, extremely, and too breathless to do these.
Patient Global Impression of Severity (PGI-S) and Patient Global Impression of Change (PGIC)
The PGI-S is a single-item questionnaire to evaluate disease severity; patients rated asthma symptoms they are currently experiencing at each study visit using a 5-point scale (none, mild, moderate, severe, very severe). The PGIC is a single question used to evaluate response to treatment since the start of the study using a 7-point scale (significantly improved, moderately improved, mildly improved, no change, mildly worse, moderately worse, significantly worse).
St George’s Respiratory Questionnaire (SGRQ)
The SGRQ is a measure of health status in patients with chronic airway obstruction , and includes 50 items addressing three domains: symptoms, activity limitations, and impact. Recall periods in the questionnaire include the past 4 weeks and the current day. A 5-point scale is used for rating symptoms and a true/false binary scale used for activity limitations. The total score is expressed as a percentage of overall impairment, with 0 and 100 representing the best and the worst possible health status, respectively. A reduction of ≥4 points in SGRQ total score is considered clinically meaningful .
Asthma Quality of Life Questionnaire (AQLQ)
The AQLQ includes 32 items that measure functional impairment related to asthma. The questions are designed to be self-completed by the patient, with a recall period of the past 2 weeks. The response scale ranges from 1 (totally impaired) to 7 (not at all impaired). A change of ≥0.5 is considered clinically important .
Asthma Control Questionnaire (ACQ-5, ACQ-6)
The ACQ measures various attributes of asthma control . ACQ-5 includes five questions (nocturnal awakening, waking in the morning, activity limitation, SOB, and wheeze) that gauge the frequency and/or severity of symptoms over the previous week. The ACQ-6 includes an additional item relating to rescue medication use. The recall period is the past week. Response options range from 0 (no impairment/limitation) to 6 (total impairment/limitation). Scores < 0.75 indicate well-controlled asthma whereas scores ≥1.5 indicate poorly controlled asthma . A change of ≥0.5 units is considered clinically important .
Quantitative psychometric analyses used a statistical analysis plan informed by previous validation work on the E-RS: COPD in COPD  and in asthma–COPD overlap (ACO) syndrome . Data were not pooled due to differences in study populations (Supplementary Table 5). This allowed for evaluation of psychometric properties across patients with moderate (205832) and moderate/severe asthma (205715).
Analyses were conducted on blinded data (205832) and interim blinded data (205715) from a PRO dataset, defined as patients included in the intent-to-treat populations with a minimum of 4 days of data for the week prior to baseline, using SAS statistical software version 9.4 (SAS Institute Inc., Cary, NC, USA) or STATA 15 (StataCorp LLC, College Station, TX, USA). All statistical tests were two-sided and used a significance level of 0.05. No imputation of missing data was performed.
Confirmatory factor analysis (CFA) was conducted using structural equation modeling to evaluate the fit of the factor structure of the E-RS: COPD in patients with moderate/severe asthma. The hypothesis that the E-RS: COPD has three factors and second order unidimensionality (ie, that the three factors load onto a single construct) was tested. First, the comparative fit index (CFI) evaluated the proportionate improvement in a model by comparing a hypothesized model against a less restricted baseline model ; values ≥0.9 indicate acceptable fit. Second, the standardized root mean residual (SRMR) measured the mean absolute difference between observed and model-implied correlations; values < 0.1 are considered acceptable . Finally, the root mean square error of approximation (RMSEA) assessed the discrepancy between predicted and observed data per degree of freedom; values < 0.08 are considered acceptable . CFA was estimated at Weeks 0 and 24 in both studies. Post hoc exploratory factor analysis (EFA) determined if there was an optimal factor structure, which includes the E-RS: COPD and the two supplemental items in a meaningful way, and was performed at Weeks 0 and 24 in both studies. In EFA, the structure or number of factors was not pre-specified; scree plots and corresponding eigenvalues were examined to determine the number of factors empirically . The psychometric properties of the tool, referred to as the E-RS: Asthma when used in asthma populations, was then assessed using the best fitting factor structure.
Reliability was assessed using internal consistency reliability and test–retest reliability. Cronbach’s alpha coefficient was estimated for the mean weekly RS-total and subscale scores at Week − 2 and at Weeks 0, 4, 12, and 24 (205832) and at Weeks − 5 and − 2 and at Weeks 0, 4, 12, 24, 36, and 52 (205715). Reproducibility of total and subscale E-RS: Asthma daily and mean weekly scores was assessed to evaluate test–retest reliability, utilizing intra-class correlation (ICC) coefficients with a two-way random effects regression model based on absolute agreement (ICC2,1)  and paired t-tests.
For daily scores, reproducibility of scores (test–retest) over consecutive days (Days 1–2, 2–3, 3–4, 4–5, 5–6, and 7–8) and over a 7-day interval (Days 1–7) from screen run-in indicated patients were stable during this period if randomized. For weekly scores, reproducibility of scores from the first to the second week of screen run-in were assessed. Additionally, reproducibility of scores for patients with no change in ACQ score at the Week − 2 visit (Visit 1 in 205832 and Visit 2 in 205715) to randomization (Visit 2 in 205832 and Visit 3 in 205715) were assessed.
Construct validity of the E-RS: Asthma total and subscale scores were assessed using convergent and discriminant validity and known-groups validity at Week 24. Convergent validity was considered supported if E-RS: Asthma scores showed moderate correlation (r > 0.40) with conceptually similar measures. A lower correlation (r < 0.40) was required to support discriminant validity. Spearman’s rank correlation coefficients were calculated between the mean weekly E-RS: Asthma score and mean weekly rescue medication use, expected peak expiratory flow (AM), expected FEV1 (AM), nighttime awakenings due to asthma symptoms, and asthma symptom severity scores. In addition, the mean weekly E-RS: Asthma score was compared with FEV1, FEV1% predicted, SGRQ (total and domain scores), AQLQ, and ACQ scores from baseline or final visit.
Known-groups validity of the total and subscale E-RS: Asthma daily and mean weekly scores was assessed using analysis of variance (ANOVA) by examining score differences during the baseline week in patients grouped according to FEV1% predicted categories, PGI-S, ACQ score and exacerbation history. The F-statistic from the ANOVA and the t-statistic from the t-tests were considered significant if both were below 0.05.
Analysis of covariance (ANCOVA) models were used to examine differences in change in total and subscale E-RS: Asthma daily scores from baseline week to Weeks 4, 12, and 24, among patients in various responder groups. Baseline scores (mean of Day − 7 to − 1) were controlled for in the models. Cohen’s effect size was calculated for E-RS: Asthma total and subscale scores among patients defined as “responders” using PGIC, SGRQ, or AQLQ.
Anchor and distribution-based methods were used to define a responder threshold for E-RS: Asthma. Using anchor-based methods, mean change in mean weekly E-RS: Asthma total and subscale scores from Weeks 0 to 24 were determined by PGIC level and assessed by plotting cumulative distribution function (CDF) graphs. The minimally important change threshold was the mean change in RS-Total and subscale scores of patients who were mildly improved or mildly worse on the PGIC. Exploratory analyses were conducted utilizing the SGRQ and AQLQ minimally important difference thresholds as anchors (change of 4.0 point and 0.5 points, respectively). Distribution-based methods were conducted as supportive information for development of a responder threshold and included an assessment of the standard error of measurement (SEM) and half standard deviation (SD). SEM was estimated by multiplying the baseline SD of the measure by the square root of one minus its reliability coefficient (ICC from the test–retest assessment) [29, 30]. Half an SD of a measure represents a good approximation of the minimally important difference .
Phase 1: qualitative study
Twenty-five patients (24 English-speaking, 1 Spanish-speaking), identified from five asthma and allergy clinical sites around the USA, participated in concept elicitation interviews. Most participants were female (n = 15; 60%) and Caucasian (n = 21; 84%), with a mean (SD) age of 48 (16.5) years (range 23–83 years). Mean (SD) age of first asthma diagnosis was 21.3 (17.8) years (Table 1).
Concept elicitation interviews
All patients reported multiple respiratory symptoms. Overall, 24 symptoms were described (Fig. 1), with 88% reported within the first seven interviews. Concept saturation was achieved by Interview 15. The most frequently reported symptoms included SOB with and without activity, difficulty breathing, and chest tightness (Fig. 1, Fig. 2).
Mapping of concepts from concept elicitation interviews to items in the E-RS: COPD showed that they were relevant to patients with asthma, and patients’ descriptions were consistent with how symptoms are described in the E-RS: COPD. Symptoms in the E-RS: COPD were also largely consistent with those previously identified during a literature review and clinician interviews (Table 2).
All patients reported that instructions were clear and understandable. Overall, feedback was positive, and there were no reports of any difficulties completing the E-RS: COPD; 24/25 (96%) of patients indicated a clear understanding of the PRO (one patient was not asked). Most patients (19/25; 76%) reported that all items were applicable to their personal experience, and 21/25 (84%) thought the items were relevant to all patients with asthma. No key respiratory symptoms were missing.
Phase 2: quantitative psychometric evaluation
A total of 421 patients were included in 205832 and 2270 patients were included in 205715. Most participants were female (205832: n = 297, 70.7%; 205715: n = 1403, 61.8%), with a mean (SD) age of 48.8 (14.7) years for the 205832 study, and 53.3 (13.1) years for the 205715 study (Table 1). As patients in the 205715 study were diagnosed with moderate/severe asthma, they had worse clinical characteristics compared with patients with moderate asthma in the 205832 study, highlighted by more exacerbations in the prior 12 months (1.4 vs 0.3), and higher baseline ACQ-5 (2.0 vs 1.8), and E-RS: COPD Total scores (8.3 vs 7.2).
CFA supported the fit of the data to the E-RS: COPD factor structure at baseline in 205832 (CFI = 0.872; SRMS = 0.061; RMSEA = 0.194) and in 205715 (CFI = 0.921; SRMR = 0.046; RMSEA = 0.154) (Supplementary Table 6). Findings supported retaining the E-RS: COPD factor structure for the 11 items for use in the moderate/severe asthma patient population. The inclusion of the two supplemental items (wheeze and SOB with strenuous physical activity) did not improve the model and were not included in the final scale. Therefore, the scoring algorithm from the E-RS: COPD was retained for subsequent analyses. The resulting tool is referred to as E-RS: Asthma in this context; subscale names are consistent across both tools.
High Cronbach’s alpha coefficients (> 0.70) were observed for RS-Total score in 205715, ranging from 0.957 at screen run-in, 0.955 at Week 0, 0.954 at Week 24, and 0.951 at Week 52. Alpha coefficients were similarly high for RS subscale scores at all timepoints, including RS-Breathlessness (0.939–0.956), RS-Cough and Sputum (0.837–0.881), and RS-Chest Symptoms (0.935–0.950). Test–retest reliability data indicated stability of RS-Total and subscale scores in both trials, with ICC coefficient values > 0.70 for all assessment timepoints.
Acceptable construct validity was demonstrated. In 205715, at baseline (Week 0), moderate relationships (r > 0.40) were observed between RS-Total score and SGRQ-Total, SGRQ Symptoms and SGRQ Activity domains (range 0.44–0.48, all P < 0.0001). Similar relationships were observed at Week 24, but correlations were higher (range 0.51–0.55, all P < 0.0001). RS-Total and subscale scores showed evidence of convergent validity with asthma symptom severity in both studies, with high correlations at baseline (0.64–0.88) and at Week 24 (0.65–0.86, all P < 0.0001). In support of known-groups validity, when PGI-S was used as an anchor, mean scores on the E-RS: Asthma increased linearly with increasing global severity in both trials (Table 3).
When PGIC was used as an anchor, statistically significant differences were observed between least squares (LS) mean RS-Total and subscale scores from baseline to Week 4, baseline to Week 12, and baseline to Week 24 (all P < 0.0001) (Fig. 3A). Observed effect sizes were moderate at all timepoints (205832: 0.42; 205715: 0.45). When SGRQ was used as an anchor, LS mean RS-Total and subscale scores showed the greatest improvement in the SGRQ change group of <− 4 compared with the other two SGRQ groups, from baseline to Weeks 4, 12, and 24 (all P < 0.0001) (Fig. 3B), with moderate effect sizes observed at all timepoints (205832: 0.44; 205715: 0.48). Lastly, when AQLQ was used as an anchor, patients with AQLQ change ≥0.5 between baseline and Weeks 4, 12, and 24 also showed the greatest reduction in E-RS: Asthma scores (all P < 0.0001) (Fig. 3C), with moderate effect sizes observed at all timepoints (205832: 0.45; 205715: 0.54).
Among patients “mildly improved” on the PGIC, mean (SD) changes of − 1.5 (5.70) and − 1.9 (3.46) were seen for RS-Total score in 205832 and 205715, with respective changes of − 0.8 (2.84) and − 0.9 (1.72), − 0.3 (1.76) and − 0.5 (1.27), and − 0.4 (1.73) and − 0.5 (1.18) for the RS-Breathlessness, RS-Cough and Sputum, and RS-Chest Symptom scores. CDFs are presented in the online supplement (Supplementary Figure 1). Similar changes in RS-Total and subscales were observed when SQRQ and AQLQ were used as anchors. Thus, among patients with a reduction of ≥4 on the SGRQ, mean (SD) changes of − 2.4 (4.74) and − 3.0 (4.31) were seen for RS-Total score in 205832 and 205715, with respective changes of − 1.1 (2.35) and − 1.4 (2.14), − 0.7 (1.59) and − 0.8 (1.39), and − 0.6 (1.45) and − 0.8 (1.39) for the RS-Breathlessness, RS-Cough and Sputum, and RS-Chest Symptom scores. Among patients with a change of ≥0.5 on the AQLQ, mean (SD) changes of − 2.6 (5.11) and − 3.4 (4.46) were seen for RS-Total score in 205832 and 205715, with respective changes of − 1.2 (2.59) and − 1.6 (2.23), − 0.7 (1.58) and − 0.9 (1.45), and − 0.7 (1.52) and − 1.0 (1.45) for the RS-Breathlessness, RS-Cough and Sputum, and RS-Chest Symptom scores. The results of the distribution-based approach for score interpretation, using SEM and half SD values for the RS-Total score ranged from 2.8 to 3.6, with lower scores found for the subscale scores ranging from 1.5 to 1.9 for RS-Breathlessness, 0.8 to 1.1 for RS-Cough and Sputum, and 0.9 to 1.1 for RS-Chest Symptoms (Supplementary Table 7).
Based on the totality of evidence across both 205832 and 205715, the proposed meaningful change thresholds for the E-RS: Asthma are as follows: RS-Total, − 2.0; RS-Breathlessness, − 1.0; RS-Cough and Sputum, − 0.7; and RS-Chest Symptoms, − 0.7.
The EXACT and E-RS: COPD were the first PROs to be qualified by the US FDA under the drug development tools initiative for use in COPD . While these were validated as a daily diary in COPD, their content validity and performance in patients with moderate/severe asthma had not been assessed. A literature review highlighted that the Asthma Daytime Symptom Diary was a potential alternative candidate PRO to assess asthma symptoms, but was at an earlier stage of development .
The FDA has indicated a willingness to consider modified or adapted PROs for use in related disease areas , and earlier studies have shown the E-RS: COPD to be reliable, valid and responsive in a sample of patients with ACO . Therefore, we hypothesized that the E-RS: COPD could be a suitable candidate for use in patients with asthma due to the overlap in symptoms reported in the literature and the use of other tools for the periodic assessment of asthma and COPD [7,8,9].
In this report, a two-phase qualitative and quantitative analysis was used to evaluate whether the E-RS: COPD was suitable for assessing respiratory symptoms in patients with moderate/severe asthma. In Phase 1, qualitative interviews with 25 patients with moderate/severe asthma were conducted. These interviews established content validity; the instrument captured the most relevant and important respiratory symptoms of asthma. Additionally, the items, response options, and recall period were understood, acceptable, and appropriate in this patient population. In Phase 2, the psychometric properties of the tool were evaluated using data from two separate clinical trials (205832  and 205715 ). The findings indicated that the fit of data to the E-RS: COPD factor structure was acceptable and, the same scoring algorithm from the E-RS: COPD was retained for use for all subsequent psychometric analyses in the asthma population (and referred to as E-RS: Asthma). The E-RS: Asthma demonstrated good psychometric properties and was highly responsive in patients with moderate/severe asthma. The proposed responder threshold of ≥ − 2.0 points on the RS-Total score can be used by clinicians to better understand and interpret clinical trial results and may identify responders to treatment.
Although wheeze is a concept that is consistently reported by patients with asthma, it is inconsistently described, challenging to translate, and is interpreted in different ways by patients in different countries. Due to this, and requirements for linguistic validation in different languages, previous studies found that a wheeze item did not load with the 11 items in the E-RS when assessed in COPD or ACO populations [6, 24]. Given differences in the frequency and severity of symptoms in patients with asthma compared to those with COPD, an additional item related to SOB with more strenuous activity was also tested since there is a possibility that symptoms and their impacts can be reversed in asthma such that more strenuous activity is possible. However, this item also did not load with the E-RS items when evaluated in a population with moderate/severe asthma and was also highly correlated with the “SOB on outdoor activities” item in the E-RS: COPD. It was therefore deemed to be redundant. Therefore, the findings from the EFA in relation to the supplemental items included in our analysis were anticipated.
Some limitations should be considered in the interpretation of our findings, primarily in relation to generalizability. First, the mean (SD) age of study populations (qualitative cross-sectional study: 48 ± 16.5 years; 205832: 48.8 ± 14.7 years; 205715: 53.3 ± 13.1 years) may limit the generalizability of the findings to younger patients, particularly adolescents where further qualitative research may be required. However, it should be noted that there was no upper age limit for enrollment in both trials, indicating that our findings are potentially relevant to a broad age range of adults. Second, patients in both phases of the analysis came from clinical studies that studied either moderate or moderate/severe asthma, thus our findings may not be applicable to patients with milder disease. Finally, in the qualitative study, patients were based in the USA, and were predominantly Caucasian, so findings may not be applicable to other countries, cultures, and ethnicities. Whilst Spanish-speaking patients were eligible and substantial effort was made to recruit Spanish-speaking participants, only one such person was interviewed. Nevertheless, the sample size (N = 25) was reasonably large for a qualitative study, and was sufficient to reach saturation; however, a more diverse sample may have captured additional concepts. Additionally, no conclusions regarding use of the tool in Spanish-speaking asthma populations can be drawn on the basis of this study, although substantial testing of the Spanish EXACT translation (14 items which include the E-RS: COPD) has been completed amongst native Spanish-speakers with COPD in the US, Spain, Mexico, Chile, Argentina, Peru, Colombia and Guatemala, indicating that the respiratory terms used are likely applicable to a broad range of Spanish speakers with respiratory illness.
There are a number of strengths to this study. First, we used a step-wise approach to evaluate content validity of an existing measure (E-RS: COPD) for use in a new disease, asthma, in line with FDA guidance  and the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) guidelines . Additionally, the psychometric evaluation study demonstrated support for the psychometric properties of the E-RS: Asthma in two large clinical trial populations with moderate/severe asthma. Similar results were obtained across the two separate studies, providing robust evidence of the properties of the instrument in this patient population and making it applicable to wider use in evaluating asthma patients and their response to treatment.
The E-RS: COPD was found to be valid and reliable for use in this asthma population. The tool will be referred to as E-RS: Asthma when used in asthma populations. The instrument was found to be comprehensive, relevant, acceptable, and understood by patients with moderate/severe asthma. Psychometric analyses support the use of the E-RS: COPD scoring structure in this patient population and the E-RS: Asthma tool showed reliability, validity, and responsiveness, in patients with moderate/severe asthma.
Availability of data and materials
Anonymized individual participant data and study documents can be requested for further research from www.clinicalstudydatarequest.com.
Asthma Control Questionnaire
Asthma Daytime Symptom Diary
Analysis of covariance
Analysis of variance
Asthma Quality of Life Questionnaire
Cumulative distribution function
Confirmatory factor analysis
Comparative fit index
Chronic obstructive pulmonary disease
Consolidated Criteria for Reporting Qualitative Research
Dry powder inhaler
Exploratory factor analysis
- E-RS: Asthma:
Evaluating Respiratory Symptoms in Asthma
- E-RS: COPD:
Evaluating Respiratory Symptoms in Chronic Obstructive Pulmonary Disease
EXAcerbations of Chronic Pulmonary Disease Tool
Food and Drug Administration
- FEV1 :
Forced expiratory volume in 1 s
Forced vital capacity
Institutional Review Board
International Society of Pharmacoeconomics and Outcomes Research
Long-acting muscarinic antagonist
Patient Global Impression of Change
Patient Global Impression of Severity
Randomized controlled trial
Root mean square error of approximation
Standard error of measurement
St George’s Respiratory Questionnaire
Shortness of breath
Standardized root mean residual
Global Initiative for Asthma (2020) Global strategy for asthma management and prevention. https://ginasthma.org/. Accessed May 2020.
Juniper, E. F. (2005). Assessing asthma quality of life: Its role in clinical practice. Breathe, 1(3), 192–204. https://doi.org/10.1183/18106838.0103.192.
Juniper, E. F., Guyatt, G. H., Epstein, R. S., Ferrie, P. J., Jaeschke, R., & Hiller, T. K. (1992). Evaluation of impairment of health related quality of life in asthma: Development of a questionnaire for use in clinical trials. Thorax, 47(2), 76–83. https://doi.org/10.1136/thx.47.2.76.
Pereira, E. D., Cavalcante, A. G., Pereira, E. N., Lucas, P., & Holanda, M. A. (2011). Asthma control and quality of life in patients with moderate or severe asthma. Jornal Brasileiro de Pneumologia, 37(6), 705–711. https://doi.org/10.1590/S1806-37132011000600002.
Leidy, N. K., Wilcox, T. K., Jones, P. W., Roberts, L., Powers, J. H., & Sethi, S. (2011). Standardizing measurement of chronic obstructive pulmonary disease exacerbations. Reliability and validity of a patient-reported diary. American Journal of Respiratory and Critical Care Medicine, 183(3), 323–329. https://doi.org/10.1164/rccm.201005-0762OC.
Leidy, N. K., Murray, L. T., Monz, B. U., … Sethi, S. (2014). Measuring respiratory symptoms of COPD: Performance of the EXACT- respiratory symptoms tool (E-RS) in three clinical trials. Respiratory Research, 15(1), 124. https://doi.org/10.1186/s12931-014-0124-z.
Buist, A. S. (2001). Definitions. In B. P. DJ, S. Rennard, et al. (Eds.), Asthma and COPD textbook; basic mechanisms and clinical management. London: Academic press, Elsevier science ltd.
Buist, A. S. (2003). Similarities and differences between asthma and chronic obstructive pulmonary disease: Treatment and early outcomes. European Respiratory Journal, 39, 30s–35s.
Nelsen, L., Gater, A., Panter, C., Tolley, C., Lee, L., & Pascoe, S. (2017). Understanding and measuring symptoms and health status in asthma COPD overlap: Content validity of the EXACT and SGRQ. Journal of Patient-Reported Outcomes, 2, 18.
Global Initiative for Chronic Obstructive Lung Disease (2021) Global strategy for the diagnosis, management and prevention of chronic obstructive pulmonary disease. https://goldcopd.org/wp-content/uploads/2020/11/GOLD-REPORT-2021-v1.1-25Nov20_WMV.pdf. Accessed 4 February 2021.
Kerwin, E., Pascoe, S., Bailes, Z., … Lee, L. (2020). A phase IIb, randomised, parallel-group study: The efficacy, safety and tolerability of once-daily umeclidinium in patients with asthma receiving inhaled corticosteroids. Respiratory Research, 21(1), 148. https://doi.org/10.1186/s12931-020-01400-5.
Lee, L. A., Bailes, Z., Barnes, N., Boulet, L. P., Edwards, D., Fowler, A., … Pavord, I. D. (2021). Efficacy and safety of once-daily single-inhaler triple therapy (FF/UMEC/VI) versus FF/VI in patients with inadequately controlled asthma (CAPTAIN): A double-blind, randomised, phase 3A trial. The Lancet Respiratory Medicine, 9(1), 69–84. https://doi.org/10.1016/S2213-2600(20)30389-1.
Leidy, N. K., & Vernon, M. (2008). Perspectives on patient-reported outcomes : Content validity and qualitative research in a changing clinical trial environment. Pharmacoeconomics, 26(5), 363–370. https://doi.org/10.2165/00019053-200826050-00002.
Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19(6), 349–357. https://doi.org/10.1093/intqhc/mzm042.
Khusial, R. J., Honkoop, P. J., van der Meer, V., Snoeck-Stroband, J. B., & Sont, J. K. (2020). Validation of online asthma control questionnaire and asthma quality of life questionnaire. ERJ Open Research, 6, 00289–02019.
US Food and Drug Administration (FDA) (2018) Chronic obstructive pulmonary disease: Use of the St. George’s Respiratory Questionnaire as a PRO assessment tool guidance for industry. https://www.fda.gov/files/drugs/published/Chronic-Obstructive-Pulmonary-Disease%2D%2DUse-of-the-St.-George%E2%80%99s-Respiratory-Questionnaire-as-a-PRO-Assessment-Tool-Guidance-for-Industry.pdf. Accessed 16 May 2021.
Leidy, N. K., Sexton, C. C., Jones, P. W., Notte, S. M., Monz, B. U., Nelsen, L., … Sethi, S. (2014). Measuring respiratory symptoms in clinical trials of COPD: Reliability and validity of a daily diary. Thorax, 69(5), 443–449. https://doi.org/10.1136/thoraxjnl-2013-204428.
Jones, P. W., Quirk, F. H., & Baveystock, C. M. (1991). The St George's respiratory questionnaire. Respiratory Medicine, 85(Suppl B), 25–31 discussion 33-27.
Jones, P. W., Quirk, F. H., Baveystock, C. M., & Littlejohns, P. (1992). A self-complete measure of health status for chronic airflow limitation. The St. George's respiratory questionnaire. American Review of Respiratory Disease, 145(6), 1321–1327. https://doi.org/10.1164/ajrccm/145.6.1321.
Juniper, E. F., Guyatt, G. H., Willan, A., & Griffith, L. E. (1994). Determining a minimal important change in a disease-specific quality of life questionnaire. Journal of Clinical Epidemiology, 47(1), 81–87. https://doi.org/10.1016/0895-4356(94)90036-1.
Juniper, E. F., O'Byrne, P. M., Guyatt, G. H., Ferrie, P. J., & King, D. R. (1999). Development and validation of a questionnaire to measure asthma control. European Respiratory Journal, 14(4), 902–907. https://doi.org/10.1034/j.1399-3003.1999.14d29.x.
Juniper, E. F., Bousquet, J., Abetz, L., & Bateman, E. D. (2006). Identifying 'well-controlled' and 'not well-controlled' asthma using the asthma control questionnaire. Respiratory Medicine, 100(4), 616–621. https://doi.org/10.1016/j.rmed.2005.08.012.
Juniper, E. F., Svensson, K., Mörk, A. C., & Ståhl, E. (2005). Measurement properties and interpretation of three shortened versions of the asthma control questionnaire. Respiratory Medicine, 99(5), 553–558. https://doi.org/10.1016/j.rmed.2004.10.008.
Nelsen, L. M., Lee, L. A., Wu, W., Lin, X., Murray, L., Pascoe, S. J., & Leidy, N. K. (2019). Reliability, validity and responsiveness of E-RS:COPD in patients with spirometric asthma-COPD overlap. Respiratory Research, 20(1), 107. https://doi.org/10.1186/s12931-019-1070-6.
Byrne, B. (2011). Structural equation modeling with Mplus: Basic concepts, applications, and programming. New York: Routledge.
Kline, R. B. (2005). Principles and practice of structural equation modeling, (2nd ed., ). New York: The Guilford Press.
Browne, M., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen, & J. S. Long (Eds.), Testing structural equation models. Newbury Park: SAGE Publications.
Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting Intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15(2), 155–163. https://doi.org/10.1016/j.jcm.2016.02.012.
Wyrwich, K. W., Tierney, W. M., & Wolinsky, F. D. (1999). Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in health-related quality of life. Journal of Clinical Epidemiology, 52(9), 861–873. https://doi.org/10.1016/S0895-4356(99)00071-2.
Wyrwich, K. W., Nienaber, N. A., Tierney, W. M., & Wolinsky, F. D. (1999). Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Medicine Care, 37(5), 469–478. https://doi.org/10.1097/00005650-199905000-00006.
Norman, G. R., Sloan, J. A., & Wyrwich, K. W. (2003). Interpretation of changes in health-related quality of life: The remarkable universality of half a standard deviation. Medicine Care, 41(5), 582–592. https://doi.org/10.1097/01.MLR.0000062554.74615.4C.
Leidy, N. K., & Murray, L. T. (2013). Patient-reported outcome (PRO) measures for clinical trials of COPD: The EXACT and E-RS. COPD, 10(3), 393–398. https://doi.org/10.3109/15412555.2013.795423.
Gater, A., Nelsen, L., Fleming, S., Lundy, J. J., Bonner, N., Hall, R., & Patient-Reported Outcome Consortium’s Asthma Working Group (2016). Assessing asthma symptoms in adolescents and adults: Qualitative research supporting development of the asthma daily symptom diary. Value Health, 19(4), 440–450. https://doi.org/10.1016/j.jval.2016.01.007.
US Food and Drug Administration (FDA) (2018) Patient-focused drug development guidance: Methods to identify what is important to patients and select, develop or modify fit-for-purpose clinical outcome assessments. https://www.fda.gov/drugs/news-events-human-drugs/patient-focused-drug-development-guidance-methods-identify-what-important-patients-and-select. Accessed 28 April 2020.
Rothman, M., Burke, L., Erickson, P., Leidy, N. K., Patrick, D. L., & Petrie, C. D. (2009). Use of existing patient-reported outcome (PRO) instruments and their modification: The ISPOR good research practices for evaluating and documenting content validity for the use of existing instruments and their modification PRO task force report. Value Health, 12(8), 1075–1083. https://doi.org/10.1111/j.1524-4733.2009.00603.x.
Editorial support in the form of preparation of the first draft based on input from all authors, and collation and incorporation of author feedback to develop subsequent drafts was provided by Alexandra Berry, PhD, of Fishawack Indicia Ltd., UK, part of Fishawack Health and was funded by GSK. ELLIPTA is owned by or licensed to the GSK group of companies. Evidera holds the copyright to the EXACT®, E-RS®: COPD, and all derivative measures.
This study was funded by GSK (study numbers 206605, 205715, 205832). GSK-affiliated authors had a role in the study design, data analysis, data interpretation, and writing of the report.
Ethics approval and consent to participate
In Phase 1, all recruitment procedures complied with current Health Insurance Portability and Accountability Act (HIPAA) regulations. Evidera obtained ethical approval from the Ethics and Independent Review Services for the study protocol. Prior to the interview, each participant signed an informed consent form. In Phase 2, Institutional Review Board (IRB) approval was obtained to comply with human patients’ research requirements prior to initiation of patient recruitment or administration of measures as part of each clinical trial (205832, 205715). No additional ethics committee or IRB approval was required for this secondary data analysis.
Consent for publication
RvM, ZB, and AF are employees of and hold stock in GlaxoSmithKline (GSK). EDB, TAH, HK, and RH are employees of Evidera. Evidera provides consulting and other research services to pharmaceutical, device, government and non-government organizations. As Evidera employees, they work with a variety of companies and organizations and are expressly prohibited from receiving any payment or honoraria directly from these organizations for services rendered. LTM was a former employee of Evidera. MT and LL were former employees of GSK at the time this work was conducted and holds stocks or shares in GSK.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Laurie Lee, Lindsey T. Murray and Maggie Tabberer at the time of the study.
Cumulative distribution function of RS-Total Change score by PGIC in (A) 205832 and (B) 205715 studies. Table S1. Eligibility criteria for the qualitative study (N = 25). Table S2. Eligibility criteria for GSK 205832 included in the psychometric evaluation. Table S3. Eligibility criteria for GSK 205715 for inclusion in the psychometric evaluation. Table S4. Study visit schedule for PRO collection in 205832 and 205715. Table S5. Summary of clinical trials 205832 and 205715. Table S6. Confirmatory factor analysis of E-RS: COPD at Week 0 in 205832 and 205715 studies. Table S7. Distribution-based approaches for the E-RS: Asthma.
About this article
Cite this article
Tabberer, M., von Maltzahn, R., Bacci, E.D. et al. Measuring respiratory symptoms in moderate/severe asthma: evaluation of a respiratory symptom tool, the E-RS®: COPD in asthma populations. J Patient Rep Outcomes 5, 104 (2021). https://doi.org/10.1186/s41687-021-00338-6
- Patient-reported outcome
- Respiratory symptoms
- Content validity