Measuring the impact of chronic low back pain on everyday functioning: content validity of the Roland Morris disability questionnaire

Abstract

Background

Robust outcome measures are needed to assess and monitor the impact of chronic low back pain (CLBP) on physical functioning. The Roland Morris Disability Questionnaire (RMDQ) is a well-established measure designed to capture the impacts of back pain on everyday functioning, with a particular emphasis on physical functioning. It has documented evaluation of psychometric properties. However, there is no documented qualitative evidence to confirm the content validity of the tool, nor have changes made for electronic administration been debriefed in participants with CLBP.

Methods

In-depth, semi-structured, concept elicitation and cognitive debriefing interviews were conducted with 23 US participants with confirmed CLBP. Interviews allowed participants to describe the impact of CLBP on their day-to-day functioning and discuss comprehension and suitability of the RMDQ. Interviews were transcribed verbatim and analyzed using thematic analysis.

Results

Concept elicitation and cognitive debriefing revealed the substantial burden associated with CLBP, highlighting 15 key areas of functional impact. These were grouped into overarching themes of mobility (walking, stairs, sitting/standing, bending/kneeling, lifting, lying down), activities (chores/housework, dressing, washing, driving, work) and other (relationships/socializing, mood, sleep, appetite), which are consistent with those evaluated within the RMDQ.

All participants found the RMDQ to be relevant with most reporting that the instructions, recall period, and response options were suitable. A few suggested minor changes, however, none were consistent or necessary to support content validity. Updates to the measure for electronic administration and to clarify the response options were well received.

Conclusion

The qualitative data from individuals with CLBP confirmed that the RMDQ has content validity and, alongside documented psychometric evidence, supports the use of the RMDQ as a reliable and valid tool to assess the impact of CLBP on physical functioning.

Background

Chronic low back pain (CLBP) is a common debilitating condition that affects many people worldwide [1]. A 2015 systematic review of studies evaluating CLBP in adults reported that all studies showed an increasing prevalence of CLBP with age; the lowest prevalence rates were in the younger age group (aged 20 to 30 years) and rates increased to peak in those 50 to 60 years of age [1]. CLBP has a substantial burden on both the individual and society, with high levels of disability and physical function impairment, sick leave and work loss, greater health care utilization and treatment costs, and an increased risk of coronary events and other comorbidities all commonly reported [2,3,4,5].

Robust outcome measures are needed to assess and monitor the impact of CLBP on physical functioning. The Roland Morris Disability Questionnaire (RMDQ) [6] is a well-established measure, with documented evaluation of psychometric properties, which is widely used in clinical trials [7,8,9,10,11,12,13,14]. The RMDQ was developed to capture the everyday functional impact of CLBP. Although it does include some broader concepts than might traditionally fall within a strict definition of physical functioning, due to the nature of the condition, it is primarily focussed on physical functioning (mobility, ability to carry out activities of daily living) [6, 15]. The RMDQ is thus acknowledged and used as a measure of physical function in CLBP [16]. It has been shown to have test-retest reliability (intra-class correlation coefficient [ICC] > 0.70) [6, 17,18,19,20] and internal consistency reliability (Cronbach’s alpha > 0.80) [20,21,22]. In research demonstrating construct validity, strongest correlations have been reported with other measures of self-reported disability and health-related quality of life (the Oswestry Disability Index: 0.79–0.80²³ and SF-36: 0.60–0.85) [22] and weakest correlations are reported with objective tests of physical function [23, 24] such as fingertip to floor (r = 0.27) [23], straight leg raise (r = 0.44) [23], and 15 m walk test (r = 0.37) [24]. These correlations reflect the pattern of relationships that would be expected for a measure of self-reported physical function in CLBP. There is strong evidence of ability to detect change on the RMDQ from its use in many CLBP clinical trials with several compounds, including duloxetine, amoxicillin/clavulanic acid, carisoprodol, diclofenac, etoricoxib, glucosamine, hydromorphone, rofecoxib, and tanezumab [7,8,9,10,11,12,13,14]. The Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials (IMMPACT) guidelines include the RMDQ as an example of a disease-specific measure that has been developed to evaluate physical function in CLBP [16].

The measure has also been successfully used in a United States (US) label claim for carisoprodol (Soma®), indicated for the relief of discomfort associated with acute, painful musculoskeletal conditions in adults, as a secondary outcome measure to support improvement in function. However, this label claim approval was given in 2009, before publication of the final Food and Drug Administration (FDA) patient-reported outcome (PRO) guidance [25].

While the RMDQ is a commonly used measure and performs well as a measure of physical functioning in CLBP, a literature review and gap analysis undertaken as part of this project identified a lack of documented content validity evidence. Although the common use of the RMDQ may suggest acceptance within the clinical community of content validity, it is necessary for this to be documented and shown qualitatively through direct feedback from patients to meet current standards of good practice [25,26,27]. No qualitative studies exploring the content validity of the RMDQ in participants with CLBP could be identified.

Content validity is a crucial property of a measure, which shows that all concepts of interest from the patient perspective have been adequately captured in the measure. Establishing content validity is an essential component of demonstrating that a PRO measures the intended concept and is fit for purpose, and this requires obtaining insights and feedback directly from patients through qualitative research to ensure that their voice is appropriately captured. The FDA PRO guidance makes clear that any additional validation builds upon this foundation [25].

The original version of the RMDQ asked participants to mark only those items that are relevant to them and leave those that are not blank. This format created an issue whereby it was not possible to know if “no response” meant that the statement was not applicable to the respondent that day, or if the item had been missed and thus represented missing data. To combat this, the study team made minor modifications to the response options of the RMDQ during migration to electronic format. The dichotomous response option has been made explicit, rather than being implied. In the modified version, respondents are asked to indicate a “yes” or “no” response to each item. In addition, the authors updated the mode of administration from the original paper-based format, to an electronic version for administration via a tablet device. The usability and feasibility of this electronic mode of administration were evaluated in a separate study [28]. Since there is a lack of documented qualitative evidence to support the content validity of the RMDQ or to support the minor modification to the response options, current regulatory acceptance cannot be assumed. Therefore, it is also necessary to debrief the updated version of the RMDQ in qualitative interviews.

The purpose of the current research is therefore to explore the content validity of the RMDQ; to qualitatively explore the patient experience of CLBP to understand what aspects of everyday functioning are most impacted by CLBP; and to explore the relevance, comprehension and patient understanding of instructions, recall period, and response options of the RMDQ.

Methods

Study design and participants

In-depth, semi-structured, face-to-face interviews were conducted. The study was designed and conducted in line with established research practices, including the guidelines provided by the ISPOR taskforce [29] and the Declaration of Helsinki and US 21 Code of Federal Regulations [30].

Inclusion criteria included a clinician confirmed diagnosis of CLBP for ≥3 months without radiation to the posterior thigh. Individuals with any comorbidities or recent surgery or trauma that could affect pain perception or participation in an interview study were excluded; this was on the basis of the recruiting clinician opinion, however examples of other conditions and surgeries to be excluded were given to guide this (Conditions: osteoporosis, rheumatoid arthritis, fibromyalgia, spinal stenosis, CLBP due to visceral disorder, seronegative spondyloarthropathy and neurogenic claudication; Surgeries: discectomy, nerve ablation in the back, kyphoplasty and nucleoplasty). Participants were identified using purposive sampling by a recruitment agency from three locations within the US: Baltimore, Maryland; St Louis, Missouri; and Los Angeles, California. This was to maximize geographic diversity and to allow for a broad spectrum of participants to be recruited for the study. Following identification, participants were invited to attend the clinic to discuss the study and to provide written informed consent. All participants who signed the consent form were enrolled and no participants withdrew from the study. The eligibility of consented participants was confirmed by their clinician who completed a Case Report Form detailing eligibility and medical history for all interested potential participants after informed consent was obtained.

Participants also completed a demographic health information form prior to conducting the interview.

Interviews

The details of those providing informed consent were passed to the researchers and participants were contacted to arrange a mutually convenient time for an interview. All interviews were completed by an experienced qualitative researcher. Interviews were undertaken in a hotel meeting room local to the participants, arranged so that all interviews from a particular site could be conducted within a two-day period. Prior to the interview, the interviewer worked on developing a rapport with the participant, sharing information about who they are, where they work, and the purpose of the interview. All interviews were audio-recorded and then transcribed verbatim and de-identified.

The interviews involved concept elicitation (CE) and cognitive debriefing (CD) utilizing a semi-structured interview guide. Following a general introduction to start, the first part of the interview was the concept elicitation phase in which the participant was asked about their CLBP and its impact on his or her everyday functioning. This included questions such as “Can you describe the symptoms of your back pain, for example, how do the symptoms feel?”, “How often do you experience symptoms?”, and “How does having back pain impact you and your life?” The second phase of the interview was the cognitive debrief of the RMDQ. Participants were asked to complete the modified version of the RMDQ using the updated yes/no response options. Screenshots of the RMDQ from the electronic device were presented to the participant for pen and paper completion. This enabled participants to see how the measure would be presented in electronic format, so allowing any changes to formatting or layout to be seen. After completing the measure, participants were asked for feedback on the instructions, items, recall period, and response options, specifically whether they were understandable, relevant, and comprehensive. The interviewer then asked the participants to review the overall content of the measure to determine whether it covered all relevant functional impacts that are important to them.

Analytical approach

De-identified transcripts were uploaded to ATLAS.ti version 7.0. Coding was undertaken and the coder (second author) met regularly with the interviewer to discuss codes, as well as any non-verbal cues and to make sure any interviewer notes were incorporated. The coding was also reviewed by the project lead (first author), who reviewed the coding for a selection of transcripts and engaged in coding discussions to ensure codes had been applied consistently and accurately and that any coding discrepancies were reconciled via consensus within the research team.

The concept elicitation data were analyzed using inductive thematic analysis [31]. The cognitive debrief section of the interviews was coded to focus on participant input pertaining to the main research questions, including relevance, comprehension, and any rewording suggestions.

Saturation analysis was undertaken, using both spontaneous and probed responses, by dividing the sample into three equal groups, based on the chronological order in which participants were interviewed. Saturation was considered met when no new topics were discussed in the final group of participants, thus it was deemed that no more interviews were necessary. Saturation analysis is only undertaken on concepts identified through concept elicitation, as it is appropriate only when there is an open-ended participant-led discussion during which concepts are being explored (ie, CE interviews).

Ethics

An Ethical Review Board (Copernicus Group Independent Review Board, US; protocol #A4091073) approved the study and study documents. Participants received a small stipend of US $125 for their participation.

Results

A total of 23 individuals (11 males [47.8%], 12 females [52.2%]) with a clinician confirmed diagnosis of CLBP for ≥3 months without radiation to the posterior thigh were interviewed; all based in the US (7 from Baltimore, 7 from St Louis, and 9 from Los Angeles). Participants were aged between 23 and 73 years (mean 53.9 years) and reported mild to very severe pain (3–10 on a 0–10 numeric rating scale; mean 6.5). The mean time since diagnosis was 7.86 years (ranging from 0.4–30.09 years), none had undergone surgery for CLBP, and 22 were currently receiving treatment (one participant did not answer this question on the demographic health information form). Demographic and descriptive data for the participants are presented in Table 1.

Table 1 Additional demographic and health information of participants (N = 23)

Overall, analysis of the qualitative data revealed a substantial burden associated with CLBP. Throughout the interviews, individuals with CLBP discussed a range of functional impacts, particularly focusing on physical functioning, all reported as salient to the individual with CLBP. A conceptual model was developed to represent this (Fig. 1), using language that was used by the participants during the interviews. For ease of illustration and interpretation, these 15 areas were grouped into overarching themes of mobility (walking, stairs, sitting and standing, bending and kneeling, lifting, lying down), activities (chores/housework, dressing, washing, driving, work), and other (relationships and socializing, mood, sleep, appetite).

Fig. 1

Conceptual Model of CLBP from the Participant Perspective. CLBP – chronic low back pain. * These themes were identified as important during the cognitive debrief phase of the interview and were not spontaneously mentioned during concept elicitation

The majority (13 of the 15) physical function impact areas identified as being important to individuals with CLBP were spontaneously discussed in the initial concept elicitation section of the interviews. A further two key areas of impact (lying down and appetite) were first raised during the cognitive debriefing of the RMDQ. Although these concepts were not discussed in depth as part of the CE section, when discussed during CD these were clearly identified by the majority of participants as being highly relevant (23/23 and 21/23 for the lying down and appetite items, respectively), the former being a common way of relieving pain or resting due to back pain, and the latter being affected by severe back pain. Thus, these were felt to be important concepts to capture and therefore these were pulled out as part of the theme development. Thus, all participant-derived qualitative data was utilized, from both sections of the interview, to inform a complete picture of the participant experience of CLBP as presented in the conceptual model.

Saturation analysis was undertaken on the 13 concepts identified during concept elicitation, with no new concepts being identified in the last round of interviews. Therefore, saturation was met.

Table 2 presents each theme from the model with quotes from the interviews that illustrate the importance of the theme to the individuals. As can be seen from the number of participants reporting each theme, there was substantial commonality between individuals in the key areas of function impacted by CLBP.

Table 2 Key qualitative themes

As in traditional PRO development, concepts identified from the interviews were reviewed to identify areas of physical function impairment that should be included in a comprehensive measure for CLBP. Items from the RMDQ were then reviewed to determine the retrofit against identified concepts. This is presented in Table 3. In some instances, there is a direct relationship between the themes identified in the interviews and a single RMDQ item. Reflecting the multi-dimensional nature of CLBP, many themes identified within the interviews represent multiple examples of activities that are impacted by the pain and associated mobility issues or physical function limitations. In these cases, several RMDQ items can be considered as capturing the broad impact of these topics being discussed in the interviews. Table 3 clearly shows that the RMDQ items capture all impacts on everyday physical function that were identified as being important to participants.

Table 3 Mapping the themes identified in coding to the RMDQ items

Although three of the concepts (washing, driving, and work) identified in the conceptual model are not explicitly captured in the RMDQ, it is felt that these are encapsulated within other items or are activities that could be better captured using other more specific measures. For example, work could be impacted by any of the functional limitations experienced in CLBP, depending upon the nature of the job. Difficulties with washing and driving, upon discussion, were both due to functional limitations to bending down and kneeling (item 11), and driving requires the need to change position (item 2). Driving and work were also activities that may not be engaged in daily by all individuals and therefore not appropriate as an item within a measure of everyday functional impact.

During the cognitive debriefing, overall feedback on the RMDQ was positive and supported the content validity of the measure. Details of participants’ feedback are provided in Table 4. All participants reported that it was comprehensive and easy to understand and complete.

Table 4 Item relevance and participants’ suggested edits

When participants discussed items that referred to activities being done ‘more slowly’ [item 3 walking slowly; item 9 dressed more slowly; item 23 go upstairs more slowly] participants not only reported thinking about how they do things more slowly with CLBP compared to before having CLBP, they also spoke about how, on days when their CLBP was worse than usual, they would undertake these activities at a slower pace, meaning they would take longer to complete a task than usual to try to avoid worsening their pain even further.

A few participants had comments, queries, or suggestions for improvements to the RMDQ instructions (n = 4) and recall period (n = 3), but these were not consistent across participants; and others gave positive general feedback on these elements of the measure but raised minor suggestions for improvement during the debriefing (recall period n = 5 and response options n = 8). However, these were not consistent and, upon review, felt to be likely due to the unusual circumstances of one-off completion of the measure during the interview rather than an issue with the measure itself. For example, querying the recall period of “today” as it applied to current one-off use in the interview would not be relevant to the measure as administered in a clinical trial.

Feedback on individual items within the RMDQ suggested that all items were relevant to almost all participants. Only three items were queried for relevance by four participants (two queried one item, the other two items were each queried by one participant). Specific feedback can be seen in Table 4. One participant queried item 1 (staying at home) because their own back pain had never been so severe (they had moderate CLBP); one other participant with moderate CLBP felt that item 9 (getting dressed more slowly the usual) did not fit in with the questionnaire; and two participants (one mild, one moderate CLBP) felt that item 15 (appetite is not very good) was not relevant to them. However, other participants did discuss how they would sometimes not feel like eating because they were in so much pain “I was, um, thinking about well my back hurts tonight I’m not gonna eat” P01–002. Additionally, although they reported the items to be relevant to them, nine participants made suggestions for minor wording edits to several items. However, the suggestions reflected preferences and were not reported across the sample.

Overall there was no consistent feedback that suggested changes were necessary to support the content validity of the tool. Improvements made to the measure in terms of the addition of dichotomous response options were well received and clearly understood. Participants were not explicitly asked to comment on the format of the RMDQ, since screenshots of the electronic tablet were being used, however, no participant raised any concerns with the style and format of the RMDQ during the interview.

Discussion

Qualitative data from individuals with CLBP highlighted the substantial burden of CLBP and its impact on participants lives. The results confirmed that the RMDQ has content validity. Although the common use of the RMDQ may suggest acceptance within the clinical community of content validity, it is an evidentiary expectation, and this should be shown qualitatively through direct feedback from patients. The conceptual model, developed using the qualitative data from this study, illustrates the symptoms and everyday functional impacts associated with CLBP that are salient to those with this condition. This demonstrates the burden of CLBP on both the individual and society, which is supportive of existing research [2,3,4,5].

When the key symptoms and functional impacts highlighted in the conceptual model were mapped to the RMDQ, this highlighted that the RMDQ was comprehensive and all symptoms and impacts identified as important by participants were captured. Although three of the concepts (washing, driving, and work) identified in the conceptual model were not explicitly captured in the RMDQ, upon review of participant quotes it was felt these were either: adequately encapsulated within other RMDQ items (ie, washing and driving); were concepts that not everyone would take part in (ie, driving, work); or were activities that could be better captured using other more specific measures (ie, work). Therefore, it was felt that these were not appropriate to include as specific items within the RMDQ, which is a measure of everyday functional impact.

Two of the concepts captured in the RMDQ (lying down and appetite) were identified by almost all participants as being important during the CD section of the interview (when prompted by the RMDQ item) however they were not raised spontaneously during CE and so the discussion around these was not as in-depth as other concepts. Although it was felt that the feedback from patients supported the importance and relevance of these concepts, it would be interesting to explore these in more detail to clarify their impact in CLBP.

Minor changes that had been made by the authors to the RMDQ for this study (changes to the layout for electronic administration and the addition of explicit dichotomous response options) were well received by participants and added clarity to their responses, allowing a distinction to be drawn between missing and “no” responses. Participants did not raise any concerns with the RMDQ being presented to them as screenshots from an electronic tablet format, where any changes to format and layout from the paper version could clearly be seen. Feedback on the usability of the electronic devices has been evaluated elsewhere [28].

All participants reported that the RMDQ was easy to understand and complete. The RMDQ was considered comprehensive and relevant, capturing all concepts relevant to participants. Although a few participants had minor comments, queries, or suggestions for improvement (for example to wording, items, recall period, or response options), there was no consistent feedback across participants and no changes to the measure were considered necessary within the proposed context of use.

Despite the RMDQ being developed prior to FDA Guidance for Industry on Patient-Reported Outcomes [25], the findings consistently demonstrate that the RMDQ has content validity. This is an essential step in the development of PRO measures [25]. When taken into consideration alongside the results of previous studies evidencing good psychometric properties [14, 17,18,19,20,21,22,23,24, 32], the current study complements this to indicate that the RMDQ is a reliable and valid (including responsive) measure of every day physical functioning in CLBP, suitable for its proposed context of use.

The number of participants involved in this study reflects the typical sample size for in-depth qualitative research. Although participants were all recruited from the US, multiple locations were used to maximize geographical diversity, and although there was a spread of age, gender, education level, pain severity and time since diagnosis, the sample population is predominantly a White and Black/African American non-Hispanic population; other ethnic groups, such as Asians, are not represented. Therefore, it would be beneficial to confirm the cultural representativeness of these findings. Saturation was reached with no new concepts being discussed in the last batch of interviews. It was not feasible to use the electronic tablet-based administration during the interviews, so screenshots of the RMDQ from the electronic device were presented to the participant for pen and paper completion. However, the usability and feasibility of this electronic mode of administration have been evaluated in a separate study [28].

Conclusions

The RMDQ was intended to capture the everyday functional impact of CLBP, which by the nature of the condition, is primarily centered on physical function. This is, therefore, a core aspect of any clinical outcome assessment measurement strategy in the evaluation of CLBP. The findings from the current qualitative research demonstrate that the RMDQ has content validity, reflects concepts experienced by participants with CLBP, and that the changes made to the measure for electronic completion were understood and suitable.