Development and content validity testing of a colonoscopy-specific patient-reported experience measure: the Patient Experience Colonoscopy Scale (PECS)

Abstract

Background

In endoscopic care, favourable patient experiences before, during and after a colonoscopy are essential for the patient’s willingness to repeat the procedure. To ensure that significant experiences are measured, patients should be involved in creating the measurement instruments. Thus, the aim of the present study was to develop a colonoscopy-specific PREM by (1) operationalising patient experiences before, during and after a colonoscopy procedure and (2) evaluating its content validity.

Methods

The colonoscopy-specific PREM was developed in two stages: (1) operationalisation with item generation and (2) content validity testing. A previously developed conceptual model, based on a systematic literature review that illustrates patients’ (n = 245) experiences of undergoing a colonoscopy, formed the theoretical basis. To assess the degree to which the PREM reflected patients’ experiences before, during and after a colonoscopy procedure, content validity was tested—through face validity with healthcare professionals (n = 4) and cognitive interviews with patients (n = 14) having experienced a colonoscopy. Content validity index (CVI) was calculated to investigate the relevance of the items.

Results

The Patient Experience Colonoscopy Scale (PECS) is a colonoscopy-specific PREM consisting of five different constructs: health motivation, discomfort, information, a caring relationship and understanding. Each construct was defined and generated into a pool of items (n = 77). After face-validity assessment with healthcare professionals, a draft 52-item version of the PECS was ready for content validity testing by the patients. During cognitive interviews the patients contributed valuable insights that led to rewording and removal of items. Results from the CVI suggest that the PECS and its content are relevant (I-CVI range 0.5–1, S-CVI/Ave = 0.86). The final PECS consists of 30 items representing a colonoscopy-specific PREM.

Conclusion

The PECS is a new 30-item PREM instrument designed for adult elective colonoscopy patients after they have undergone the procedure. Each item in the PECS derives from a conceptual model based on a systematic literature review. Patients and healthcare professionals were involved in developing the PECS, which measures colonoscopy-specific patient experiences before, during and after the procedure. The content validity testing positively contributed to the development of the PECS. Psychometric properties need to be evaluated further.

Background

Colonoscopy is a fundamental procedure for diagnosis and treatment of clinical disorders of the lower gastrointestinal tract as well as for colorectal cancer screening [1,2,3,4]. However, patients who undergo the procedure may experience anxiety and may find themselves in an exposed and/or awkward situation and be embarrassed during the colonoscopy [5,6,7,8,9]. These negative experiences can be alleviated if the patients’ individual needs are met by the healthcare professionals, as such care promotes positive experiences [8, 10]. Which in turn pave the way for patients’ acceptance of and willingness to repeat the procedure [11,12,13,14].

Positive patient experience is associated with good care quality [15,16,17,18], and it is thus important for the healthcare to give the highest quality of care as well as gaining knowledge about how the patients experienced the colonoscopy procedure [19]. The patient perspective can be incorporated into care evaluation by collecting patient-reported experience measures (PREMs) through clinically relevant questions that matter to the patients [20]. This strongly aligns with the European Society of Gastrointestinal Endoscopy (ESGE) quality improvement initiative which recommends that patient experiences should be self-reported and measured routinely [21]. The PREMs should assess how the patients have experienced the provided care but also, and most importantly, provide support in identifying areas in need of improvement [22]. However, healthcare often fails to collect the measures that are most meaningful to patients [23] and that capture the essence of patient involvement when developing adequate measures [24].

There is no standard approach to measuring colonoscopy-specific patient experiences [21] and existing measures rarely report patient involvement during the development process [25]. Lack of patient involvement may lead to the value of the PREMs for the patients being questioned [25], since they, though experts of their own experiences [20], have not been a part of the development process [26].

Adult patients’ experiences of undergoing a colonoscopy was reported, in a systematic review, as a conceptual model developed by synthesising data from thirteen qualitative studies [27]. The conceptual model’s five main concepts were compared with eight existing multidimensional colonoscopy-specific PREMs, and the result showed that none of the identified PREMs fully covered the conceptual model [27]. These findings support and strengthen the argument for patient involvement during the development process of a colonoscopy-specific PREM.

Methods

The aim of the present study was to develop a colonoscopy-specific PREM by (1) operationalising patient experiences before, during and after a colonoscopy procedure and (2) evaluating its content validity.

This is a psychometric study with both a quantitative and a qualitative design where the instrument in question was developed and validated in a two-stage process: Stage 1—operationalisation of the conceptual model including item generation, and Stage 2—evaluation of its content validity. The purpose of a new colonoscopy-specific PREM is to assess the quality of the care provided before, during and after colonoscopies by identifying potential areas of improvement. The target population for the instrument is adult patients who have undergone an elective colonoscopy.

Operationalisation and item generation

The process of instrument development was inspired by Wolfe and Smith [28]. A conceptual model, based on a systematic review describing adult patients’ experiences before, during and after a colonoscopy procedure, formed the theoretical basis for the instrument [27]. In this study, colonoscopy-specific patient experiences are presented in five different constructs—Health motivation, Discomfort, Information, A caring relationship and Understanding—which correspond to the conceptual model [27].

Colonoscopy-specific patient experiences are abstract by nature and cannot be directly observed as a measure [29]. Due to this, an operationalisation process was required to identify the intended meaning of each construct and transform them into empirical observations [28]. The process started with definitions of the constructs, followed by specifications of the different domains in each construct. Subsequently, observable aspects of the domains were identified as indicators aiming to reflect how the domains might be directly observed [28] and answered by questionnaire respondents (Table 1). This was done in an iterative process, where the research group, during recurrent workshops, discussed the indicators until consensus was reached that each of them could be derived back to the conceptual model [27] and that they were related to clinical practice as well as to the target population.

Table 1 An overview of the constructs regarding definitions, domains, and indicators

Out of all indicators, a pool of items was generated. The items were expressed as statements where complex sentences and difficult wordings were avoided. The answers should indicate the respondents’ level of agreement, based on their experiences of undergoing a colonoscopy, on a bipolar ordinal Likert scale with four response categories, from strongly agree to strongly disagree [30]. Mostly, high levels of agreement indicate a favourable experience, whilst low levels of agreement suggest a less positive experience. However, some items are reversed, meaning that high levels of agreement suggest a less positive experience and vice versa.

Content validity

To assess the degree to which the items complied with the patients’ experiences, content validity was performed in accordance with COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria, i.e., relevance, comprehensiveness and comprehensibility [31]. The content validity was tested through face validity with healthcare professionals and cognitive interviews with patients. In addition, the content validity index was calculated among patients to investigate the relevance of the items [32].

Face validity

Healthcare professionals at one endoscopy unit at a university hospital in the southern part of Sweden, with more than three years’ experience of endoscopic care, were invited to participate in the study during May 2022. Four healthcare professionals (two registered nurses and two endoscopists with a range of experience in endoscopic care from 5 to 23 years) accepted and were individually interviewed by the first author (AR). Two interviews were performed face to face, one was conducted via a digital platform and the last one by telephone. All interviews took place in a quiet room and the healthcare professionals were undisturbed. They were asked about the items’ clinical relevance and their wording, and about response categories and if they believed that the items were an adequate reflection of how they perceived patients’ experiences before, during and after a colonoscopy [33]. Additionally, they were asked if any key aspects were missing. The healthcare professionals’ reflections were noted and discussed by the research group before a first draft of the instrument was created for the cognitive interviews.

Cognitive interviews

To encourage the patients to reveal their detailed thoughts of the items’ meaning, cognitive interviews were used [34]. The data collection took place between May and November 2022 and was conducted at one endoscopy unit at a university hospital in the southern part of Sweden. The sample was prospective since all patients who met the inclusion criteria during three specified data collection periods, were invited. Patients who fulfilled the inclusion criteria of being adult (>18 years), Swedish speaking, able to participate in an interview and scheduled for an elective colonoscopy, received an invitation letter 2–4 weeks prior to the procedure. Out of 42 invited patients, 14 accepted study participation, 14 declined and 14 were either cancelled or re-scheduled for a colonoscopy and therefore did not match the inclusion criteria. The patients who accepted study participation were contacted by the first author (AR) within a week after their procedure for scheduling an individual interview (Table 2).

Table 2 Characteristics of patient participants (n = 14)

They could choose the location for the interview, resulting in interviews via telephone (n = 5), via a digital platform (n = 4), at Malmö University (n = 4) and in the home of one patient (n = 1). During the cognitive interviews all patients were undisturbed and had the instrument at hand. Before the interviews began, the patients were introduced to the think-aloud method and instructed to verbalise their thought process while reading each item [35]. Patients were encouraged to give their view of the items rather than relay their experience of the colonoscopy. Nonetheless, for the PREM’s comprehensiveness, it was emphasised that the patients should highlight if any key aspect from their own colonoscopy experience was missing. For clarity, the patients were coached during the interviews, by means of concurrent probing questions, to further explain their reasoning [34]. The COSMIN criteria for content validity, regarding relevance, comprehensiveness and comprehensibility, were used [31] as a guiding principle for an interview guide during the interviews. The patients underwent the cognitive interviews within 21 days after their colonoscopy and the interviews lasted approximately 46 minutes (range in time 17–89 minutes). All interviews were audio recorded, transcribed verbatim and read individually by all authors in the research group, followed by joint consensus discussions. Interviews were conducted in four subsequent rounds, and after each round, adjustments, such as rewording and revision of the items, were agreed on, based on identified problems. After three rounds of interviews, no new data regarding the items were added. However, to deepen the understanding of comprehensibility regarding the response options, four of the patients who were available took part in a second interview. After the last round, no new issues needed to be addressed.

Content validity index

To determine the content validity index (CVI), six of the included patients were asked to assess each item for relevance on a scale ranging from 1 = not relevant to 2 = somewhat relevant, 3 = quite relevant and 4 = highly relevant [32]. A content validity index of individual items (I-CVI) below 0.78 was considered less relevant [36]. During consensus discussions, I-CVI values < 0.78 were weighted against the theoretical basis in the conceptual model to decide possible item removal. In addition, the average proportion of items that achieved a rating of 3 or 4 was calculated as S-CVI/Ave, and a value above 0.9 was considered excellent [32].

Results

The developed PREM instrument was named the Patient Experience Colonoscopy Scale (PECS) and consists of five constructs and 25 domains: Health motivation (n = 3), Discomfort (n = 10), Information (n = 6), A caring relationship (n = 3) and Understanding (n = 3), as shown in Table 1.

Operationalisation and item generation

During operationalisation, the colonoscopy-specific indicators (n = 75) were generated into 77 items. As an example, the indicator ‘Responsive to individual needs, easing anxiety’ was generated into an item as follows: ‘I felt that my need for sedation/pain medication was met at the Endoscopy Unit’. During workshops, the research group processed all items. When similar items arose, they were either modified or merged and item reduction was thereby achieved. This was, for example, relevant when items about feeling safe were present multiple times. During this process, 22 items were removed due to conceptual ambiguity. Furthermore, all three items concerning the domain Logistical planning were removed after consensus discussions in the research group, since the logistical issues that patients might experience with, for instance, transportation back and forth to the hospital, were perceived as out of the healthcare control area. This resulted in a draft 52-item instrument being developed for use in content validity testing.

Content validity

Healthcare professionals and patients were involved in the content validity development process (Fig. 1).

Fig. 1

PREM development process, Stage 2—content validity

Face validity

During the interviews, the healthcare professionals stressed the importance of asking patients about their understanding of the given information, since adequate bowel preparation is clinically relevant and affects adenoma detection. They also stated that not all patients experience discomfort and/or pain during a colonoscopy and that sedated patients might suffer amnesia, which could cause uncertainties when answering items regarding information after the procedure. These insights led to the inclusion of several dichotomous screening items. In addition, feedback was given regarding negatively worded items, which led to positive rewording. The healthcare professionals highlighted problems with items being too close to one another, potentially measuring the same aspect of a domain, and therefore perhaps being difficult for patients to distinguish between when answering. The research group decided to keep the identified items regardless of this feedback, aiming to get additional feedback from the patients during the cognitive interviews. Furthermore, all healthcare professionals emphasised that the instrument consisted of too many items and that this might make it a challenge to use in clinical practice. Lastly, they thought that the items included seemed to reflect the constructs.

Cognitive interviews

The interviews with the patients were conducted in four rounds, and after each round, issues problematised by the patients were addressed, and revisions, such as rewording and removal of items, were made, prompted by the results (Table 3). Most items were considered relevant for the construct of interest. However, some items were considered less relevant and therefore removed.

Table 3 Examples of item revision based on cognitive interviews, rounds 1 to 4

It was also expressed that items that concerned the construct Discomfort and its domain Dietary guidelines were relevant, which was explained by the fact that several patients wished to give the staff quality improvement proposals regarding the information they had received. In addition, in the colonoscopy context, both conscious and deep sedation are common, although some patients choose to be awake during the procedure. This reality made it difficult for the patients to know what answer to choose regarding items that concerned the construct Information. Three out of five patients in the first round did not actually remember if they had been awake or asleep during the procedure, and this gave rise to the decision to keep items from the domain Sharing of information while the only item regarding the domain Understanding information was deleted. Nonetheless, none of the patients had difficulties recalling their overall experiences of the colonoscopy procedure.

All patients were asked at the end of the interview if they missed important aspects of a colonoscopy experience in the instrument and none of them thought that any key aspects were missing. With few exceptions, patients appeared to clearly understand the items as intended. However, simplifying rewordings were recommended by the patients, and the PREM introduction and the text were revised twice, according to those recommendations.

The patients were asked to share their thoughts about the response options. Most of them (n = 11) endorsed the response options and stressed that four alternatives were enough and that being ‘forced’ to ‘take a stand’ (positive/negative) was a good thing. However, two patients would have preferred dichotomous response options (yes/no), while one patient would have liked a response option in the middle (neutral) that would have enabled having no opinion. One patient suggested the use of numbers instead of words (1 to 4) for the response options, and another patient suggested simplifying the response options by changing strongly agree to just agree. Apart from these views, the response options were understood by all the patients as intended and therefore left without revision.

Content validity index

I-CVI values ranged from 0.33 to 1.00, and the S-CVI/Ave was 0.82. Nine items had an I-CVI below 0.78, and they were discussed in the research group, which resulted in keeping five of them due to their contribution to the conceptual comprehensiveness (Table 4). After the deletion of four items, the final PREM had I-CVI values that ranged from 0.50 to 1.00, and the S-CVI/Ave was 0.86.

Table 4 Overview of items whose I-CVI was < 0.78

After content validity testing, the PECS comprises 30 items forming five subscales which correspond to five constructs. The items are distributed over the time periods before, during and after a colonoscopy procedure (Table 5).

Table 5 Item distribution regarding constructs and time periods

Discussion

In the current study, the PECS was developed through the operationalisation of patient experiences before, during and after a colonoscopy procedure. The PECS measures colonoscopy-specific patient experiences in an adult population after an elective procedure, and its content validity was tested according to COSMIN guidelines [31] where both patients and healthcare professionals were involved. The PECS is tentatively multidimensional and consists of 30 items, each of them derived from a conceptual model which describes and depicts how adult patients experience undergoing a colonoscopy [27].

The operationalisation of the colonoscopy-specific indicators resulted in a 52-item draft version of the PECS. This version had a clear theoretical basis, since the conceptual model from the systematic literature review included 13 qualitative research articles reporting how adult patients (n = 245) experienced undergoing a colonoscopy procedure [27]. This demonstrable connection to a conceptual model ensures that the scale is based on patients’ experiences and not on what healthcare professionals believe patients are experiencing, which is an important factor for capturing experiences that matter to the patients [20, 25]. A similar scale, the Newcastle ENDOPREM™, which aims to assess endoscopic patient experiences, apart from colorectal cancer screening [37], was developed using COSMIN guidelines [38] and cognitive interviews. However, for that scale the concept elicitation was based on interviews with only 10 patients who had undergone a colonoscopy [7], which may be considered a restricted theoretical approach [39]. In addition, the target population differs between the Newcastle ENDOPREM™ and the PECS, the latter being uniquely a colonoscopy-specific PREM intended for all adult patients that need to undergo the procedure.

Healthcare professionals’ concerns about the patients’ time to fill out a questionnaire, have been reported as a limitation for PREM usage [40]. The length of a questionnaire has an impact on response rates, where shorter is preferable [41]. In this study, the healthcare professionals gave valuable insights during face validity assessment when they highlighted the PECS as being too extensive with too many items for routine clinical use. Their comments prompted the balancing act of retaining a comprehensive instrument, where all constructs were represented, while developing a clinically useful instrument, where item reduction did not entail omitting any key aspects. This resulted in a 57-item version of the PECS that was due for further content validity testing with patients involved.

The cognitive interviews provided valuable insights into the respondents’ interpretation and comprehension of the items, and they confirmed that the statements reflected the constructs and domains as intended. However, during the four rounds of interviews some rewording was needed. For example, a statement that had to be reworded was the one regarding the impact on sleep (Table 1), an item that derived from patients describing their lack of sleep due to constant toilet visits caused by laxative during the night before the colonoscopy [9]. Not getting enough sleep during and after the bowel preparation causes exhaustion and tiredness [8, 13]. Nevertheless, the patients considered the first version of the item to be open to interpretation since sleep could be either positively or negatively affected, even though they assumed that the item most likely referred to sleep being affected in a negative way. This understanding resulted in a rewording of the item, where ‘not affected’ was changed to ‘negatively affected’. However, in the next two rounds, patients still expressed ambiguities, as sleep difficulties can be caused by different factors, such as bad dreams or, as reported by McEntire et al., by experienced anxiety prior to the colonoscopy due to fear of pain during the procedure or fear of the impending result of the examination [10]. After discussion within the research group, it was decided that the reworded item should be retained pending the response from the content validity index calculation. However, these insights emphasise the importance of cognitive interviews in uncovering the target population’s understanding of the items. Through the systematic capturing of the cognitive processes of the respondents, potential pitfalls that could compromise the content validity of the PECS were identified and rectified [42]. This approach not only contributed to the methodological rigour of the colonoscopy-specific PREM development but also ensured that the patients’ perspective was captured [43] and that the cognitive interviews worked as intended [42].

Individuals undergoing a colonoscopy are heterogeneous, making it challenging to generate items relevant to the entire target population. The result showed that nine items had unsatisfactory I-CVI (<0.78), and consequently four of those items were removed. An example of an item with unsatisfactory I-CVI was the statement regarding interest in watching the TV monitor during the colonoscopy (Table 4), where some patients found the item highly relevant while others considered it irrelevant, due to being asleep or sedated, or simply did not wish to see their intestines. Regardless, previous results show that unsedated patients experienced less pain and anxiety if they received detailed information while they watched the TV monitor during their colonoscopy procedure [44]. It is, arguably, challenging to find varied and relevant items that suit the whole population [45], and the relevance of this specific item was discussed both by the patients and in the research group throughout the whole development process. The I-CVI was 0.33, a result that played a decisive role when the research group finally decided to remove the item. However, not all items below 0.78 were deleted, because of their contribution to the conceptual comprehensiveness. Accordingly, the decision to retain five of the items with I-CVI values below the suggested value, in turn resulted in a S-CVI/Ave value slightly lower (0.86) than the recommended 0.9 [32].

Strengths and limitations

The present study has, through a thorough theoretical approach, enabled the development of a colonoscopy-specific PREM. This was made possible by the process of operationalisation, including the identification of indicators that laid the foundation for item generation. Nonetheless, patient experience is a multidimensional construct [15] and when trying to fit reality into a specific construct, there is a risk that the theoretical underpinnings do not correspond entirely to the real world. Consequently, usage of the COSMIN methodology for content validity [31] provided conditions for testing if the content of the PECS was ‘an adequate reflection of the construct to be measured’ [33]. While researchers could be considered to be experts on theoretical concepts and on the operationalisation process to generate items, patients who have undergone a colonoscopy are the true experts of their own experiences. When these patients are involved in cognitive interviews, they can evaluate how the theoretical operationalisation process corresponds to reality and also suggest solutions for potential difficulties and ambiguities [42]. Moreover, for the purpose of revising the colonoscopy-specific PREM, between the four rounds of cognitive interviews, the whole research group participated in analysing the data, which may be considered a strength due to the researchers’ varied clinical competence and research experiences. Furthermore, in this study, the recruited patients were diverse concerning indication, colonoscopy experiences, sedation and bowel preparation, which is a strength since a variety of different patients’ perspectives were considered. Apart from the increased possibility to identify problems, this variety of patients also enhances the transferability of the PECS, in that it can be used in different settings.

The current study was carried out with a relatively small sample, which is common in qualitative research; hence the intention of the study is to confirm how the target population understands the items and not to generalise the results [23]. Furthermore, in the quantitative part of this study, the cut-off values (I-CVI > 0.78 and S-CVI/Ave > 0.90) suggested by Lynn [36] and recommended by Polit and Beck [32], were used, in order to minimise the element of chance and calculate the real agreement among the six raters. Even so, having six raters rather than fewer allows for a more diverse range of perspectives to be considered, which may contribute to a more consistent and accurate relevance assessment [46]. Moreover, one purpose of cognitive interviewing is to verbalise the participants’ thought processes [42], which means that this method assumes that the target population is able to provide such verbal reports [47]. However, it has been highlighted that not all cognitive processes can be verbalized [48]. In its current form, the PECS is only applicable in a Swedish colonoscopy context, due to the language, and the scale needs to be further tested for internal consistency [49] to confirm whether the number of items is appropriate. In addition, more advanced psychometric approaches, e.g., the Rasch measurement theory [50], are needed to further evaluate the PECS. Lastly, validity is a complex concept which may be examined from different perspectives and future studies can advantageously be designed according to modern validity theory when examine the PECS further [51, 52].

Conclusion

Patient experiences are essential for healthcare quality and useful in evaluating provided care and identifying potential areas of improvement. Colonoscopy-specific patients’ experiences can be captured by a 30-item PREM named the PECS, which may, through its solid theoretical underpinnings, be a valuable addition to the endoscopic care and to future quality improvement initiatives. Both patients who have undergone a colonoscopy procedure and healthcare professionals have been involved in the development of the PECS, and it seems to contain key aspects of importance and be understood by the target population as intended, as well as consisting of items relevant to the constructs being measured. However, its psychometric properties need to be evaluated further.