Skip to main content

Evaluation of the content validity of patient-reported outcome (PRO) instruments developed for use with individuals with phakic presbyopia, including the Near Activity Visual Questionnaire-presbyopia (NAVQ-P) and the near vision correction independence (NVCI) instrument

Abstract

Background

Presbyopia is the age-related deterioration in the ability to focus on close objects. In order to develop a patient-reported outcome (PRO) instrument to assess near vision functioning, the Near Activity Visual Questionnaire (NAVQ) was adapted to incorporate modern technology (e.g. smartphones) and to be appropriate for use in phakic presbyopia, leading to the development of the NAVQ-Presbyopia (NAVQ-P). Additional single-item instruments of near vision correction independence (NVCI), correction preference (NVCP), and vision satisfaction (NVS) were also developed. The study aimed to evaluate the content validity of the NAVQ-P and additional instruments in individuals with phakic presbyopia.

Methods

Participants in the US (n = 15), Germany (n = 10) and France (n = 10) took part in face-to-face, qualitative, cognitive debriefing interviews. Seven healthcare professionals (HCPs) were also interviewed to assess the clinical relevance of the PRO instruments. Interviews started with open-ended qualitative concept elicitation questioning; participants then completed the PRO instruments on an electronic tablet using a “think-aloud” process and were asked about their understanding and relevance of each item, instruction, response scale and recall period. Interviews were conducted in two rounds allowing for modifications between rounds.

Results

The participants interpreted the majority of the PRO instruments and recall period correctly and consistently. They were able to select an appropriate response option without difficulty. Minor modifications were made to the PRO instruments based on interview findings. Instruction/item wording was modified to include reference to use of a magnifying glass, in addition to glasses and contact lenses. Two items were added to assess difficulty with precision tasks (e.g. sewing) and taking longer to adjust from distance to near vision. HCPs confirmed the relevance of the concepts being measured for presbyopia and recommended the addition of an item assessing contrast sensitivity.

Conclusions

Developed in accordance with the FDA PRO Guidance, the findings support content validity of the NAVQ-P as a suitable, well-understood instrument of relevant near vision functioning concepts in individuals with phakic presbyopia. The NVCI and additional PRO instruments are appropriate to assess near vision correction independence, correction preference, and vision satisfaction. Future work will assess the psychometric properties of the NAVQ-P and additional PRO instruments.

Introduction

Presbyopia occurs when the physiologically normal age-related reduction in the eye's focusing range reaches a point, when optimally corrected for distance vision, that the clarity of vision at near is insufficient to satisfy an individual's requirements [1, 2]. It is hypothesized to be caused by a loss of lens elasticity preventing focal point change [3, 4]. While the etiology of this condition is not fully elucidated, recent research suggests that an increase in lens rigidity is the primary causative mechanism [5, 6]. Presbyopia is expected to be experienced in about 80% of people aged 40 years or above [1]. Individuals with presbyopia have difficulty with near vision function tasks (e.g. reading or threading a needle [7, 8]) and experience burden associated with wearing glasses [9]. Consequently, presbyopia has a significant impact on individuals’ health related quality of life [7, 8, 10,11,12,13,14] and entails substantial humanistic and economic burden [15].

Existing clinical assessment tools (such as visual acuity assessment through use of a standardised Snellen chart) lack adequate assessment of the individual experience of presbyopia, highlighting the need for patient-reported outcome (PRO) instruments in this specific population. A recent literature review found there was a paucity of PROs developed for use in phakic presbyopia in line with the Food and Drug Administration (FDA) PRO guidance [16]. Phakic presbyopia is presbyopia that occurs for an individual who still has a natural lens, as opposed to pseudophakic presbyopia where the individual no longer has a natural lens (such as following surgery). There are a number of treatment considerations when managing pseudophakic presbyopia in comparison to phakic presbyopia such as navigating corneal scars and residual corneal irregularities from prior incisions.{Paley [17]} Instruments such as the Near Activity Visual Questionnaire (NAVQ) [18], National Eye Institute Visual Function Questionnaire (NEI VFQ-25) [19], National Eye Institute Refractive Error Quality of Life Instrument-42 (NEI RQL-42) [20] were among those identified as PRO instruments that assess vision outcomes of presbyopia or similar conditions. The FDA guidance outlines the requirement of evidence of content validity in a given context of use for existing, modified, or newly created PRO instruments used to support claims in approved medical product labelling [16, 21]. The NAVQ was the only PRO identified by the literature review as a suitable instrument to assess patient-reported near vision function [16, 18]. The other identified instruments had limitations such as lack of focus on presbyopia and insufficient evidence to support the psychometric properties.

The NAVQ was originally developed and validated in a population which included individuals with pseudophakic presbyopia [18]. Although the NAVQ was developed in line with the FDA guidance, modifications were required to ensure the NAVQ was suitable for use in clinical trials with a purely phakic population, and that the instrument assessed difficulties experienced with the use of modern information technology devices due to near-vision problems (smartphones, computers, and tablet devices which were not as widely used when the NAVQ was first developed). To inform modifications to the NAVQ, first an update of the literature review and a social media listening study was conducted to explore the lived experience of presbyopia [16, 22]. This information was used to generate a preliminary conceptual model to summarize the key symptoms and impacts in presbyopia.

Based on the preliminary conceptual model, the NAVQ was adapted for use with individuals with phakic presbyopia and the resulting instrument is called the NAVQ-Presbyopia (NAVQ-P). Additional single-item instruments for the assessment of near vision correction independence (NVCI), near vision correction preference (NVCP) and near vision satisfaction (NVS) were also developed, along with two global items to assess patient global impression of severity of near vision function (PGIS-Presbyopia) and patient global impression of change in near vision function (PGIC-Presbyopia). All instruments were designed for electronic completion using a tablet device. Cognitive debriefing (CD) of the electronic NAVQ-P, NVCI and additional instruments was required to evaluate content validity of the instruments in individuals with phakic presbyopia.

The aim of the study was to evaluate the content validity of the NAVQ-P, NVCI and the additional instruments (NVCI, NVCP, NVS, PGIS-Presbyopia and PGIC-Presbyopia) in individuals with phakic presbyopia through the conduct of qualitative research, with the ultimate objective to develop PRO instruments suitable for use as clinical trial endpoints in phakic presbyopia.

Methods

Study design

This was a non-interventional, qualitative study involving participants with phakic presbyopia and healthcare professionals (HCPs). In-depth, face-to-face, semi-structured interviews were conducted with thirty-five individuals with phakic presbyopia to assess the content validity of the PRO instruments. Seven additional interviews were conducted with HCPs to obtain a clinical perspective on the relevance of the items included within the PRO instruments.

There were five stages to the research as outlined in Fig. 1. A purposive sampling approach involved enrolment of individuals with phakic presbyopia in the US (n = 15), France (n = 10) and Germany (n = 10), and HCP representation from the US (n = 3), France (n = 2), Japan (n = 1), and Germany (n = 1). These countries were selected to provide a representation from the US and Europe, and a HCP from Japan was interviewed to explore if there were any apparent differences in Asia. Both samples participated in combined concept elicitation (CE) and CD interviews. Findings from the CE section of the interviews (stage 2) have been published separately and are therefore not described in this article [23].

Fig. 1
figure1

Study methodology. *CE concept elicitation, CD cognitive debriefing, HCP healthcare professional, FDA US Food and Drug Administration, PRO patient-reported outcome, NAVQ-P Near Activity Visual Questionnaire for Presbyopia, NVCI Near Vision Correction Independence, NVCP Near Vision Correction Preference, NVS Near Vision Satisfaction, PGIS Patient Global Impression of Severity, PGIC Patient Global Impression of Change

CD interviews were conducted across two rounds with individuals with phakic presbyopia (round 1/stage 3: n = 17; round 2/stage 5: n = 18) and HCPs (round 1/stage 1: n = 3; round 2/stage 5: n = 4). Conducting the interviews in rounds enabled iterative modifications to the NAVQ-P, NVCI and additional instruments to be tested in a subsequent round of interviews (Fig. 1). Expert clinical input to study design, findings and instrument modification was provided by the developer of the original NAVQ [18]. Ethical approval was obtained in accordance with requirements per study country (Additional file 1).

Study sample

Partner recruitment agencies worked with ophthalmologists/optometrists to recruit individuals with phakic presbyopia who met the inclusion and exclusion criteria (Additional file 1). Referring ophthalmologists/optometrists confirmed the participant’s eligibility by completing a Case Report Form (CRF) and ensured written informed consent was obtained using an Information and Consent Form (ICF) prior to any other study activities and prior to any personal data being shared (Additional file 1). The recruitment agencies collected demographic information using the demographic form and participants were remunerated for taking part. HCPs (N = 7) were identified based on their area of expertise and contribution to the field. HCPs (ophthalmologists or optometrists) were recruited from the US, Germany, France, and Japan.

Cognitive debriefing interview procedure

The aim of the CD interviews with individuals with phakic presbyopia was to assess relevance and understanding of item wording, instructions, recall period and response options of the PRO instruments and the usability of the electronic PRO (ePRO) tablet device (Samsung Galaxy Tab E). Given seeing small text on a digital screen was identified in the previous literature as an impact of presbyopia, the ePRO was developed with this in mind (Roboto font size 23) and participants were asked specifically about whether they found the font size easy to read. Interview administration was not considered given the design of the ePRO was developed to be easy for individuals with phakic presbyopia to read.

The interviews were conducted by trained, experienced interviewers using a semi-structured interview guide (Additional files 2 and 3). Minor updates were made to the interview guide between rounds of interviews to correspond with modifications made to the PRO instruments. Additional probes were also added to the interview guide following round 1 (stage 3) interviews to explore any differences between participants who had comorbid myopia and those who did not. Figure 1 outlines the instrument version debriefed at each stage of the study. The study team sought feedback from the US Food and Drug Administration (FDA) between round 1 and round 2 interviews (stage 4).

A ‘think-aloud’ process was employed which involved participants being asked to speak their thoughts aloud as they read all instructions and completed each item on the ePRO device [21]. Specifically, interviews with individuals with phakic presbyopia utilised ‘think aloud’ discussion to elicit in-depth evidence on the understanding, relevance and interpretation of the PRO instruments. Targeted probing was used to ensure elicitation of feedback on item relevance and whether any important concepts were missing. The interview process was designed in line with best practice and regulatory standards in the assessment of content validity (FDA Guidance, International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Good Research Practice [21, 24,25,26]). All interviews with individuals with phakic presbyopia were conducted in the participant’s local language and lasted approximately 60 min (approximately 45 min was spent on the CD part of the interview).

HCPs were asked to provide feedback on the item wording, whether they perceived the concept to be relevant to phakic presbyopia, missing concepts, and any comments on the response options or recall period. All HCP interviews were conducted by telephone in English and lasted approximately 60 min (approximately 30 min was spent on the CD part of the interview).

Data analysis

Planned analyses and subgroup analyses was detailed in a qualitative analysis plan (QAP) prior to data collection. Verbatim transcripts were qualitatively analyzed using thematic analysis methods and ATLAS.ti software [27, 28]. Verbatim patient quotes were highlighted and grouped by theme/topic. Frequency counts were generated per item and instruction of the NAVQ-P, NVCI and additional PRO instruments to indicate understanding and relevance (yes/no/unclear), along with the generation of a list of participant verbatim statements for each coding domain [29]. Subgroup comparisons identified patterns in instrument interpretation between individuals with phakic presbyopia according to presbyopia severity (mild vs. moderate/severe), age of participant, country (US vs. France vs. Germany), and presence of co-morbid myopia or not, in line with the sampling quotas specified (Additional file 1).

Results

Demographic and clinical characteristics

HCPs (N = 7)

HCPs were interviewed in the US (n = 3/7, 42.9%), France (n = 2/7, 28.6%), Japan (n = 1/7, 14.3%) and Germany (n = 1/7, 14.3%). HCPs were practicing ophthalmologists (n = 5/7, 71.4%) or optometrists (n = 2/7, 28.6%). All HCPs had spent over 10 years managing individuals with presbyopia and self-reported treating an average of > 31 individuals with presbyopia every month (n = 5/7, 71.4%) or 21–30 individuals per month (n = 2/7, 28.6%). HCPs reported that their routine appointments with individuals with presbyopia were typically approximately once per year (n = 3/7, 42.9%), twice per year (n = 2/7, 28.6%) or monthly (n = 2/7, 28.6%). Four HCPs (n = 4/7, 57.1%) reported that they experienced presbyopia themselves, two HCPs (n = 2/7, 28.6%) reported that they did not, and one HCP (n = 1/7, 14.3%) did not comment on whether they experienced presbyopia themselves or not. Table 1 provides an overview of HCP demographic characteristics.

Table 1 Healthcare professional demographic information

Individuals with phakic presbyopia (N = 35)

The thirty-five individuals with phakic presbyopia interviewed were based across the US (n = 15), France (n = 10) and Germany (n = 10). The mean age of the sample was 53.5 years old (range: 40–65). There were more females (n = 21/35, 60.0%) than males (n = 14/35, 40.0%) interviewed. A similar number of Caucasian (n = 11/35, 36.7%) and Black/African American (n = 9/35, 30.0%) participants were interviewed. Data on race or ethnicity was not obtained for the French participants (n = 10/35, 28.6%) in line with French legislation.

There were a higher percentage of participants who had ‘moderate-severe’ phakic presbyopia (n = 21/35, 60.0%) compared with ‘mild’ phakic presbyopia (n = 14/35, 40.0%); based on their near addition (ADD) results provided by the referring physician. Most participants (n = 22/35, 62.9%) were using glasses for near vision correction except for six participants who used contact lenses (n = 6/35, 17.1%). Data on current correction method was missing for eight participants (n = 8/35, 22.9%). Table 2 provides an overview of participant demographic and clinical characteristics.

Table 2 Demographic and clinical characteristics of individuals with phakic presbyopia

Stage 1: HCP interviews (round 1, n = 3) CD findings

Three HCPs (US n = 2, Germany n = 1) were debriefed the NAVQ-P v1.0 and NVS v1.0 (at this point the satisfaction question formed part of the NAVQ). These HCPs indicated that the NAVQ-P assessed concepts relevant to individuals with phakic presbyopia. Two HCPs (n = 2/3, 66.7%) noted that the instructions in the NAVQ-P v1.0 may cause individuals who have co-morbid myopia to answer incorrectly, given that they take their glasses off to be able to see up close. As a result, the instructions were updated from ‘… when you were not wearing glasses/contact lenses’, to ‘… when you were not wearing glasses/contact lenses to see things close to you (less than an arm’s length away)’.

“But people that are nearsighted who don’t have anything on, no glasses, no contact lenses, nothing, if they’re nearsighted to the proper degree, they will never have any symptomatic presbyopia as long as their glasses and their contact lenses are not used.” (HCP 2)

Minor changes were recommended to two items to ensure that the visual task examples provided in items were of equivalent difficulty. This included removal of the example ‘items on a menu’ from item 1 (which assessed reading small print) since this task may not be equivalent to other provided examples (e.g. newspaper), given the potential for dimmed lighting in restaurant settings. The example of ‘gardening’ was also removed from item 9 (which assessed seeing objects up close to engage in hobbies) as the HCPs felt that gardening mostly involved intermediate vision. Additionally, an item to assess contrast sensitivity was added to the NAVQ-P v1.0 based on HCP feedback. All revisions were implemented ahead of the intermediary participant CE interviews (NAVQ-P v2.0).

“Not necessarily because newspaper people might have more light than perhaps in a dimly lit restaurant.” (HCP1)

“The number of tasks that you do that are actually near tasks are pretty small. I mean you’re digging a hole and you’re raking the, the ground and you’re, you know—gardening is not a, a near vision intense hobby.” (HCP2)

Stage 2: Interviews with individuals with phakic presbyopia (CE-only, n = 15) CE findings

Findings from an initial round of CE interviews with a separate sample of 15 individuals with phakic presbyopia (independent of combined CE and CD rounds) contributed to the modification of the NAVQ-P v2.0 forming the NAVQ-P v3.0. Revisions included the addition of relevant item examples, inclusion of an item to assess seeing objects up close in bright light, and the rewording of an item which assessed difficulties with contrast sensitivity to ensure that language was patient-friendly. The item which assessed satisfaction with near vision (from the original NAVQ) was separated from the NAVQ-P v2.0 to form the Near Vision Satisfaction (NVS) instrument. Four other single-item instruments were also created: NVCI, NVCP, PGIS-Presbyopia, and PGIC-Presbyopia.

Stage 3: Interviews with individuals with phakic presbyopia (round 1, n = 17) CD findings

Item wording, response options and the recall period of the NAVQ-P v3.0 and additional instruments were generally well understood and participants appeared to interpret most items correctly and consistently (Fig. 2). Three NAVQ-P v3.0 items were misunderstood in round 1 interviews (summarized in Table 3). These items were misunderstood by two or more participants including item 1 (which assessed seeing objects in bright light, understood by n = 7/17, 41.2%), items 11 (which assessed contrast sensitivity, understood by n = 15/17, 58.8%), and item 12 (which assessed maintaining focus for near vision activities, understood by n = 8/17, 52.9%). Overall conceptual relevance of the NAVQ-P v3.0 was analyzed collectively across the two rounds of CD interviews (Fig. 3).

“That needs to be changed …it says seeing objects close to you in bright light such as seeing dashboard in a car. The dashboard doesn’t have lighting that bright …What you see is the light that reflects out there that comes.” (F55-MILD-R1-US3)

Fig. 2
figure2

Summary of item understanding during round 1 CD interviews

Table 3 Modifications made following round 1 and round 2 CD interviews
Fig. 3
figure3

Summary of overall conceptual relevance (round 1 and round 2 combined)

Round 1 instrument modifications

Modifications were made to the NAVQ-P v3.0 (becoming NAVQ-P v4.0) based on Round 1 interview findings, including the addition of ‘magnifying glass’ alongside ‘wearing glasses/contact lenses’ in the instructions and item stem (the first part of each question) to ensure that individuals would consider all possible forms of vision correction. The second instruction (which read ‘If you did not do the described activity or you have stopped for reasons that are not related to your vision then please select the ‘N/A or stopped doing this for non-visual reasons' option’) was removed as it was determined to be redundant. Revisions were made to the ‘not applicable’ response option wording to read: ‘I did not do this activity in the past seven days’. Examples within three items were updated, one item was reworded and two new items to assess ‘seeing fine detail’ and ‘difficulty adjusting from far vision to close vision’ were developed. Updates were also made to the NVCI, NVCP, NVS, PGIS-Presbyopia and PGIC-Presbyopia (see Table 3 for further information about modifications made to all items). Notably, an alternative set of response options (‘Never’ to ‘Always’) was developed for the NCVI to be debriefed alongside the original response options (‘None of the time’ to ‘All of the time’) in round 2.

Stage 4: FDA feedback on NAVQ-P v4.0 and additional instruments (Type-C meeting)

Based on FDA feedback obtained, the NAVQ-P v4.0 was updated to form v5.0. Modifications were made to the instructions, item stem, and four items. One new item was incorporated to assess the ability to see things when glare is present. Findings from round 1 qualitative interviews identified that this concept was a distinct construct to difficulty seeing things in bright light, thus, an additional item was warranted. Additional wording modifications were made to the PGIS-Presbyopia following feedback from the FDA to create PGIS-Presbyopia v3.0. No further changes were made to the NVCI v2.0, NVCP v1.0, NVS v2.0, and the PGIC-Presbyopia v2.0.

Stage 5: HCP interviews (round 2, n = 4) CD findings

Four HCPs (France n = 2, US n = 1, Japan = 1) debriefed the instruments in round 2 (see versions in Fig. 1). These HCPs agreed that the NAVQ-P v5.0 and additional instruments would be well understood by individuals with phakic presbyopia and supported the clinical relevance of concepts assessed. No missing concepts of importance were highlighted. Minor wording changes were suggested, mainly concerning the examples provided within item wording. The changes were revisited following round 2 of CD interviews with individuals with phakic presbyopia to ensure that revisions were in-line with participant feedback and understanding.

“The only part of it that might get confusing is, um, you’re saying, um, close to you in dim light, but then you use the term reading a book by lamplight. So what you’re inherently doing in this second portion of the questions, is, um, it’s, it’s almost like the patient is- they’re not in dim light anymore because they are beside a lamplight.” (HCP4)

Stage 5: Interviews with individuals with presbyopia (round 2, n = 18) CD findings

Item wording, response options and recall period of the NAVQ-P v5.0 and additional instruments were generally well understood and interpreted consistently (Fig. 4).

Fig. 4
figure4

Summary of item understanding during round 2 CD interviews

Minor issues in understanding or consistent interpretation were identified for four NAVQ-P v5.0 items (Table 3). Item 1 (difficulty reading small printed text on paper) was misinterpreted by n = 2/18 (11.1%) as asking about reading from a tablet or smartphone. Item 3 (difficulty reading small digital text on a tablet device) was misinterpreted by n = 2/18 (11.1%) to be about reading from a laptop. Item 7 (difficulty seeing the keypad on a smartphone or tablet screen) demonstrated an inconsistent interpretation of ‘keypad’, with n = 6/18 (33.3%) participants discussing keyboards or buttons rather than focussing on the keypad on the screen. Item 15 (ability for the eye to adjust between different vision fields) was generally misunderstood by n = 3/18 (16.7%) and also misinterpreted by n = 2/18 (11.1%) to relate to a change in lighting.

“A keypad is something where you press something down, not a touchscreen.” (F40-MILD-R2-DE1)

Round 2 instrument modifications

Modifications to NAVQ-P v5.0 item wording and response options were made (forming NAVQ-P v6.0). See Table 3 for further information about updates to items. Item ordering was also adjusted: the order of the items assessing ‘vision when glare is present’ and ‘vision in bright light’ was reversed to avoid respondents thinking about glare when responding to the bright light item. The alternative set of response options were retained for the NVCI (v3.0). No changes were made to the NVCP v2.0, NVS 3.0, PGIS-Presbyopia v3.0 and PGIC-Presbyopia v2.0.

Item relevance assessed across round 1 and round 2 interviews with individuals with phakic presbyopia (n = 35)

Given the concepts assessed did not change across both rounds (apart from some new items being added), relevance is summarised collectively for both round 1 (stage 3) and round 2 (stage 5) interviews with individuals with phakic presbyopia. All concepts of the NAVQ-P were considered relevant to at least 30% of individuals with phakic presbyopia (Fig. 3). Twelve of the fifteen (n = 12/15, 80.0%) item concepts in the NAVQ-P were considered relevant to at least 50% of participants. The NVCI instrument was relevant to a total of 26 participants (n = 26/35, 74.3%). Assessment of relevance was not applicable to the NVCP, NVS, PGIS-Presbyopia and PGIC-Presbyopia instruments.

“Reading small printed text on paper such as newspaper—um, oh extremely difficult as I said before. Everything has to be large print.” (M50-MOD-R2-US8)

The concepts listed below demonstrated lower relevance (relative to other items) for participants with phakic presbyopia. Participants either did not report difficulty with the described visual activity in relation to their presbyopia or did not perform the task in the past seven days (Fig. 3). These items were retained, pending psychometric evaluation, but were flagged as potential candidates for deletion at a later stage.

  • Reading on a tablet device: n = 15/35 (42.9%) participants reported as relevant.

  • Seeing things in bright light: n = 11/35 (31.4%) participants reported as relevant.

  • Ability for the eye to adjust between different vision fields: n = 8/18 (44.4%) participants reported as relevant.

“I don’t have difficulties to adjust. Because in any way, I’m blurry from a distance, I’m blurry at close distance so...” (F58-MOD-R2-FR2)

General feedback on the instruments from round 1 and round 2 interviews with individuals with phakic presbyopia (n = 35)

No important concepts were identified as missing by participants. Participants were asked about the usability of the ePRO, with most participants (n = 11/15, 73.3%) who were asked in round 2 (stage 5) confirming that the text size was adequate, and n = 15/17 (88.2%) reporting no concerns with navigation on the device. A small number of participants (n = 3/35, 8.6%) estimated how long it would take them to complete the instruments, which ranged from 10 (n = 2/3, 66.7%) to 20 min (n = 1/3, 33.3%). However, these estimations may be inflated given that the instruments were completed during the interview which involved discussing the response to each item following completion.

“I think I would have needed 10 min.” (F44-MILD-R2-DE3)

Discussion

To address the need for a PRO that assesses phakic presbyopia near vision functioning, the present study details the modification and content validity testing of the NAVQ-P and additional instruments (NVS, NVCI, NVCP, PGIS-Presbyopia and PGIC-Presbyopia). Findings suggest that the NAVQ-P demonstrates content validity as an assessment of near vision functioning which reflects the most important concerns of individuals with phakic presbyopia. The additional instruments are appropriate to assess near vision correction independence (NVCI), near vision correction preference (NVCP), and near vision satisfaction (NVS). The study involved the successful modification of the original NAVQ to address the limitations of the use of this instrument in clinical studies with individuals with phakic presbyopia.

The study had been designed in line with best practice and standards outlined in regulatory guidance on the steps necessary to establish content validity and feedback from the FDA was obtained [21, 24, 25, 30]. As a result, the NAVQ-P is a product of rigorous research involving an initial literature and instrument review and social media listening, multiple rounds of combined qualitative CE and CD interviews (with individuals with phakic presbyopia and HCPs) and engagement with regulators. Findings from an initial literature and social media review informed revision of the original NAVQ [16, 22]. The NAVQ-P was then subject to CD with individuals with phakic presbyopia and mapped alongside findings from qualitative CE interviews (described elsewhere) [23] to ensure all important concepts were included. The concepts included in the NAVQ-P also reflect findings from previous literature investigating the individual experience of presbyopia [8, 10,11,12, 14, 31]. The two rounds of CD interviews allowed for modifications to be implemented to the instrument following round 1 interviews, which could then be evaluated during round 2 interviews.

Participants demonstrated good understanding of the NAVQ-P item wording, recall period and response options in round 2 (stage 5) interviews. The three new items that were added following round 1 CD interviews and FDA feedback were well understood and had reasonably high relevance to presbyopic participants; supporting inclusion in the NAVQ-P. A small number of modifications were made following round 2 based on the CD findings to enhance interpretation. Additionally, formatting changes were made to enhance comprehension. HCPs confirmed that the NAVQ-P concepts were clinically relevant to individuals with phakic presbyopia.

As the NAVQ-P and additional instruments were developed and cognitively debriefed in an electronic mode of administration (on a tablet device), the interviews confirmed usability of the ePRO and that the font size of an electronic administration of instruments was appropriate for individuals with phakic presbyopia. In addition, throughout the development of the NAVQ-P it became apparent that individuals who experienced both myopia and presbyopia used their glasses differently to see up close. Those with comorbid myopia took their long-distance glasses off to see up close, while those with phakic presbyopia only put their reading glasses on to see up close. With this in mind, the instructions were updated to specifically ask participants to think about when they are not wearing their glasses to see things close to them. As a result, the instrument is suitable for use in individuals with phakic presbyopia who do or do not have comorbid myopia.

A key strength of this study was that clinical relevance and development of the instruments was ensured via collaboration with the NAVQ developer and specialist HCPs. Regulatory advice guided further development of the NAVQ-P and NVCI instruments and was pivotal in ensuring that the NAVQ-P and NVCI meet the quality standards for content validity, supporting use of these instruments as clinical trial endpoints with potential to support label claims. The NAVQ-P provides a unique opportunity to assess near vision functioning that is specific to individuals with phakic presbyopia. Other PROs commonly used in ophthalmology populations to assess visual function (such as the NEI RQL-42 or the NEI VFQ-25) assess some concepts that are not relevant to presbyopia, such as ability to do distance vision activities or limitations with peripheral vision, have been developed with individuals who have received refractive eye surgery, or require individuals to respond to items thinking about their vision when using vision correction [19, 20].

The study sample included individuals with phakic presbyopia from multiple countries (US, Germany, and France) and quotas were used to ensure the sample had diverse demographic and clinical characteristics. This enables confidence that findings are representative of the wider phakic presbyopia population and provides evidence of cross-cultural validity, specifically in the US and Europe. Additionally, the HCPs from the US, France, Japan and Germany all confirmed clinical relevance of the NAVQ-P and additional instruments in these countries, further supporting cross-cultural validity and clinical relevance. A wide range of refractive error was included in the sample, with this study confirming the suitability of the use of the NAVQ-P and additional instruments in individuals with phakic presbyopia, regardless of their refractive error (i.e. severity).

While multiple countries were included, all were western countries in highly developed nations, therefore, further research in countries in Asia, South America and/or Africa in the future would be of interest to provide further insight regarding the degree to which the findings can be generalized cross-culturally.

Conclusions

The study findings reported here support the content validity of the newly adapted NAVQ-P, NVCI, NVCP, NVS, PGIS-Presbyopia and PGIC-Presbyopia in individuals with phakic presbyopia. Psychometric evaluation is planned to support finalization of scoring (with possible item reduction), and assess the validity, reliability, and importantly ability to detect change over time for the instruments, ultimately confirming the adequacy of the NAVQ-P and additional instruments as clinical trial endpoints in support of potential labelling claims.

Availability of data and materials

The data that support the findings of this study are available from Novartis Pharma AG but restrictions apply to the availability of these data, which were used under license for the current study, and so are not publicly available. Data are however available from the authors upon reasonable request and with permission of Novartis Pharma AG.

References

  1. 1.

    Holden BA, Fricke TR, Ho SM, Wong R, Schlenther G, Cronjé S, Burnett A, Papas E, Naidoo KS, Frick KD (2008) Global vision impairment due to uncorrected presbyopia. Arch Ophthalmol 126(12):1731–1739

    Article  Google Scholar 

  2. 2.

    Wolffsohn JS, Davies LN (2019) Presbyopia: effectiveness of correction strategies. Prog Retin Eye Res 68:124–143

    Article  Google Scholar 

  3. 3.

    Glasser A, Campbell MC (1999) On the potential causes of presbyopia. Vis Res 39(7):1267–1272

    Article  Google Scholar 

  4. 4.

    Heys KR, Cram SL, Truscott RJ (2004) Massive increase in the stiffness of the human lens nucleus with age: The basis for presbyopia? Mol Vis 10:956–963

    PubMed  Google Scholar 

  5. 5.

    Garner WH, Garner MH (2016) Protein disulfide levels and lens elasticity modulation: applications for presbyopia. Investig Ophthalmol Vis Sci 57(6):2851–2863

    CAS  Article  Google Scholar 

  6. 6.

    David G, Pedrigi RM, Humphrey J (2017) Accommodation of the human lens capsule using a finite element model based on nonlinear regionally anisotropic biomembranes. Comput Methods Biomech Biomed Engin 20(3):302–307

    CAS  Article  Google Scholar 

  7. 7.

    Kandel H, Khadka J, Goggin M, Pesudovs K (2017) Impact of refractive error on quality of life: A qualitative study. Clin Exp Ophthalmol 45(7):677–688

    Article  Google Scholar 

  8. 8.

    Kandel H, Khadka J, Shrestha MK, Sharma S, Neupane Kandel S, Dhungana P, Pradhan K, Nepal BP, Thapa S, Pesudovs K (2017) Uncorrected and corrected refractive error experiences of Nepalese adults: a qualitative study. Ophthalmic Epidemiol 1–15

  9. 9.

    Opthalmology American Academy of Ophthalmology (2017) What is presbyopia? https://www.aao.org/eye-health/diseases/what-is-presbyopia. Accessed 22nd January 2021

  10. 10.

    Goertz AD, Stewart WC, Burns WR, Stewart JA, Nelson LA (2014) Review of the impact of presbyopia on quality of life in the developing and developed world. Acta Ophthalmol 92(6):497–500

    Article  Google Scholar 

  11. 11.

    McDonnell PJ, Lee P, Spritzer K, Lindblad AS, Hays RD (2003) Associations of presbyopia with vision-targeted health-related quality of life. Arch Ophthalmol 121(11):1577–1581

    Article  Google Scholar 

  12. 12.

    Lu Q, Congdon N, He X, Murthy GV, Yang A, He W (2011) Quality of life and near vision impairment due to functional presbyopia among rural Chinese adults. Investig Ophthalmol Vis Sci 52(7):4118–4123

    Article  Google Scholar 

  13. 13.

    Williams S, Brian G, Toit Rd (2012) Measuring vision-specific quality of life among adults in Fiji. Ophthalmic Epidemiol 19(6):388–395

    Article  Google Scholar 

  14. 14.

    Frick KD, Joy SM, Wilson DA, Naidoo KS, Holden BA (2015) The global burden of potential productivity loss from uncorrected presbyopia. Ophthalmology 122(8):1706–1710

    Article  Google Scholar 

  15. 15.

    Dhariwal M, Thakker D, Bouchet C, Lemp-Hull J (2018) Global burden of presbyopia: a systematic literature review. Value Health 21(3):274–274

    Article  Google Scholar 

  16. 16.

    Sharma G, Chiva-Razavi S, Viriato D, Naujoks C, Patalano F, Bentley S, Findley A, Johnson C, Arbuckle R, Wolffsohn J (2020) Patient-reported outcome measures in presbyopia: a literature review. Br Med J Open Ophthalmol 5(1):e000453

    Article  Google Scholar 

  17. 17.

    Paley J (2016) Phenomenology as qualitative research: A critical analysis of meaning attribution. Routledge, UK

  18. 18.

    Buckhurst PJ, Wolffsohn JS, Gupta N, Naroo SA, Davies LN, Shah S (2012) Development of a questionnaire to assess the relative subjective benefits of presbyopia correction. J Cataract Refract Surg 38(1):74–79

    Article  Google Scholar 

  19. 19.

    Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Berry S, Hays RD (2001) Development of the 25-list-item national eye institute visual function questionnaire. Arch Ophthalmol 119(7):1050–1058

    CAS  Article  Google Scholar 

  20. 20.

    Berry S, Mangione CM, Lindblad AS, McDonnell PJ, Investigators N-RFG (2003) Development of the National Eye Institute refractive error correction quality of life questionnaire: focus groups. Ophthalmology 110(12):2285–2291

    Article  Google Scholar 

  21. 21.

    US Food and Drug Administration (2009) Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims. FDA, Maryland

  22. 22.

    Wolffsohn JS, Leteneux-Pantais C, Chiva-Razavi S, Bentley S, Johnson C, Findley A, Tolley C, Arbuckle R, Kommineni J, Tyagi N (2020) Social media listening to understand the lived experience of presbyopia: systematic search and content analysis study. J Med Internet Res 22(9):e18306

    Article  Google Scholar 

  23. 23.

    Bentley S, Findley A, Chiva-Razavi S, Naujoks C, Patalano F, Johnson C, Arbuckle A, Wolffsohn JS (in press) Understanding the visual function symptoms and associated functional impacts of phakic presbyopia. J Patient-Reported Outcome

  24. 24.

    US Food and Drug Administration (2020) Patient-focused drug development: collecting comprehensive and representative input. Guidance for industry, Food and Drug Administration staff, and other stakeholders. FDA, Maryland

  25. 25.

    US Food and Drug Administration (2019) Patient-focused drug development: methods to identify what is important to patients. Guidance for Industry, Food and Drug Administration staff, and other stakeholders. Draft guidance. FDA, Maryland

  26. 26.

    Patrick DL, Burke LB, Gwaltney CJ, Leidy NK, Martin ML, Molsen E, Ring L (2011) Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO good research practices task force report: part 2—assessing respondent understanding. Value Health 14(8):978–988

    Article  Google Scholar 

  27. 27.

    ATLAS.ti.Scientific Software Development GmbH B, Germany (2013) Atlas.ti.software version 7

  28. 28.

    Braun V, Clarke V (2006) Using thematic analysis in psychology. Qual Res Psychol 3(2):77–101

    Article  Google Scholar 

  29. 29.

    Willis GB (2015) Analysis of the cognitive interview in questionnaire design. Oxford University Press, Oxford

    Google Scholar 

  30. 30.

    US Food and Drug Administration (2018) Patient-focused drug development guidance public workshop: methods to identify what is important to patients & select, develop or modify fit-for-purpose clinical outcome assessments. Discussion document for Patient-Focused Drug Development Public Workshop on Guidance 3. FDA, Maryland

  31. 31.

    Patel I, West SK (2007) Presbyopia: prevalence, impact, and interventions. Commun Eye Health 20(63):40

    Google Scholar 

Download references

Acknowledgements

The authors wish to extend a special thanks to the individuals with presbyopia and HCPs who participated in the interviews and provided valuable insight into their experience of living with/managing individuals with presbyopia. The authors would also like to acknowledge Bradley Mason for helping to develop the first draft of the manuscript.

Funding

Funding was received from Novartis Pharma AG.

Author information

Affiliations

Authors

Contributions

SB, AF and CJ conducted data collection and performed the qualitative data analysis. RA, SB, and AF developed the first draft of the manuscript. All authors were involved in the design of the study, reviewed the study protocol and contributed to the reporting and interpretation of the findings and writing and reviewing of this manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Sarah Bentley.

Ethics declarations

Ethics approval and consent to participate

Ethical approval was obtained from an Institutional Review Board (IRB), Copernicus Group Independent Review Board (CGIRB; IRB ref: ADE1-18-049), prior to any study related activities in the US, and from Salus IRB (IRB ref: Novartis CUNR844A2001) prior to any study related activities in Germany. In line with ethical requirements for research in France at the time of the research, a letter was submitted to the Conseil National de l'Ordre des Médecins (CNOM) to notify them of the study procedures and payments to recruiting health professionals. Written and verbal informed consent was obtained from every participant before each interview was conducted and before any other study activities.

Consent for publication

Not applicable.

Competing interests

JW is a developer of the original NAVQ and has been engaged with Novartis on other projects and has received honoraria for speaker events, ad boards and other related activities. RA, SB, and AF are employed by Adelphi Values Patient-Centered Outcomes team, which received funding from Novartis Pharma AG to conduct the research described in this manuscript. The Adelphi Values team also work with many other pharmaceutical companies to perform Patient-Centered Outcomes research. CJ was employed by Adelphi Values Patient-Centered Outcomes at the time of the study. SC-R, FP,CN and GS are employed by Novartis Pharma AG. FP and CN own shares in Novartis.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Additional information on methodology.

Additional file 2

. Example HCP interview guide questions.

Additional file 3

. Example interview guide questions with participants with presbyopia.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Bentley, S., Findley, A., Chiva-Razavi, S. et al. Evaluation of the content validity of patient-reported outcome (PRO) instruments developed for use with individuals with phakic presbyopia, including the Near Activity Visual Questionnaire-presbyopia (NAVQ-P) and the near vision correction independence (NVCI) instrument. J Patient Rep Outcomes 5, 109 (2021). https://doi.org/10.1186/s41687-021-00379-x

Download citation

Keywords

  • Presbyopia
  • Content validity
  • Patient-reported outcome
  • Cognitive debriefing
  • Qualitative
  • Interviews
  • NAVQ-P