Skip to main content

Table 1 Evaluation criteria for key components of patient-reported outcome measure (PRO) measure development and testing

From: Capturing the patient experience in systemic lupus erythematosus: Are widely used measures fit-for-purpose and adherent to FDA PRO guidance recommendations?

Target population

Yes/no

1.*

Is the target population transparent?

Clear reporting of the decisions used in the study regarding selection of the target population; defines the intended population the researchers targeted to generalize results of PRO development and testing

 

2.*

Is the target population clearly specified?

Sufficient details are provided about the intended population to understand the generalizability of the study as well as the study population that should be targeted for PRO development and testing

 

3.*

Are the study-population characteristics reported for all phases of development?

Study characteristics are reported in item generation, cognitive interviewing, and other psychometric property testing

 

4.*

Are relevant study-population characteristics (age, sex, condition severity, race, etc.) explicitly identified for the phases of development conducted?

All the important demographic/clinical characteristics are reported for each phase. This may be based upon objectives, inclusion/exclusion criteria, etc. that indicate the target population

 

5.*

Are the study-population characteristics reported consistent across all phases of development?

The percentage of participants for each characteristic are similar across phases

 

6

Are minimum recruitment targets for the study population stated to ensure the study population is representative of the target population of interest for each phase?

Sample size targets are provided for demographic/clinical characteristics

 

Concepts measured

Yes/no

7.*

Is the approach for derivation of items appropriate for item generation?

Using patient interviews/focus groups, open-ended questions during interviews

 

8.*

Is the approach for derivation of items comprehensive for item generation?

Identifying concepts/items that matter most to patients including symptoms and/or activities experienced by the majority of the patients representing the target population

There is evidence that interviews/focus groups encompassed a wide range of patients representing the target population

 

9.*

Was cognitive interviewing conducted via patient input?

 

10.*

Was the feedback collected during cognitive interviewing considered and, if not, was an explanation provided for why feedback was not incorporated?

 

11

Was quantitative testing of items/concepts conducted?

Testing of other psychometric properties (e.g., reliability, construct validity, etc.)

 

Measurement properties

Yes/no

12.*

Is there evidence that saturation was achieved during item generation?

No new relevant information emerges from interviews/focus group discussions

 

13.*

Is there evidence that saturation was achieved during cognitive interviewing?

No new relevant information emerges from interviews/focus group discussions

 

14.*

For each of the relevant study population characteristics identified for the target population, were minimum targets met?

Minimum targets are met, evidence of representativeness, etc

 

15

Are reliability results reported?

 

16

Are construct-validity results reported?

 

17

Are ability-to-detect-change results reported?

 

Documentation

Yes/no

18

Is the study population clearly specified?

Sufficient details are provided about the study population to generate a reproducible study

 

19

Is there documentation of elicited concepts (e.g., transcripts, quotes, saturation grid, etc.)?

 

20

Is there documentation of cognitive interviewing feedback being incorporated (e.g., stated as revisions made, item-tracking matrix, etc.)?

 

21

Are the number of participants for each phase clearly reported, including characteristics of the study population?

 

22

Is there documentation of appropriate content of the measure?

Exact words used by patients to represent concepts, response options, recall period, etc

 

23

Is the item generation-process transparent?

Clear reporting of the decisions used in the study, what the researchers actually did

Item generation techniques, theoretical approaches, source of items

 

24

Is the cognitive-interviewing process transparent?

Clear reporting of the decisions used in the study, what the researchers actually did

 

25

Are the quantitative techniques transparent?

Clear reporting of the decisions used in the study, what the researchers actually did. The methods for domain generation, testing of reliability, construct validity, etc

 

26

Are the other psychometric testing results clear?

The number of domains generated, reliability estimates, construct validity results, etc

 
  1. *Please see instructions in the Additional file 1: Appendix 1.