Table 2 The five inferences for measurement of theoretical constructs as related to the Health Literacy Questionnaire (HLQ)

1. Scoring inference

The scoring inference assumes that users of the HLQ will abide by the warrant of the HLQ scoring instructions. The evidence for this inference is derived from development information about the HLQ items, scales and response options and scoring procedures, which includes that scoring is free from bias [7, 63, 94]. Statements about how a study scores a PROM need to be clear because this provides evidence for the assumption that the scoring has taken place as intended. The validity of the scoring inference is the basis for the validity of all other inferences.

2. Generalisation inference

The warrant for the generalisation inference follows the same principle of any generalisation study: that scores are estimates (representative) of the scores that other similar respondents (from a universe of possible respondents) would get on the same or similar measure (e.g., a translated HLQ). The assumption is that context is not relevant to scale score interpretations (i.e., that time, place, language/culture or other contextual factors do not present validity threats). The Standards states that evidence for generalisation (relations to other variables) stems from meta-analyses and statistical summaries of past studies (e.g., cumulative databases) (p.18) [7]. While reliability evidence is relevant to every inference, it is predominantly applicable to the generalisation inference [47], and is reported as such in this study.

3. Extrapolation inference

The extrapolation inference is the first step in the process of linking the observed HLQ scores to the nine-domain health literacy theory. It is this inference that underpins the majority of psychometric “construct validity” testing in health measurement. The warrant for this inference is that the HLQ scale scores are accurate representations of the corresponding HLQ health literacy domains (i.e., the target scores). This warrant assumes that the nine scale scores account for a range of attributes, resources and competencies that people need for accessing, understanding, and using health information and services to manage their health [94]. The evidence for this inference [7] includes:

• information derived from the processes used to develop the HLQ items, scales and response options, and how respondents interpret and understand these;

• the internal structure of the HLQ domains using methods that, for example, test if response patterns conform to the nine scale scores;

• and the relationships between the HLQ scales, which while related, are distinctly independent from each other.

4. Theory-based interpretation inference

The warrant for the theory-based interpretation inference is the nine-domain health literacy theory, which is operationalised in the high and low score descriptions [94]. This warrant assumes that the HLQ health literacy theory (i.e., domain descriptions) explains the scales and items, and that the item and scale scores provide appropriate estimates of the theory [63]. The evidence for this inference is derived from evaluation of the content of the HLQ items and how respondents engage with the items [7].

5. Implications (or utilisation) inference

The overarching warrant for the implications (or utilisation) inference is the rules for decisions based on HLQ data. Bachman (pp.18–20) [12] describes four specific warrants for a data utilisation argument:

• Relevance: that the score-based interpretation is relevant to the decision to be made.

• Utility: that the score-based interpretation is useful for making the intended decision.

• Intended consequences: that the consequences of using the assessment and making intended decisions will be beneficial to individuals, the program, company, institution, or system, or to society at large.

• Sufficiency: that the assessment/s provide sufficient information for making the decision.

The main assumption underlying these four warrants is that the HLQ health literacy construct (operationalised through the nine scale scores) embodies several factors (the nine theoretical domains) that influence health outcomes (and potentially health equity). The evidence for this assumption is based primarily on the validity-related consequences of data-based decisions [7, 12, 54, 58, 63, 72].