Patient sample
Cancer patients were recruited in 2017 at the Medical University of Innsbruck, Austria. We aimed to include patients of different age, diagnosis groups, and treatment modalities.
Eligibility criteria for patients were a diagnosis of cancer, age > 18, sufficient command of German, no overt cognitive impairments, and written informed consent. Clinical data was gathered from medical charts comprising information on the diagnosis (ICD-10), treatment approach (curative/palliative), previous and current treatment modalities (e.g. surgery, chemotherapy, or radiotherapy), current medication, and comorbidities (e.g. heart problems, arthritis/rheumatism, or asthma). Sociodemographic information was collected as part of the valuation survey (see below).
All respondents provided written informed consent. Ethical approval was obtained from the Medical University of Innsbruck [AN215-0016]. All patients who agreed to participation in the QLU-C10D valuation survey completed the entire interview. Survey completion and cognitive interviews took 30–45 min.
Matched general population controls
The general population control group was drawn from the QLU-C10D Austrian general population valuation which has been performed in 2017 [22]. Recruitment and assessment were performed by Survey Engine (www.surveyengine.com), a company specialized in the web-based conduction of DCEs using internet panels. For the present study, we matched a control group from the 1000 Austrian general population respondents to the patient sample according to age, sex, and education to obtain a case control ratio of 1:4.
QLU-C10D valuation survey
For QLU-C10D valuations a standardised methodology is in place. The survey comprises questions on sociodemographic information, 16 DCE choice sets (selected out of a total of 960; described below), self-report questionnaires on health status (QLQ-C30, Kessler-10, EQ-5D-3L), and feedback questions on the clarity of health state presentations (assessed on a 5-point Likert scale from ‘very clear’ to ‘very unclear’), the difficulty in comparison to other surveys (respondents are asked to compare to any other survey they might have participated in which could be none in the case of cancer patients or could be questionnaire studies they might be familiar with; response options are ‘easier’, ‘similar’, ‘more difficult’, and ‘can’t tell’;), the difficulty to make a decision between the health states (assessed on a 5-point Likert scale including the options ‘very difficult’, ‘difficult’, ‘neither/nor’, ‘easy’, ‘very easy’), and the strategy on how a decision was reached (options: ‘no strategy’, ‘focus on a few aspects’, ‘focus on highlighted aspects’, ‘focus on most aspects’, ‘focus on all aspects’, and ‘other strategy’). All survey material is provided in the supplementary material. The survey is administered web-based. Each DCE choice set comprises two hypothetical QLU-C10D health profiles (i.e., the 10 HRQOL domains with different levels of impairments on 4 levels from “not at all” to “severe”) and survival times in that health states (one, two, five, or ten years). To keep cognitive burden on a manageable level, impairments on only five domains differed between the two options in each choice set (highlighted in yellow). The respondents are asked to select their preferred health profile (see example in Fig. 1). More details on the DCE and the valuation survey can be found in prior publications [12,13,14, 16]. The QLU-C10D valuation methodology has been intensively investigated, including testing the impact of different graphical presentations, impact of ordering of attributes, and test–retest reliability [13,14,15].
Mixed-methods approach for pilot testing in cancer patients
Based on Collins [23], Mullin et al. [24] and Atkinson et al. [25] the following aspects were assessed with regard to the applicability of the QLU-C10D valuation methodology in cancer patients: comprehension, i.e. understanding of the task (e.g. How clear/unclear is the purpose of the survey/this explanation to you? Could you repeat this in your own words?), retrieval, i.e. the information processing strategy including recall of information (e.g. Do you have a particular strategy?), judgement, i.e. the process of formulating an answer to each question (e.g. How easy/difficult was your choice?), response (e.g. How do you feel about your choice?), and burden, i.e. the perceived importance or reasonability of the task which is linked to collaboration motivation (e.g. How relevant do you consider this tasks to be? Would you consider the tasks suitable for other patients as well? What would you change?).
This was achieved by employing a mixed-methods approach.
The qualitative part comprised a cognitive interview with verbal probing [26] with cancer patients covering the mental process in capturing the provided information and in giving responses. The interview was performed alongside the completion of the QLU-C10D valuation survey. The quantitative part encompassed the comparison of responses to the feedback questions incorporated in the survey between patients and matched control group respondents. Survey material has been provided in the supplementary files (Additional file 1: Appendix A).
Analyses and sample size considerations
Sample size considerations were based on the recommendations of Lancaster et al. [27], and specifically on Morse [28] and Glaser and Strauss [29] focussing on the concept of content saturation for qualitative approaches. The concept of content saturation with approx. 30 participants was again confirmed in a content analysis involving 560 studies by Mason et al. [30].
Qualitative data was analysed based on the Grounded Theory Approach, described by Glaser [31]. Interview data were independently reviewed by two researchers performing inductive coding using Microsoft Excel.
Quantitative data was analysed using Chi-square tests for comparing frequencies (feedback questions). To show that included patients and general population controls indeed differed with regard to functioning and health status we also compared EORTC QLQ-C30 scores (assessed as part of the valuation survey—see above). This was done using Mann–Witney-U tests. A significance level of 0.05 was applied. Statistical analyses were conducted using IBM SPSS 23.0.