Table 1 Studies used to test theories 1 and 2

Primary care

Specialist mental health

Oncology or Palliative care

Screening/ Assessment

Monitoring

Standard

Individual

Neudert et al. [80]

Completion of Sickness Impact Profile, SF-36 and SEIQOL-DW by 42 patients with amyotrophic lateral sclerosis in Germany

All patients completed the Sickness Impact Profile; patients randomly allocated to complete SF-36 or SEIQOL-DW. Questionnaire to assess patient’s perceptions of validity and distress caused by completion

✓

✓

• Patients’ ratings for the perceived validity of the instrument in measuring their quality of life were significantly higher for the SEIQOL-DW than the SIP or SF-36.

• The SEIQoL-DW had the lowest rating for emotional distress, followed by the SF-36 and then the SIP, with the difference between the SEIQoL-DW and the SIP being statistically significant.

Mallinson [85]

Completion of SF-36 by 56 people over 65 referred for physiotherapy in UK

Qualitative analysis of standardised interviewer administration of the SF-36

✓

• Technical issues included double barrelled questions, use of unfamiliar or vague terms.

• Conceptual issues included: items premised on implicit assumptions about what is ‘normal’ that was not shared by respondents.

Westerman et al. [81]

Completion of SEIQOL-DW by 31 patients with small cell lung cancer in the Netherlands

Qualitative analysis of cue elicitation process during interviewer administration of SEIQOL-DW.

✓

✓

• Cues are co-constructed by interviewers and patients.

• Reducing patient narratives into five cues can result in loss of meaning

Farquhar et al. [82]

Completion of SEIQOL-DW by 13 patients with COPD in the UK

Qualitative analysis of three step process of interviewer administration of SEIQOL-DW.

✓

• Cue selection can distort patient experiences.

• Patients used different frames of reference to rate severity of cues and misunderstood instructions.

Dowrick et al. [65]

Use of standardised PROMs to detect patients with depression in UK

Qualitative interviews with 24 patients and 34 GPs to explore their experiences of PROM use

✓

✓

✓

• Some patients felt PROMs completion helped them to articulate their feelings and communicate these to GP

• Completing a PROM made some patients feel GP was taking them seriously.

• PROMs completion helped some patients reflect and increase their understanding of their condition.

• Some patients found PROMs didn’t reflect complexity of their condition.

Leydon et al. [75]

Use of standardised PROMs to detect patients for depression in UK

Qualitative interviews with 24 patients and 34 GPs to explore their experiences of PROM use

✓

✓

✓

• GPs felt developing a relationship with patients was important to detect and manage depression.

• GPs perceived PROMs as mechanistic and felt they trivialised patients’ emotions.

• GPs felt uncomfortable asking patients to complete a PROM and found it difficult to integrate them into flow of consultation.

Mitchell et al. [83]

Use of standardised PROMs to detect patients for depression in UK

Four qualitative focus groups to explore primary care practitioners experiences of PROMs use

✓

✓

✓

• GPs felt PROMs were intrusive and duplicated what they already knew about patients, so did not influence management.

• GPs adapted the way they administered the PROM to fit into the flow of clinical work in ways that sometimes compromised the validity of the PROM.

Pettersson et al. [76]

Use of standardised PROMs to detect and manage patients for depression in Sweden

Qualitative interviews with 27 GPs to explore their experiences of PROM use

✓

✓

✓

• GPs perceived PROMs did not add to what they already gathered from patients by verbal interaction.

• GPs suggested PROMs distracted them from noticing patient’s non-verbal communication.

• Most GPs felt PROMs constrained the narrative of the patient, interfered with their ability to listen to the patient.

Hall et al. [70]

Pilot study of session by session outcome monitoring using the Strengths and Difficulties questionnaire in Children and Young People’s Improving Access to Psychological Therapies programme in the UK

Qualitative interviews with 10 clinicians, 8 administrative staff and 15 families to explore their views and experiences of PROMs use

✓

✓

✓

• Clinicians felt completing PROMs during the session got in the way of the therapeutic process while referring to results created awkward silences and distracted patients.

• Families felt completing the PROM was helpful in its own right as it provided an opportunity to reflect and assess their condition.

• Patients felt PROMs completion helped them to understand their progress if clinicians discussed findings with them.

Wolpert et al. [74]

Child mental health services prior to implementation of a symptom checklist; childrens’ diabetes services currently using the PedsQL in the UK

Focus groups with 5 children accessing mental health services; 4 interviews with young people and their mothers accessing diabetes services; 3 interviews with mothers whose children accessed diabetes services to explore views of PROMs

✓

✓

✓

• Children held mixed views about value of PROMs and questioned whether they could capture the dynamic nature of their experiences.

• Young people felt they needed to develop rapport with clinicians before sharing sensitive information.

• Clinicians reported a tension between fluid nature of patient experiences and standard format of PROM.

• Parents of children with diabetes were concerned that focusing consultations on emotional issues might mean less time was available for discussing diabetes management.

Sharples et al. [27]

Standardised PROMs used within child and adolescent mental health services in one mental health trust in the UK

Semi-structured interviews with nine clinicians in from CAMHS in one mental health trust in London, UK

✓

✓

✓

• Clinicians felt mandatory use of PROMs had a negative impact on clinician therapist relationships by taking time away from sessions, not considering differences in client needs and could alienate patients

• Clinicians felt patients liked the measured because they provided a structured framework for discussion

Stasiak et al. (2012)

Range of standardised PROMs in routine use to monitor outcomes in child and adolescent mental health services in New Zealand

Five focus groups with 34 children and 21 family members to explore experiences and views of PROMs use for routine outcomes monitoring

✓

✓

✓

• Young people questioned whether the PROMs captured the dynamic and changing nature of their symptoms.

• Young people felt completing a PROM was easier and less embarrassing than talking to a clinician about difficult issues but also felt PROMs should be administered by someone they had developed rapport with.

• Parents felt completing a PROM signalled someone was taking their concerns seriously but wanted reassurance these data were going to inform the care of their child.

Cheyne and Kinn [67]

Use of the SEIQoL with 20 clients receiving in drug and alcohol counselling service over six months as part of an RCT

Three focus groups with three counsellors to explore SEIQoL use at 12 weeks, 18 weeks and 24 weeks, completion of a questionnaire and review of patient case notes

✓

✓

✓

✓

• Counsellors perceived that SEIQoL completion enabled service users to reflect on their life and identify issues to tackle in therapy.

• Counsellors felt use of the SEIQoL prompted them to listen and reflect more on what the service user was saying and supported the process of relationship building with service users.

Alves et al. [19]

Use of two standardised measures - the Treatment Outcomes Profile (TOP), Clinical Outcome Routine Evaluation-Outcome Measure (CORE-OM) and two individualised measures – Psychological Outcomes Profile (PSYCHLOPS) and the Personal Questionnare(PQ)in drug and alcohol services in Portugal

One focus group with ten service users; eight who had completed the measures on entry to treatment only and two had also completed them at 7 months

✓

✓

✓

✓

✓

• Service users felt use of the PROMs had helped them reflect on their clinical situation

• Individualised measures were perceived to provide the freedom to talk about any topic, whether it was related to substances use or not

• Standardised PROMs raised awareness about the quantity of alcohol or drugs used

Hagelin et al. (2007)

Nurses use of the EORTC QLQC-30 during assessments within in-patient palliative care in Sweden

Questionnaire with closed and open ended questions to explore 26 nurses’ experiences of using the EORTC QLQC-30

✓

✓

✓

• Nurses perceived the EORTC QLQC-30 captured issues patients may not speak about verbally and provided a useful structure to their discussion with patients

• Nurses felt EORTC QLQC-30 was more useful when accompanied by a discussion with the patient and was a complement but not a substitute for other nursing assessments.

Mills et al. (2008;[71])

Completion of a weekly diary for 16 weeks comprising the EORTC-QLQC-30 by patients with inoperable lung cancer in three centres in Northern Ireland. No formal feedback of diary data to clinicians

RCT: diary completion but no feedback vs no completion. Primary outcomes: HRQoL measured by FACT-G. Structured questionnaire to assess use at 8 and 16 weeks. Semi-structured interviews with 7 patients who completed the diary.

✓

✓

✓

• Intervention group had worse total FACT-G scores compared to the control group

• 60% patients said it was useful to complete the diary but only 23% of patients reported explicitly sharing the diary with clinicians

• Reasons for not sharing included clinicians not asking about it, forgetting to take it to clinic and feeling they could tell clinicians hoe they felt without the diary.

Slater and Freeman [66]

Nurses use of the palliative care outcome scale (POS) in a day hospice in UK

One focus group with nine patients who has been using the POS for at least three months to explore their experiences

✓

✓

✓

• Patients felt completion of the POS helped them to reflect on their feelings

• Patients perceived the POS was useful in helping them share their concerns with nurses

• Some patients felt some of the questions were too distressing to answer

Slater and Freeman [77]

Nurses use of the palliative care outcome scale (POS) in a day hospice in UK

Focus group with eight nurses to explore their experiences of using the POS

✓

✓

✓

• Nurses perceived that patients may not report issues on the POS but reveal them through verbal interaction.

• Nurse felt talking to patients more beneficial than asking them to complete the POS.

• Nurses felt completing the POS could be distressing for patients who may not wish to confront their problems directly.

Hughes et al. [78]

Clinicians use of the palliative care outcome scale (POS) in the UK

Qualitative telephone interviews with a purposive sample of 22 people who had experience of using the POS in the routine care of patients;

✓

✓

✓

• Clinicians felt the POS was difficult to use when patients were too ill and perceived some patients found it intrusive or upsetting.

• Some clinicians found the questions ‘difficult to ask’ or found the responses ‘difficult to deal with’.

• Clinicians altered, reworded or omitted items to adapt the POS for use in local circumstances

Hughes et al. [79]

The use of the POS in non-specialist palliative care settings in the UK

Interviews with 13 members of staff and 3 patients to explore use of POS

✓

✓

✓

• Of 25 sites invited to implement the POS, 15 agreed but four subsequently withdrew.

• Anticipated recruiting 240 patients across 11 sites; 21 patients actually recruited.

• Participating sites perceived use of POS as a research exercise rather than part of routine practice.

• Some nurses reluctant to use the POS as because they were concerned it may raise issues they were ill-equipped to manage.

• Nurses expressed concerns that asking patients to complete the POS would ‘tar’ their relationship with the patient

Eischens et al. [86]

Eight nurses randomly assigned to implement either the McGill Quality of Life Questionnaire or the Hospice Quality of Life Index-Revised (HQLI) during patient assessments for one week in a hospice setting in the US. In the subsequent week they utilised the other PROM.

Eight nurses interviewed one week after PROMs administration to explore their experiences of use.

✓

✓

✓

• For the MQOL, nurses felt ‘a good rapport was essential for the patients to answer the questions truthfully’

• For the HQLI, nurses perceived it would be too overwhelming for patients.

• All nurses reported the MQOL had enabled them to identify an area of patient care they had overlooked but only one nurses reported the HQLI had provided new information about a patient.

Gamlen and Arber [84]

Specialist nurses use of the Symptoms and Concerns Checklist (SCC) during their first assessments with patients with cancer in a community setting in the UK

Semi structured interviews and non participant observation of six specialist nurses conducting first assessments with patients with cancer in the community using the SCC.

✓

✓

✓

• Nurses placed great importance on hearing the patient’s story in their own words as the words patients used provided insight into how they were coping.

• Some nurses felt that the SCC constrained the relationship building process with patients

• Nurses delayed completion of the form until they had developed rapport with patients

• SCC was useful to validate what nurses had gathered through verbal interaction and prompted them to explore some of the non-physical aspects of the patients’ experiences.

Annells and Koch [68]

Pilot study of the implementation of two PROMs during first assessments and follow up in a district palliative care service in Australia over a 40 week period. 59 patients randomly allocated to be assessed using either the McGill Quality of Life Questionnaire and the Client Generated Index.

Eight nurses made notes following each initial (n = 59) and follow up (n = 8) assessment and were interviewed. Notes and interviews were qualitatively analysed.

✓

✓

✓

✓

✓

• CGI was recommended as the most appropriate tool for use at first assessments but was not recommended for use at follow up

• Completion of the CGI encouraged patients to reflect on their life in relationship to their current situation, gave patients ‘permission to be emotional’ and allowed them to ‘tell their story’.

• Nurses perceived completion of the CGI provided new information that they would not have usually uncovered within their assessments.

Kane et al. (2017)

The use of the Integrated Palliative Care Outcomes Scale with 25 patients in a palliative care service for people with Chronic Heart Failure in Ireland

Four nurses worked in the clinic and were interviewed about their experiences of using the measures. Eighteen patients who completed the intervention were interviewed about their experiences of using the PROM

✓

✓

✓

• Nurses felt the IPOS provided a comprehensive review of patients needs, especially psychosocial concerns, and opened up a conversation

• Nurses had some concerns about the IPOS opening up conversations on topics they did not know how to deal with

• Patients felt IPOS gave them a vocabulary to explain their experiences and gave them permission to raise these with clinicians

• Patients felt PROMs completion helped them to reflect on their symptoms and what they could do to manage them

Krawczyk and Sawatzky (2017)

Palliative care team selected PROMs and PREMs to implement in a palliative care service including the Edmonton Symptom Assessment-Revised, the McGill Quality of Life Questionnaire, the Canadian Health Evaluation Project Lite Questionnaire. The PROMs were collected using a tablet with patients over a nine week period

Three focus group with three groups of clinicians, interviews with two nurses two weeks after the start of the project and at completion, interviews with three patients, 50 h of observations of nurses and patient use of PROMs over 9 week period

✓

✓

✓

• Nurses preferred to sit with patients while they completed the PROMs as this enabled them probe patients’ answers and engage in dialogue

• Patients felt the PROMs enabled them to articulate their feelings

• Nurses felt that using the PROMs as an object of ‘mutual focus’ created a shared space where conversations about patients experiences could occur, which in turn helped nurses to build relationships with patients.

• Nurses reported using some of the questions in their routine verbal interactions with patients.

Krawczyk et al. [24]

Palliative care team selected PROMs and PREMs to implement in a palliative care service including the Edmonton Symptom Assessment-Revised, the McGill Quality of Life Questionnaire, the Canadian Health Evaluation Project Lite Questionnaire. The PROMs were collected using a tablet with patients over a nine week period

Five focus groups with clinicians and 24 interviews with clinicians

•

Kettis-Linblad et al. [69]

Non-routine use of the SEIQoL-DW by oncologists caring for patients with gastro-intestinal cancer in two hospitals in Sweden.

Qualitative interviews with eight oncologists and 20 patients to explore their experiences of using the SEIQoL

✓

✓

✓

• PROMs completion provided an opportunity for self-reflection, which increased patients’ self-awareness and helped patients feel that the doctor is willing to listen to them.

• Doctors used the SEIQoL to increase their knowledge of the patient and some discussed the findings with patients.

• Doctors felt the SEIQoL gave a more complete picture of the patient

Chen et al. (2013)

Systematic review of quantitative studies evaluating the feedback of PROMs data in oncology settings

Developed a theory of change to inform search strategy, outcome indicators and inclusion and exclusion criteria. Carried out a narrative synthesis of included studies

✓

✓

✓

✓

• Included 27 studies; 16 RCTs, 2 before & after, 9 observational studies

• 23 studies examined the impact of PROMs feedback on patient provider communication; 21 studies reported some positive effect.

• 11 studies examined the impact of PROMs feedback on monitoring treatment response; all 11 studies reported a positive impact

• 15 studies examined impact on patient outcomes, 13 reported a positive effect.

Basch et al. (2016[20])

Weekly web based electronic symptom monitoring using National Cancer Institute’s Common Terminology Criteria for Adverse Events in patients receiving chemotherapy for cancer in one centre in the US.

Two arm RCT usual care vs web based PROMs collection & feedback;.

✓

✓

✓

• More patients in intervention arm (21%) experienced clinically significant improvement in HRQoL compared to usual care (11%)

• Fewer patients in intervention arm (28%) experienced clinically significant worsening in HRQoL compared to usual care (37%).

• Patients in intervention arm had higher median survival (31.2 months) compared to usual care (26 months)

Velikova et al. (2004)

Regular touch screen completion of EORTC QLQC-30 and HADs immediately prior to clinic visits for patients with cancer receiving chemotherapy in one centre in the UK.

Three arm RCT; PROMs completion & feedback vs PROMs completion alone vs no PROMs completion..

✓

✓

✓

• Intervention patients showed greater improvements in HRQoL compared to control arm but not the attention control arm.

• Number of symptoms discussed greater in intervention than control arm but no differences in number of non-specific functional issues discussed.

• Clinicians explicitly referred to HRQoL data in 66 out of the 103 (64%) intervention encounters.

• Improvements in patient well-being associated with explicit use of HRQoL data in the consultation.

Takeuchi et al. [93]

Regular touch screen completion of EORTC QLQC-30 and HADs immediately prior to clinic visits for patients with cancer receiving chemotherapy in one centre in the UK.

Secondary, longitudinal content analysis of tape recorded consultations in control and intervention arms of Velikova et al. (2004) RCT. Data from 198 patients who completed four consecutive consultations with one of 28 oncologists; in total 792 consultations were included in the analysis.

✓

✓

✓

• Difference in the number of symptoms discussed between control and intervention groups largest at first consultation.

• No difference in number of functional or psychosocial issues discussed between control and intervention arms

• Severity of symptoms predictive of whether discussed but no relationship between severity of functional problems and discussion.

• Patients predominantly initiated discussions about symptoms and functional problems.

Detmar et al. [88]

Paper and pencil completion of the EORTC-QLQC-30 immediately prior to clinic visits for patients with cancer receiving chemotherapy in one centre in the Netherlands.

Crossover RCT PROMs completion vs no PROMs completion.

✓

✓

✓

• Statistically significant difference in total number of HRQoL issues discussed in fourth consultation between intervention arm (mean 4.5) and control arms (mean 3.7).

• No differences in patient management between control and intervention arms

• Two of the eight subscales on the SF-36 showed statistically significantly better function in the intervention arm compared to the control arm.

Berry et al. [92]

Touch screen completion of the Electronic Self Report Assessment-Cancer (ESRA-C) immediately before two clinic visits by patients with cancer receiving chemotherapy in two centres in the US.

RCT comparing PROMs feedback vs usual care. Primary outcome was discussion of symptoms and quality of life issues (SQLI)

✓

✓

✓

• When symptoms were reported at ‘threshold’ (severe) levels, they were more likely to be discussed in the intervention than the control arm.

• Some symptoms were commonly discussed in both arms, irrespective of whether they were at threshold.

Detmar et al. [88]

No PROM used

Survey of 273 patients undergoing chemotherapy and 10 oncologists to explore their preferences and self-reported behaviour for raising and discussing issues

✓

• Patients’ and clinicians’ perceptions about who is responsible for initiating discussion about HRQoL issues within the consultation vary depending on the issue.

• Patients wanted to discuss and clinicians were willing to initiate discussions about physical symptoms.

• Fewer doctors saw it as their primary task to initiate discussion of emotional issues and none reported initiating a discussion about these issues by themselves.

Taylor et al. [94]

RCT comparing no PROMs completion with completion of EORTC QLQC-30 but no feedback to clinicians but data treated as a cohort study

Patient questionnaire to explore their preferences for discussion of issues. Content analysis of consultations between 212 patients who had four complete consultation recordings and 36 doctors to examine actual behaviour.

✓

• Clinicians and patients have different preferences for raising and discussing social and emotional issues.

• Both patients and doctors preferences indicated that they saw it as more appropriate to raise emotional rather than social functioning issues with doctors.

• In reality, social functioning was more likely to be discussed within the consultation than emotional functioning

Velikova et al. [98]

No PROM used

Eight focus groups with 31 patients and four focus groups with 16 clinicians to explore their views on the use of PROMs in clinical practice.

✓

• Clinicians perceived emotional issues are not routinely discussed in consultations with doctors.

• Patients felt it was the doctor’s role to treat the cancer and the nurse’s role to address emotional issues.

• Clinicians expressed concerns about PROMs raising topics that they could not treat

Absolom et al. [90]

No PROM used

Interviews with six clinical nurse specialists, eight oncologists, four surgeons and five ward sisters to explore current roles and responsibilities in detection and management of emotional distress (ED)

✓

• Detection and management of emotional distress was more often undertaken by nurses, and was seen as more clearly part of their remit

• Oncologists and surgeons prioritised cancer treatment and the management of ED was not considered a key part of their role

• Oncologists and surgeons felt able to address emotional distress related to cancer and its treatment but broader emotional problems were seen as the nurse’s role.

Greenhalgh et al. [30]

Regular touch screen completion of EORTC QLQC-30 and HADs immediately prior to clinic visits for patients with cancer receiving chemotherapy in one centre in the UK. Graphical representation of patients’ scores over time provided to oncologist during clinic visit.

Analysis of purposively selected 18 consultations from the intervention arm and four consultations from the attention control arm using conversation analysis to explore how clinicians referred to HRQoL data

✓

✓

✓

• Doctors had to reconcile patients’ verbal reports of symptoms and their PROMs scores.

• Patients invited to account and explain their PROMs scores

• If high PROMs scores was not a problem for the patient or was not related to cancer, conversation tended to be closed down.

Green et al. [22]

Routine collection of the Edmonton Symptom Assessment System (ESAS) across 14 regional cancer centres in Canada

Survey of 960 clinicians to assess their views of using the ESAS in routine patient care. Study reports findings from 353 nurses who responded

✓

✓

✓

✓

• 59% of nurses agreed ESAS improves the efficiency of their meeting with patients

• Free text comments also indicated nurses has concerns that ESAS may length the visit when patients report symptoms not related to cnacer

• 84% nurses agreed ESAS was a useful starting point to assess patient symptoms; free text comments emphasised it opened up a conversation about symptoms

Pereira et al. [26]

Routine collection of the Edmonton Symptom Assessment System (ESAS) across 14 regional cancer centres in Canada

Survey of 960 clinicians to assess their views of using the ESAS in routine patient care.

✓

✓

✓

✓

• 67% of physicians agreed or strongly agreed ESAS helps patients report their symptoms compared to 84% of PSO staff

• 43% of physicians agreed or strongly agreed that ESAS improves the efficiency of the meeting compared with 60% of nurses

• 67% of physicians agreed that ESAS acts as a useful starting point to assess symptoms compared to 95% of PSO staff.

• 79% of physicians indicated they always or often looked at patients’ ESAS scores compared with 89% of nurses.

• 61% of physicians said they always or often talked about ESAS symptoms to their patients compared to 85% of nurses.