- Open Access
Feasibility of collecting retrospective patient reported outcome measures (PROMs) in emergency hospital admissions
© The Author(s) 2018
- Received: 14 November 2017
- Accepted: 16 October 2018
- Published: 15 November 2018
Outcome of emergency admissions is usually limited to mortality with little attempt to capture the views of health status of survivors. This is because of the challenge of determining patient reported outcome measures (PROMs) for the period before their emergency admission. The aim was to assess the feasibility of collecting retrospective PROMs to capture the pre-admission health status of patients admitted as emergencies.
Prospective study of two cohorts: patients undergoing primary coronary angioplasty for acute ST elevation myocardial infarction (STEMI) in five hospitals and emergency laparotomy (EL) for gastro-intestinal conditions in 11 hospitals. Three rates were calculated: proportion of patients eligible for inclusion; proportion of eligible patients invited to participate; proportion of invitees who participated. Staff views were thematically analysed to understand factors that affected recruitment.
About 85% of patients were eligible of whom most were invited to participate (84% EL; 79% STEMI). The proportions of invitees agreeing to participate differed between STEMI (92%) and EL (72%), probably reflecting greater post-intervention morbidity in the latter.
Variation between hospitals was observed in the proportion deemed eligible (EL 72–97%; STEMI 63–100%), proportion invited (EL 60–93%; STEMI 71–96%) and the proportion of invitees agreeing to participate (EL 55–92%; STEMI 67–100%). While this might reflect case-mix differences between hospitals, it suggests there is scope for less well performing hospitals to improve their recruitment processes.
The extent to which this initial feasibility study was able to assess selection bias was limited to the age and sex of patients. There was no bias evident for EL patients but for STEMI, younger men were more likely to participate.
It appears to be feasible to collect retrospective PROMs from patients admitted unexpectedly as emergencies for the two conditions studied. The relevance of these findings to other causes of emergency admissions needs to be established. In addition, these findings justify the case for a large, multi-site study that could explore unresolved concerns about selection bias, particularly those arising from the clinical characteristics of patients. It would also enable estimates of the extent of variation in PROMs between hospitals to determine the usefulness of using PROMs in emergency admissions.
- Patient reported outcome measures
- Health status
- Health-related quality of life
- Feasibility, emergency admissions, STEMI, emergency laparotomy
In England, emergencies account for about 40% of all hospital admissions, with the number of admissions having increased by 47% over the last 15 years. Two-thirds of hospital beds are occupied by people admitted as emergencies and the cost is approximately £12.5 billion annually . There is concern about variations in outcomes between providers [2–4]. While quite reasonably this has largely focused on mortality, there is also a need to consider outcome in terms of the health status of those who survive. To date, few attempts have been made to use patient reported outcome measures (PROMs) to determine patients’ perception of any change in their health status. As the aim of healthcare is to restore a patient’s health to his or her full potential, it is desirable to be able to compare patients’ outcome with their health status before the sudden and unexpected event that led to an emergency admission. The use of PROMs would enable clinicians to review the impact of their care on individual patients and allow organisations, including regulators, to assess and compare the outcomes of different providers.
Using PROMs in emergency admissions presents the methodological challenge of how to capture a pre-event measure for such patients as pre-existing data are, inevitably, not available. A recent literature review  found strong agreement in elective patients between the PROM they reported before admission with their later recall of that pre-admission health status (via a retrospective PROM). This has been confirmed in England in a recent study of elective surgical patients . These findings suggest that a retrospective PROM can provide a means of obtaining baseline health status in the absence of a prospectively collected contemporary report. Assuming this is also true for emergency patients (something that inevitably can never be established through direct testing), it is important to know whether it would be feasible to collect retrospective PROMs in such patients and the optimal methods for achieving this.
Feasibility might differ from the situation with elective admissions because, unlike elective admissions, emergency patients are acutely unwell and may be distressed. In addition, the immediate clinical priority is their surgical or medical assessment and intervention. Thus, it would not be possible to collect a PROM until after initial treatment, during their recovery period some days later. Feasibility may also be influenced by the mode of administration and questionnaire design.
Only three studies have reported recruitment rates when using retrospective PROMs following emergency admissions. Two focused on trauma cases and one on acute lung injury. Gabbe and colleagues achieved 50% recruitment in trauma patients in two major Australian hospitals during their inpatient stay but boosted this to 77% by contacting them afterwards at home by mail and telephone . Toien and colleagues who sought consent while trauma patients were in hospital in Norway and then surveyed them by mail afterwards achieved 50% recruitment . And Gifford and colleagues reported 70% recruitment among survivors of acute lung injury in four major hospitals in USA .
The aim of this exploratory study was to assess the feasibility of capturing retrospective PROMs in emergency admissions for a common medical (primary coronary angioplasty for acute ST elevation myocardial infarction) and surgical (emergency laparotomy for gastro-intestinal system) reason in a representative sample of NHS hospitals. The primary objectives were to explore the three stages of recruitment: the proportion of emergency admissions that were eligible for inclusion; the proportion of eligible patients who were invited to participate by staff; and the proportion of patients invited who participated. The secondary objectives were: to determine the representativeness of recruited patients as regards their age and sex; and to compare recruitment rates in different hospitals to determine the potential maximum rate obtainable and the associated organisational factors.
Choice of conditions
The two clinical conditions were selected as both are the subject of a national clinical audit which aims to collect detailed clinical data from all cases. The National Emergency Laparotomy Audit (NELA) includes all patients over the age of 17 years undergoing an emergency laparotomy for gastrointestinal conditions in NHS hospitals in England and Wales . The Myocardial Ischaemia National Audit Project (MINAP) collects data on all patients with acute ST elevation myocardial infarction (STEMI) who undergo a primary coronary angioplasty [11, 12]. Patients who met the national clinical audit criteria and were alive at discharge were considered for inclusion in this study. Patients were excluded if: they were not literate in English; judged not to have sufficient cognitive ability; or were not resident in the UK.
A multi-site study was carried out to ensure there would be some variation in the detailed organisation of patient recruitment and data collection. This would allow us to gain insights into the relative merits of recruiting in different settings and with different personnel involved . For emergency laparotomy, 14 hospitals were selected on the basis of their high case ascertainment rates in NELA of which 13 agreed to participate. For STEMI patients, five primary angioplasty centres in London and the surrounding area were invited and all participated.
Sites were asked to recruit all eligible patients during a 15 week period. The study received ethics approval from South East Coast - Brighton & Sussex Research Ethics Committee (REC reference: 16/LO/2053) and it was incorporated in the NIHR Research Network Portfolio. Each site nominated someone to be the site lead (usually a consultant or senior research director) responsible for overseeing local data collection. Site leads then nominated study leads who undertook the data collection and liaised directly with one of us (EK) if any queries arose, completed a study log (see below), stored and returned the data. At some sites the site lead and study lead were the same person. Study leads at each site could delegate recruitment to appropriate members of the clinical team so the number of staff involved could vary.
Staff were provided with training in the form of video clips and written materials. These materials were developed by EK from prior experience of collecting a retrospective PROMs for elective patients in two cohort studies . Video materials were produced with the support of the University media team and research partners from the earlier study. EK also visited or held a telephone conference with staff at each site prior to the start of data collection (Appendix: Study Flow diagrams).
Patients were invited to participate once emergency medical and surgical treatments had been completed and as close to the discharge date as possible to ensure the immediate effects of the intervention (such as a general anaesthetic) were minimised. Clinical staff explained the study to patients and provided written information. Written consent was obtained from participating patients. Staff added a sticky label which included patients’ NHS numbers and some socio-demographic data (date of birth, sex, address). A questionnaire was completed by recruited patients once during their inpatient stay. Those impeded by physical disability or sensory impairment could be assisted by staff or family members reading aloud the questions and/or recorded responses on the questionnaire. They were cautioned to avoid influencing the patients’ views.
Each study lead was required to complete a log covering every patient who met the national clinical audit criteria during the recruitment period. Staff recorded whether a patient met the eligibility criteria for inclusion in the study and if they were invited to participate. The date of consent of participants was also recorded. Patients’ reasons for declining to participate were recorded if an explanation was offered without direct questioning.
The questionnaires (paper hardcopy) included demographic information, self-reported co-morbidities, a disease-specific PROM and a generic PROM. The questionnaires contained instructions asking patients to recall how they were 1 month before their current admission. A systematic review identified suitable PROMs with adequate psychometric properties. Clinicians were then consulted in an unstructured meeting (a formal consensus development method was not used) to determine the final choice. This included consideration of the length and likely burden on patients of instruments.
For emergency laparotomy, the Gastro-Intestinal Quality of Life Index (GIQLI), developed by Eypasch and colleagues was selected . It consists of 36 questions relating to the gastrointestinal system and the impact of symptoms and treatment on individuals’ physical, emotional and social status. It takes 5–10 min to complete and has good test-retest reliability (intra-class correlation coefficient = 0.92), and internal consistency (Cronbach’s alpha > 0.90). The GIQLI is the most commonly used validated PROM in studies investigating outcomes in emergency abdominal surgery .
For STEMI, the Seattle Angina Questionnaire (SAQ-7) is a 7 item health status measure for patients with coronary artery disease that has well-established validity, reliability, sensitivity to clinical change, and prognostic value [17, 18]. Scores range from 0 to 100, where higher scores indicate fewer symptoms and higher health-related quality of life. SAQ-7 has good domain coverage (symptom burden, functional status, and quality of life), psychometric properties (validity, sensitivity), feasibility to implement (questionnaire length, language availability, and cost to implement), and clinical interpretability (knowledge of how to interpret scores in a clinically meaningful way) .
Both groups completed a generic PROM, the EQ-5D-3 L. This has five items concerning the domains of mobility, usual activities, personal care, pain/discomfort and anxiety/depression. It takes up to 5 minutes to complete . For each of these questions, the respondent chooses from three responses indicating the level of their function. A multi-attribute utility score where death and perfect health are represented by 0 and 1 are calculated . Scores less than 0 are considered worse than death and 1 is the maximum score possible. The EQ-5D-3 L was used rather than the EQ-5D-5 L as the former is still the version used in the National PROMs Programme in England.
For each condition, three rates were calculated: the proportion of all admissions with the condition that staff considered met the eligibility criteria for inclusion; the proportion of eligible patients invited to participate by staff; and the proportion invited who participated. In addition, the representativeness of those participating was assessed by comparison with all those included in the national clinical audit, though this was only possible for age and sex as clinical data were not available. The performance of hospital sites was compared to establish the maximum possible rates that could be obtained.
At the completion of the study, information was sought from the site leads using a structured form administered by telephone interview or email. These observations, supplemented by a field diary kept by EK, were subjected to thematic analysis to identify the factors that facilitated and impaired patient participation to learn how data collection might be maximised.
Of the 13 hospitals that agreed to participate, 11 collected data for the full 15 week duration of the study. Two hospitals stopped after 1 month due to local staff changes and their data are not included in the analyses. Those two hospitals were the only ones where the site lead was a non-clinical audit manager.
In all 11 participating hospitals the site lead was either a consultant surgeon or anaesthetist. They took responsibility for identifying patients from their NELA database and ward lists and provided oversight of the data collection. In nine sites the study lead was a nurse (usually a research portfolio nurse). They invited and consented patients, and undertook the data collection on weekdays. In the other two sites doctors took on these tasks. Some sites had additional staff to support managing the study log, arranging paperwork and covering periods of leave.
Of the 466 eligible patients, 395 (85%) were invited to participate. The main reasons for not inviting patients was that the patient was discharged rapidly (e.g. transfer to another hospital, self-discharge) or discharged at weekends when staff collecting data were not at work.
Of the 395 invited, 268 (72%) patients agreed to participate and completed a questionnaire. Of the 127 who declined to participate, the most common reason recorded by staff was that they were feeling too tired to complete the questionnaire.
Emergency Laparotomy recruitment overall and by hospital (n = 11)
N1 Number of admissions discharged alive
N2 Number of eligible patients
N3 Number invited to take part
N4 Number participated
N2/N1 Percentage of admissions deemed eligible
N3/N2 Percentage of eligible patients invited
N4/N2 Percentage of eligible patients participating.
N4/N3 Percentage of invited patients participating.
N4/N1 Percentage of admissions participating
Patients who participated were representative of all admissions as regards sex (male 47% v 48%) and age (median 66 v 67 years) .
Primary angioplasty for STEMI
All five sites participated for the full study duration of 15 weeks. The site leads in four hospitals were the hospital or cardiology department research manager or director. In the other hospital a nurse consultant was site lead. The study lead responsible for recruiting patients and collecting data during weekdays in all sites was a nurse (primarily research portfolio nurses, with the support of ward nurses). Some sites had additional administrative research staff to support managing the study log and arranging paperwork.
Of the 547 eligible to participate, 432 (79%) were invited by staff to participate. The main reasons for not inviting patients was that the patient was discharged rapidly (e.g. transfer to another hospital, self-discharge) and those discharged at weekends or at night when staff collecting data were not at work. Of the 432 invited, 396 (92%) patients participated and completed a questionnaire. Of the 36 who declined to participate, most provided no reason.
STEMI recruitment overall and by hospital (n = 5)
N1 Number of admissions discharged alive
N2 Number of eligible patients
N3 Number invited to take part
N4 Number participated
N2/N1 Percentage of admissions deemed eligible
N3/N2 Percentage of eligible patients invited
N4/N2 Percentage of eligible patients participating.
N4/N3 Percentage of invited patients participating.
N4/N1 Percentage of admissions participating
Patients who participated were more likely to be male (79% v 72%) and slightly younger (median: males 60 v 63 years; females 67 v 71) than all those included in the national clinical audit .
Staff identified facilitators and obstacles at each stage of recruitment. .
Identification of eligible patients
Staff found identification of eligible EL patients easier if the site lead was also involved in the national clinical audit. Identification was facilitated by combining their NELA register, emergency theatre lists and consultants’ knowledge of patients. This was easiest in sites with a real time NELA register and electronic patient trackers. Similarly, for STEMI identification was aided by the existence of pathway activation records. Conversely, for EL the relocation of patients (such as from ITU to ward) could delay identification as a patient could be temporarily ‘lost’. This was rarely a problem for STEMI as patients were admitted to a designated ward or coronary care unit and rarely moved to other locations.
Inviting patients to participate
Timely identification of patients and their location was crucial to enable study leads to invite patients. The main reason that patients were not invited was because of missing the target period of 1–2 days before discharge. This was a particular problem at weekends. As many STEMI admissions stayed less than 48 h, patients admitted on a Friday would be discharged over the weekend and thus risk not being invited as study leads were not available. The site that managed to capture all patients (R) did not routinely discharge patients over the weekend. One proposed solution is to involve members of the ‘on-call’ clinical team at weekends.
An additional challenge with EL patients was predicting when this window of opportunity would occur or when discharge would occur as there was greater variation between patients. One way of coping with this with EL patients was for staff to invite them as soon as they felt there was an opportunity to speak to them, such as after stepping down from ITU to the ward.
Gaining agreement from patients to participate
Staff felt that patient participation was more likely if they were approached in an open and positive manner, explaining the purpose of the study clearly. Also, bringing in members of the clinical team directly involved in their care helped.
Patients attitudes about the reasons for PROMs, their health status and the extent to which they had come to terms with their emergency admission were factors that affected their agreement to participate. Patients understood and welcomed the value of PROMs when their purpose was explained by engaged staff.
Most patients were glad to be asked for their views. The perceived time involved affected some decisions. STEMI patients welcomed the brevity of the questionnaire and while some EL patients were initially perceived the questionnaire to be too long, once they had seen that the questions were straightforward to complete (closed rather than open), most agreed to participate.
The main reason patients declined was they did not feel well enough to complete a questionnaire. Acceptance was greater once patients had had time to come to terms with the significant medical events they had experienced. As staff, for ethical reasons, were not able to revisit a declined invitation when a patient felt better, there was a delicate balance needed between avoiding decline and missing the opportunity, such that they were discharged home already. Given that the speed of recovery varied between patients, it was difficult to make the right judgment. Staff tried to invite as close to discharge as possible even if that risked missing patients.
Patients can successfully be recruited to complete PROMs during their inpatient admission following significant emergency treatment (primary angioplasty and emergency laparotomy). Identification of relevant patients presented few difficulties, partly because the patients were also being included in a national clinical audit. It may prove to be more problematic if no such audit existed.
Of those patients admitted, 86% met the eligibility criteria to be invited to complete a PROM questionnaire about their pre-admission health status. Most of those deemed eligible were invited (85% emergency laparotomy; 79% STEMI). The main reasons for not inviting patients was that the patient was discharged rapidly (e.g. transfer to another hospital, self-discharge) or at weekends or out-of-hours when staff collecting data were not at work.
Agreement by patients to participate differed between the two conditions: 92% for STEMI patients but only 72% for emergency laparotomy. This probably reflected the greater post-intervention morbidity of the latter group. Despite the modest participation rate of laparotomy patients, they were representative of all such patients as regards age and sex, though might have differed in other respects (clinical severity, comorbidities etc). The observation that STEMI participants were more likely to be male and younger is of little concern given the very high participation rate among this group.
There was variation in the eligibility, invitation and participation rates between the hospitals. While some of this might reflect case-mix differences between hospitals (e.g. English literacy), these differences suggest that there is scope for less well performing hospitals to improve their recruitment processes. The reason for low eligibility in some sites - 72% for laparotomy in hospital A and 63% for STEMI in hospital Q - requires investigation to see if these rates are clinically justified. Similarly, low invitation rates (62% in hospital J and 69% in hospital N) suggest that overall recruitment could be enhanced given that other sites achieved invitation rates of over 90%.
The lower proportions of patients agreeing to participate in some hospitals (55% in hospital F for laparotomy and 67% in hospital Q for STEMI) may reflect case-mix differences but it might be because staff were less enthusiastic and effective in how they approached and invited patients.
Comparison with other studies
This is the first study in England to demonstrate the feasibility of collecting retrospective PROMs in emergency hospital admissions. The only previous studies to report on recruitment rates of emergency admissions involved either trauma patients [7, 8] or critical care survivors (not all of whom were emergency admissions to hospital) . Despite all three studies being concentrated in only 1–4 sites, in our multi-site study we achieved similar proportions of admissions participating (49% and 62%) to those previously reported (50–77%).
Strengths and limitations
Its strengths are that it considered both a common medical and surgical reason for an emergency admission, included both a disease-specific and generic PROM that varied in length, and we observed the recruitment performance in a wide range of 16 hospitals which differed in terms of annual volume of cases, teaching status and geographical location.
Despite this, some caution is needed in interpreting the generalisability of the results. First, the hospitals that participated in the emergency laparotomy study were those that were achieving a high case ascertainment rate in the national clinical audit so may have characteristics and a culture that is more likely to support the collection of PROMs. As regards the STEMI sites, all five were located in and around London (for greater ease of access for the research team) so may differ from other parts of the country. This may explain why patients were slightly younger than that seen nationally.
Second, it is possible that the response of staff and patients to collecting PROMs for the two clinical conditions selected might not be replicated with other reasons for emergency admission. This will need to be investigated in subsequent implementation of retrospective PROM collection.
Third, about 14% of patients were excluded from this study as ineligible (those not literate in English and those with cognitive impairment). The proportion excluded varied between hospitalsfor EL (3–21%) and for STEMI (0–37%). Given that such differences could introduce some selection bias, future comparisons of hospitals’ outcomes could be undermined if this was not taken into account. Investigations are needed to establish if such differences reflect the populations being served or the perceptions of staff as to the ability of patients to participate. In addition, recruitment of people not literate in English might be increased with the provision of questionnaires in other languages or translation services. For patients with cognitive impairment, the use of proxy-reported PROMs should be investigated.
Fourth, the extent of any selection bias was limited by the lack of data on patients’ clinical characteristics. A further feasibility study with larger samples of patients, linked to their clinical characteristics, is needed so sub-group analyses could quantify the extent of any selection bias. This would also permit investigation of any desirability bias affecting which patients agree to participate.
Finally, given that the only means of obtaining patient reported health status in emergency admissions is by the use of retrospective PROMs, there will always be some uncertainty as to the impact of recall bias and response shift. However, unless it is believed that these biases differ systematically between hospitals, there is little risk to the meaningfulness of hospital comparisons.
Implications for practice
While the overall rates of eligibility, invitation and participation were good, they could be improved if those hospitals with lower rates adopted some of the processes that higher performing hospitals used. There are several potential ways of increasing recruitment in all three stages:
integrate PROMs into the collection of data for the national clinical audit
automatic PROM reminders as part of national clinical audit
encourage the research nurses to participate in ward rounds to increase the support of ward nurses
involve other clinical staff at weekends (if discharges fall on this day)
engage all relevant clinical staff to ensure the aim and purpose of collecting recalled PROMs is understood
embed PROMs collection with the completion of discharge forms
reduce staff workload by simplifying the patient information sheets and consenting procedure.
invite patients to participate as close to the discharge date as possible
This initial feasibility study has suggested that it is feasible to collect retrospective PROMs from patients admitted unexpectedly as emergencies for the two conditions studied across a variety of types of hospitals in the NHS. The relevance of these findings to other causes of emergency admissions needs to be established. In addition, these findings justify the case for a large, multi-site study that includes clinical information on participants could explore the unresolved concerns about selection bias, particularly those arising from the clinical characteristics of patients. It would also enable estimates of the extent of variation in PROMs between hospitals to determine the usefulness of using PROMs in emergency admissions.
We thank: The patients and staff of 16 hospitals that participated in the study. NELA project team. Site and Study leads.
Collaborators: Mike Grocott, David Murray, David Saunders, Jose Lourtie, Steffen Petersen, Mark Vertue, Joanne Riches, Julie Saunders, Mervyn Andiapen, Jonathan Breeze, Amy Hoare, Alison Pottle, Paula Rogers, Claire Prendergast, Michael Lewis, Gill Pout, Patricia Dickens, James Kirkby-bott, Pauline Bartlett, Guy Titley, Emma Willett, Nina Barratt, Tanuja Shah, Kathleen Holding, Lianne Hufton, Veeranna Shatkar, Ruwan Weerakkody, Caron Baldwin, Sarah Hare, Annette Woods, Ewen Griffiths, Arlo Whitehouse, Jugdeep Dhesi, Jane Okello, Philip Braude, Karen Wilson, Kirsty Gibson, Abdul Quddus, Davina Ross-Anderson, Katherine MacGloin, Hasan Mukhtar, Kathryn Simpson, Kayleigh Gilbert.
The research and EK is funded by Economic & Social Research Council doctoral fellowship. Grant Reference: ES/J500021/1.
The research was supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care North Thames at Barts Health NHS Trust. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health.”
EK was the Principle Investigator of the study. NB is the Doctoral Supervisor. EK and NB wrote the paper. Both authors read and approved the final manuscript.
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and national research committee (NHS Health Research Authority) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. NHS ethical approval obtained from South East Coast - Brighton & Sussex Research Ethics Committee (REC reference: 16/LO/2053). Informed consent was obtained from all individual participants included in the study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- National Audit Office. (2013). Emergency admissions to hospital: Managing the demand. Hc, 739(October), 4. doi:https://doi.org/10.1016/S0140-6736(56)90286-0.View ArticleGoogle Scholar
- NHS England. (2016). A&E Attendances and Emergency Admissions. May 2016 Monthly Report. https://www.england.nhs.uk/statistics/wp-content/uploads/sites/2/2016/06/Monthly-AE-Report-May-16.pdf.
- Holt, P., Sinha, S., Ozdemir, B., Karthikesalingam, A., Poloniecki, J., & Thompson, M. (2014). Variations and inter-relationship in outcome from emergency admissions in England: A retrospective analysis of hospital episode statistics from 2005-2010. BMC Services Research, 14, 270.View ArticleGoogle Scholar
- The Royal College of Surgeons of England. (2011). Emergency surgery: Standards for unscheduled care. RCSENG professional standards and regulation. Retrieved from https://www.rcseng.ac.uk/library-and-publications/rcs-publications/docs/emergency-surgery-standards-for-unscheduled-care/
- Kwong, E., & Black, N. (2017). Retrospectively patient-reported pre-event health status showed strong association and agreement with contemporaneous reports. Journal of Clinical Epidemiology, 81, 22–32. https://doi.org/10.1016/j.jclinepi.2016.09.002.View ArticlePubMedGoogle Scholar
- Kwong, E., Neuburger, J., & Black, N. (2018). Agreement between retrospectively and contemporaneously collected patient-reported outcome measures (PROMs) in hip and knee replacement patients. Quality of Life Research. https://doi.org/10.1007/s11136-018-1823-6.View ArticleGoogle Scholar
- Gabbe, B. J., Cameron, P. a., Graves, S. E., Williamson, O. D., & Edwards, E. R. (2007). Preinjury status: Are orthopaedic trauma patients different than the general population? Journal of Orthopaedic Trauma, 21(4), 223–228. https://doi.org/10.1097/BOT.0b013e31803eb13c.View ArticlePubMedGoogle Scholar
- Tøien, K., Bredal, I. S., Skogstad, L., Myhren, H., & Ekeberg, O. (2011). Health related quality of life in trauma patients. Data from a one-year follow up study compared with the general population. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 19(1), 22. https://doi.org/10.1186/1757-7241-19-22.View ArticlePubMedPubMed CentralGoogle Scholar
- Gifford, J., Husain, N., Dinglas, V., Colantuoni, E., & Needham, D. M. (2010). Baseline quality of life before intensive care: A comparison of patient versus proxy responses. Critical Care Medicine, 38(3), 855–860. https://doi.org/10.1097/CCM.0b013e3181cd10c7.View ArticlePubMedPubMed CentralGoogle Scholar
- NELA Project Team. (2016). The second patient report of the National Emergency Laparotomy Audit, (July), 1–160. Retrieved from http://www.nela.org.uk/reports Google Scholar
- Chung, S., Gale, C. P., & Sundström, J. (2015). Comparison of hospital variation in acute myocardial infarction care and outcome between Sweden and United Kingdom: Population based cohort study using nationwide clinical registries. BMJ, 351, h3913.View ArticleGoogle Scholar
- National Institute for Clinincal Outcomes Research, Herrett, E., Smeeth, L., Walker, L., & Weston, C. (2010). Myocardial Ischaemia National Audit Project. Heart, 96(16), 1264–1267. https://doi.org/10.1136/hrt.2009.192328.View ArticleGoogle Scholar
- Runkel, P. J., & McGrath, J. E. (1973). Research on human behavior: A systematic guide to method. New York: Holt, Rinehart and Winston. American Educational Research Association.Google Scholar
- Kwong, E., & Black, N. (2017). Retrospectively collected patient reported outcomes show strong association and agreement with their contemporaneous reports. Journal of Clinical Epidemiology, 81, 22–32. https://doi.org/10.1016/j.jclinepi.2016.09.002.View ArticlePubMedGoogle Scholar
- Eypasch, E., Williams, J. I., Wood-Dauphinee, S., Ure, B. M., Schmülling, C., Neugebauer, E., & Troidl, H. (1995). Gastrointestinal quality of life index: Development, validation and application of a new instrument. The British Journal of Surgery, 82(2), 216–222. https://doi.org/10.1002/bjs.1800820229.View ArticlePubMedGoogle Scholar
- Stevens, D. J., Blencowe, N. S., McElnay, P. J., Macefield, R. C., Savović, J., Avery, K. N. L., & Blazeby, J. M. (2016). A systematic review of patient-reported outcomes in randomized controlled trials of unplanned general surgery. World Journal of Surgery, 40(2), 267–276. https://doi.org/10.1007/s00268-015-3292-1.View ArticlePubMedGoogle Scholar
- Spertus, J. A., Winder, J. A., Dewhurst, T. A., Deyo, R. A., McDonell, M., & Fihn, S. D. (1995). Development and evaluation of the Seattle angina questionnaire: A new functional status measure for coronary artery disease. Journal of American College of Cardiology, 25(2), 333–341.View ArticleGoogle Scholar
- Dougherty, C. M., Dewhurst, T., Nichol, W. P., & Spertus, J. (1998). Comparison of three quality of life instruments in stable angina pectoris: Seattle angina health survey (SF-36), and quality of life index. Cardiac Epidemiology, 51(7), 569–575.Google Scholar
- McNamara, R. L., Spatz, E. S., Kelley, T. A., Stowell, C. J., Beltrame, J., Heidenreich, P., & J. (2015). Standardized outcome measurement for patients with coronary artery disease: Consensus from the international consortium for health outcomes measurement (ICHOM). Journal of the American Heart Association., 4(5). doi:https://doi.org/10.1161/JAHA.115.001767.
- EuroQol Group. (1990). EuroQol - a new facility for the measurement of health-related quality of life. Health Policy, 16, 199–208.View ArticleGoogle Scholar
- Dolan, P., & Roberts, J. (2002). Modelling valuations for Eq-5d health states: An alternative model using differences in valuations. Medical Care, 40, 442–446.View ArticleGoogle Scholar