The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews

Foster, Alexis; Croot, Liz; Brazier, John; Harris, Janet; O’Cathain, Alicia

doi:10.1186/s41687-018-0072-3

Journal of Patient-Reported Outcomes

Table 3 Risk of Bias Assessment (ROBIS) of the reviews

From: The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews

	Author
	Antunes, 2014 [22]	Bantug, 2015^b [36]	Boyce, 2014 [23]	Duncan, 2012 [35]	Greenhalgh, 2017^c [24]	Howell, 2015^d [37]
Domain 1: Concerns regarding specification of study eligibility criteria	Low	High- No description of the exclusion criteria	High- No description of the exclusion criteria	Low	Low	High- No description of the exclusion criteria
Domain 2: Concerns regarding methods used to identify and/or select studies	Unclear- No information on whether more than one researcher supported the search process	Low	Unclear- No information on whether more than one researcher supported the search process	Unclear- No information on whether more than one researcher supported the search process	High- Sought to identify studies which supported/challenged programme theories rather than identify all the available literature	High- No searching beyond electronic databases
Domain 3: Concerns regarding methods used to collect data and appraise studies	Low	High- No quality appraisal	Low	Low	High- Did not synthesis all relevant studies nor conduct quality appraisal because of it being a realist synthesis	High- Lack of information on which studies were included or description of the studies. No quality appraisal
Domain 4: Concerns regarding the synthesis and findings	Low	Low	Low	Low	High- As did not include all relevant studies there are issues with the synthesis	High- Concerns about the synthesis for example it was not clear which studies were included in the synthesis
Did the interpretation of findings address all of the concerns identified in Domains 1 to 4?	Probably yes	Probably no	Probably yes	Probably yes	Probably yes	Probably no
Was the relevance of identified studies to the review’s research question appropriately considered?	Yes	Yes	Yes	Yes	Yes	Probably yes
Did the reviewers avoid emphasizing results on the basis of their statistical significance?^a	Yes	Yes	Yes	Yes	Yes	Probably no
Overall risk of bias in the review	Low	Unclear	Low	Low	Unclear	Unclear

‘Probably’- On the ROBIS there is the option to select ‘probably yes’, or ‘probably no’ in cases where the reviewer is not entirely sure. For example if it appeared that a review considered the relevance of the studies it included but the review did not include all the information on this to make the reviewer certain
^aThe ROBIS considers statistical significance but because the reviews are qualitative this question should be whether a review presented all its findings rather than cherry picking the results
^bPlease note that Bantug (2016) [36] was an integrative review so would not have undertaken some elements assessed by the ROBIS
^cPlease note that Greenhalgh (2017) [24] was a realist synthesis so would not have undertaken some elements assessed by the ROBIS such as including all relevant articles
^dPlease note that Howell (2015) [37] was a scoping review so would not have undertaken some elements assessed by the ROBIS such as quality appraisal

Back to article page