Skip to main content

Advertisement

Table 3 Risk of Bias Assessment (ROBIS) of the reviews

From: The facilitators and barriers to implementing patient reported outcome measures in organisations delivering health related services: a systematic review of reviews

  Author
Antunes, 2014 [22] Bantug, 2015b [36] Boyce, 2014 [23] Duncan, 2012 [35] Greenhalgh, 2017c [24] Howell, 2015d [37]
Domain 1: Concerns regarding specification of study eligibility criteria Low High- No description of the exclusion criteria High- No description of the exclusion criteria Low Low High- No description of the exclusion criteria
Domain 2: Concerns regarding methods used to identify and/or select studies Unclear- No information on whether more than one researcher supported the search process Low Unclear- No information on whether more than one researcher supported the search process Unclear- No information on whether more than one researcher supported the search process High- Sought to identify studies which supported/challenged programme theories rather than identify all the available literature High- No searching beyond electronic databases
Domain 3: Concerns regarding methods used to collect data and appraise studies Low High- No quality appraisal Low Low High- Did not synthesis all relevant studies nor conduct quality appraisal because of it being a realist synthesis High- Lack of information on which studies were included or description of the studies. No quality appraisal
Domain 4: Concerns regarding the synthesis and findings Low Low Low Low High- As did not include all relevant studies there are issues with the synthesis High- Concerns about the synthesis for example it was not clear which studies were included in the synthesis
Did the interpretation of findings address all of the concerns identified in Domains 1 to 4? Probably yes Probably no Probably yes Probably yes Probably yes Probably no
Was the relevance of identified studies to the review’s research question appropriately considered? Yes Yes Yes Yes Yes Probably yes
Did the reviewers avoid emphasizing results on the basis of their statistical significance?a Yes Yes Yes Yes Yes Probably no
Overall risk of bias in the review Low Unclear Low Low Unclear Unclear
  1. ‘Probably’- On the ROBIS there is the option to select ‘probably yes’, or ‘probably no’ in cases where the reviewer is not entirely sure. For example if it appeared that a review considered the relevance of the studies it included but the review did not include all the information on this to make the reviewer certain
  2. aThe ROBIS considers statistical significance but because the reviews are qualitative this question should be whether a review presented all its findings rather than cherry picking the results
  3. bPlease note that Bantug (2016) [36] was an integrative review so would not have undertaken some elements assessed by the ROBIS
  4. cPlease note that Greenhalgh (2017) [24] was a realist synthesis so would not have undertaken some elements assessed by the ROBIS such as including all relevant articles
  5. dPlease note that Howell (2015) [37] was a scoping review so would not have undertaken some elements assessed by the ROBIS such as quality appraisal