User experience of applying AMSTAR 2 to appraise systematic reviews of healthcare interventions: a commentary

Abstract

Background ‘A Measurement Tool to Assess Systematic Reviews, version 2’ (AMSTAR 2) is a validated 16-item scale designed to appraise systematic reviews (SRs) of healthcare interventions and to rate the overall confidence in their results. This commentary aims to describe the challenges with rating of the individual items and the application of AMSTAR 2 from the user perspective. Discussion A group of six experienced users (methodologists working in different clinical fields for at least 10 years) identified and discussed the challenges in rating of each item and the general use of AMSTAR 2 to appraise SRs. A group discussion was used to develop recommendations on how users could deal with the identified challenges. We identified various challenges with the content of items 2–16 and with the derivation of the overall confidence ratings on AMSTAR 2. These challenges include the need (1) to provide additional definitions (e.g., what constitutes major deviations from SR protocol on item 2), (2) to choose a rating strategy for multiple conditions on single items (e.g., how to rate item 5 if studies were selected in duplicate, but consensus between two authors was not reported), and (3) to determine rules for deriving the confidence ratings (e.g., what items are critical for such ratings). Based on these challenges we formulated specific recommendations for items 2–16 that AMSTAR 2 users could consider before applying the tool. Summary Our commentary adds to the existing literature by providing the first in-depth examination of the AMSTAR 2 tool from the user perspective. The identified challenges could be addressed by additional decision rules including definitions for ambiguous items and guidance for rating of complex items and derivation of confidence ratings. We recommend that a team consensus regarding such decision rules is required before appraisal procedure begins. Trial registration Not applicable.

Quality of systematic reviews on physiotherapy interventions for musculoskeletal disorders is critically low: a meta-epidemiological study

“…Retrospectively, our choices were consistent with the latest published recommendations. 22 We also found a tendency for a floor effect of the AMSTAR-2 tool, as described in a previous research: 23 thus, one might wonder whether indeed the SR-MA were of low quality or the instrument was too rigid and demanding. However, a high standard is definitively desirable and, after all, we did find some high-quality reviews, a finding that clearly indicate that such levels of methodological quality can be achieved.…”

A Systematic Literature Review on Inflammatory Markers in the Saliva of Patients with Multiple Sclerosis: A Cause or a Consequence of Periodontal Diseases

“…The quality and risk of bias of the included studies were evaluated using appropriate assessment tools. The Newcastle–Ottawa Scale (NOS) was used for observational studies [ 37 ], while the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were employed for systematic reviews and meta-analyses [ 36 ], using A Measurement Tool to Assess Systematic Reviews (AMSTAR 2) [ 38 , 39 ]. The quality assessment was conducted autonomously by the two authors, and any inconsistencies were handled through consensus.…”

“…The authors selected reviews of the literature that were pertinent by evaluating the critical components of the AMSTAR 2 tool [ 39 ]. These components include the comprehensiveness of the literature search, the justification for excluding certain studies, the evaluation of bias in the included studies, the appropriateness of the meta-analytic methods, the incorporation of bias assessment in the discussion of results, and the examination of potential publication bias [ 39 ].…”

“…The critical categories in the AMSTAR 2[39] tool for evaluating the quality of the systematic reviews chosen for determining eligibility.…”