Cochrane Methods

Methodology Register

Background: Systematic reviews, including Cochrane systematic reviews, are an important resource for evidence-based healthcare. To our knowledge, the availability and quality of systematic reviews in eyes and vision has not been previously assessed formally. Objectives: To identify and characterize published eyes and vision systematic reviews available through searching major medical bibliographic databases; to compare Cochrane with non-Cochrane reviews; and to assess glaucoma reviews in depth. Methods: Search strategy: Two of us (CT and BA) developed a search strategy using eyes and vision keywords derived from 15 controlled health and medical vocabularies in the Unified Medical Language System thesaurus tool. We combined eyes and vision terms with those developed to identify systematic reviews in PubMed. In November 2006, we searched PubMed and The Cochrane Library, and in March 2007, we searched EMBASE. Eligibility: Eligible reports were systematic reviews, defined as full text review articles using a methodology that appeared to include a clearly formulated research question, explicit methods to identify primary studies, and predetermined inclusion and exclusion criteria. We included reviews with and without meta-analyses of primary studies' results. Systematic reviews were eligible if they related to the etiology, epidemiology, prevention, diagnosis, therapeutic intervention, practice patterns, economic evaluation, or health care utilization of eye diseases or visual impairment in humans. We excluded systematic reviews that evaluated only animal or in vitro studies, and reports that surveyed the literature to review criteria for disease diagnosis, prognosis, or treatment. Eligibility Assessment, Classification, and Quality Assessment: Two of us (TL and RWS) independently reviewed all unique retrieved citations for eligibility. We classified records by publication year, eye condition, and Cochrane status (Cochrane or non-Cochrane). We searched PubMed and EMBASE by author and keyword for co-publications of Cochrane reviews. We retrieved full text reports of glaucoma reviews to determine final eligibility; we assessed their methodological quality by using a form adapted from the Critical Appraisal Skills Program (CASP), evaluating 9 major quality parameters with 18 items. Discrepancies between authors were resolved by discussion. Results: PubMed and The Cochrane Library searches identified 2,418 unique records. Three hundred and sixty three reports fulfilled initial eligibility criteria, of which 51 were Cochrane systematic reviews. The number of systematic reviews increased over 10-fold from 1992 (n = 4) to 2005 (n = 60). Although many topics were covered, almost half of the reviews were associated with common eye conditions. We found journal co-publications for 19.6% (10/51) of Cochrane reviews. We are currently reviewing the results of the EMBASE search. Quality Assessment: We assessed the quality of 39 of 49 glaucoma systematic reviews (full text unavailable, n = 5; multiple publications, n = 3; cost-effectiveness review, n = 2). Twenty seven (69.2%) authors reported using pre-determined eligibility criteria, 25 (64.1%) searched more than one database, and 23 (59.0%) evaluated the quality of included studies. Two or more authors performed review tasks in less than half of the reviews. A majority of reviewers searched MEDLINE, about half also searched EMBASE and/or The Cochrane Library. Less than half searched for non-English language or unpublished studies. Authors of 16 reviews fulfilled all methodological criteria. Cochrane reviews fulfilled all methodological criteria more often (8 of 8 Cochrane reviews) than non-Cochrane reviews (8 of 31 non-Cochrane reviews). Conclusions: Our results revealed an increase in the application of systematic review methodology to assess the evidence in the eyes and vision literature in recent years. Twenty-percent of Cochrane reviews had co-publication in journals. Enormous challenges remain for achieving high quality systematic reviews. Many ocular conditions are only sparsely covered and review methodology can be improved.