Cochrane Methods

Methodology Register

Objective: There is general concern whether data that have not undergone peer review, such as conference abstracts, should be included in meta-analyses and systematic reviews in the absence of a full report. The aim of this study is to assess the concordance of data between conference abstracts of randomized controlled trials (RCTs) and their subsequent full reports, to estimate the reliability of abstract data. Methods: Using 1988 and 1989 abstract volumes of the Association for Research in Vision and Ophthalmology and the American Academy of Ophthalmology and contact with the authors, we identified abstracts and subsequent full reports of RCTs. Information of use to those conducting meta-analysis, including number of units randomized, continuous and discrete outcome data, p-values, and outcome direction, was extracted from both the abstracts and full reports. A disagreement was defined as any difference in reported values between the two sources of data Results: Data were gathered from 62 pairs of abstracts and corresponding full reports. Thirty-five percent (14/40) of the time, abstracts that clearly included data on the number randomized disagreed with the full report on this number. Reasons for disagreement on the number randomized included misrepresentation of the number analyzed as the number randomized (n=6), presentation of interim results in the abstract (n=5), and other reasons (n=3). The number randomized reported in the abstract was almost always lower than that in the full report. The stated outcome direction differed between abstracts and full reports 9% (4/44) of the time. Data on continuous outcomes, discrete outcomes, and p-values differed 50% (15/30), 40% (6/15), and 10% (3/30) of the time, respectively. Only rarely were the differences in results large, and it was not clear whether the use of abstract results over full report results would affect a summary measure in a meaningful way. Conclusions: Disagreements between data in conference abstracts and corresponding full reports occurred often. Though the differences in value were generally not large, those conducting metaanalysis should contact the original authors to confirm abstract data when there is no full report. OTHER PUBLICATIONS OF THIS RESEARCH: Chokkalingam A, Scherer R, Dickersin K. Agreement of data in abstracts compared to full publications. Controlled Clinical Trials 1998;19:S61-2. Chokkalingam A, Scherer R, Dickersin K. Agreement of data in abstracts compared to full publications [abstract] Controlled Clinical Trials 1998;19(3S):61-2S.