Evidence of methodologic bias in the derivation of the Science Citation Index impact factor

Study objective: The 'impact factor' published in Science Citation Index (SCI) is widely used in the scientific community to measure the relative importance of a medical journal. In contrast to all other indicators of academic growth in emergency medicine, impact factors for emergency medicine journals have remained low and unchanged since the inception of the specialty. We wished to investigate this incongruity. Methods: We examined the methodology used to derive the SCI's journal impact factor. Results: The impact factor for journals is defined mathematically as the number of times a journal is cited over a period of time (the numerator) divided by the number of articles published by that journal during the same period (the denominator). Citation counts are derived from examination of all references contained in a subset of journals known as 'source' journals. No emergency medicine journals are included in this group. The only source of citations for emergency medicine journals is from journals outside of emergency medicine. This produces small numerators with relatively constant denominators, leading to low impact factors. Conclusion: The apparent failure of emergency medicine journals, as measured by the SCI impact factor, to keep pace with other indicators of academic development of the field is at least in part attributable to a methodologic bias inherent in the derivation of this factor.