Impact factor

From Wikipedia, the free encyclopedia
Jump to: navigation, search

The impact factor (IF) of an academic journal is a measure reflecting the average number of citations to recent articles published in the journal. It is frequently used as a proxy for the relative importance of a journal within its field, with journals with higher impact factors deemed to be more important than those with lower ones. The impact factor was devised by Eugene Garfield, the founder of the Institute for Scientific Information. Impact factors are calculated yearly for those journals that are indexed in the Journal Citation Reports.

Contents

Calculation[edit]

In a given year, the impact factor of a journal is the average number of citations received per paper published in that journal during the two preceding years.[1] For example, if a journal has an impact factor of 3 in 2008, then its papers published in 2006 and 2007 received 3 citations each on average in 2008. The 2008 impact factor of a journal would be calculated as follows:

A = the number of times that articles published in that journal in 2006 and 2007, were cited by articles in indexed journals during 2008.
B = the total number of "citable items" published by that journal in 2006 and 2007. ("Citable items" are usually articles, reviews, proceedings, or notes; not editorials or letters to the editor.)
2008 impact factor = A/B.

(Note that 2008 impact factors are actually published in 2009; they cannot be calculated until all of the 2008 publications have been processed by the indexing agency.)

New journals, which are indexed from their first published issue, will receive an impact factor after two years of indexing; in this case, the citations to the year prior to Volume 1, and the number of articles published in the year prior to Volume 1 are known zero values. Journals that are indexed starting with a volume other than the first volume will not get an impact factor until they have been indexed for three years. Annuals and other irregular publications sometimes publish no items in a particular year, affecting the count. The impact factor relates to a specific time period; it is possible to calculate it for any desired period, and the Journal Citation Reports (JCR) also includes a 5-year impact factor.[2] The JCR shows rankings of journals by impact factor, if desired by discipline, such as organic chemistry or psychiatry.

Use[edit]

The impact factor is used to compare different journals within a certain field. The ISI Web of Knowledge indexes more than 11,000 science and social science journals.[3]

Criticisms[edit]

Numerous criticisms have been made of the use of an impact factor. For one thing, the impact factor might not be consistently reproduced in an independent audit.[4] There is a more general debate on the validity of the impact factor as a measure of journal importance and the effect of policies that editors may adopt to boost their impact factor (perhaps to the detriment of readers and writers). In short, there is some controversy about the appropriate use of impact factors.[5]

Validity as a measure of importance[edit]

The impact factor is highly discipline-dependent, perhaps due to the speed with which papers get cited in a field. (The percentage of total citations occurring in the first two years after publication varies highly among disciplines from 1 to 3 percent in the mathematical and physical sciences to 5–8 percent in the biological sciences.[6]) Accordingly, one cannot compare journals across disciplines on the basis of their relative impact factors. Furthermore, the impact factor is based on the arithmetic mean number of citations per paper, yet citation counts follow a Bradford distribution (i.e. a power law distribution) and therefore the arithmetic mean is a statistically inappropriate measure.[7] For example, about 90% of Nature's 2004 impact factor was based on only a quarter of its publications, and thus the importance of any one publication will be different from, and in most cases less than, the overall number.[8] Furthermore, the strength of the relationship between impact factors of journals and the citation rates of the papers therein has been steadily decreasing since articles began to be available digitally.[9]

This problem is exacerbated when the use of impact factors is extended to evaluate not only the journals, but the papers therein. The Higher Education Funding Council for England was urged by the House of Commons Science and Technology Select Committee to remind Research Assessment Exercise panels that they are obliged to assess the quality of the content of individual articles, not the reputation of the journal in which they are published.[10] (The effect of outliers can be seen in the case of the article "A short history of SHELX", which included this sentence: "This paper could serve as a general literature citation when one or more of the open-source SHELX programs (and the Bruker AXS version SHELXTL) are employed in the course of a crystal-structure determination". This article received more than 6,600 citations. As a consequence, the impact factor of the journal Acta Crystallographica Section A rose from 2.051 in 2008 to 49.926 in 2009, more than Nature (31.434) and Science (28.103).[11] The second most cited article in Acta Crystallographica Section A in 2008 had only 28 citations.[12]) Last, journal ranking lists constructed based on the impact factor only moderately correlate with journal ranking lists based on the results of an expert survey.[13]

Finally, impact factor is a journal metric and should not be used to assess an individual researcher or research institution.[14][15]

Editorial policies that affect the impact factor[edit]

A journal can adopt editorial policies that increase its impact factor.[16][17] For example, journals may publish a larger percentage of review articles which generally are cited more than research reports.[18] Therefore review articles can raise the impact factor of the journal and review journals will therefore often have the highest impact factors in their respective fields. Journals may also attempt to limit the number of "citable items", i.e. the denominator of the IF equation, either by declining to publish articles (such as case reports in medical journals) which are unlikely to be cited or by altering articles (by not allowing an abstract or bibliography) in hopes that Thomson Scientific will not deem it a "citable item". (As a result of negotiations over whether items are "citable", impact factor variations of more than 300% have been observed.[19]) (Interestingly, items which are not considered to be "citable" (and therefore do not enter into the denominator of the impact factor) can, if cited, still enter into the numerator (despite the ease with which such citations could be excluded)). This effect is hard to evaluate, for the distinction between editorial comment and short original articles is not always obvious. "Letters to the editor" might refer to either class. Another less insidious tactic is that a journal may publish a large fraction of its papers, or at least the papers expected to be highly cited, early in the calendar year. This gives those papers more time to gather citations. Several methods, not necessarily with nefarious intent, exist for a journal to cite articles in the same journal which will increase the journal's impact factor.[20][21]

Beyond editorial policies that may skew the impact factor, journals can take overt steps to game the system. For example, in 2007, the specialist journal Folia Phoniatrica et Logopaedica, with an impact factor of 0.66, published an editorial that cited all its articles from 2005 to 2006 in a protest against the "absurd scientific situation in some countries" related to use of the impact factor.[22] The large number of citations meant that the impact factor for that journal increased to 1.44. As a result of the increase, the journal was not included in the 2008 and 2009 Journal Citation Reports.[23] Coercive citation is a practice in which an editor forces an author to add spurious self-citations to an article before the journal will agree to publish it in order to inflate the journal's impact factor. A survey published in 2012 indicates that coercive citation has been experienced by one in five researchers working in economics, sociology, psychology, and multiple business disciplines, and it is more common in business and in journals with a lower impact factor.[24] However, cases of coercive citation have occasionally been reported for other scientific disciplines.[25]

Responses[edit]

Because "the impact factor is not always a reliable instrument", in November 2007 the European Association of Science Editors (EASE) issued an official statement recommending "that journal impact factors are used only – and cautiously – for measuring and comparing the influence of entire journals, but not for the assessment of single papers, and certainly not for the assessment of researchers or research programmes".[5]

In July 2008, the International Council for Science (ICSU) Committee on Freedom and Responsibility in the conduct of Science (CFRS) issued a "Statement on publication practices and indices and the role of peer review in research assessment", suggesting many possible solutions, e.g. considering a limit number of publications per year to be taken into consideration for each scientist, or even penalising scientists for an excessive number of publications per year.(e.g. more than 20).[26]

In February 2010, the Deutsche Forschungsgemeinschaft (German Research Foundation) published new guidelines to evaluate only articles and no bibliometric information on candidates to be evaluated in all decisions concerning "... performance-based funding allocations, postdoctoral qualifications, appointments, or reviewing funding proposals, [where] increasing importance has been given to numerical indicators such as the h-index and the impact factor".[27] This decision follows similar ones of the National Science Foundation (US) and the Research Assessment Exercise (UK).[citation needed]

Other measures of impact[edit]

Related indices[edit]

Some related values, also calculated and published by the same organization, are:

  • the immediacy index: the number of citations the articles in a journal receive in a given year divided by the number of articles published.
  • the cited half-life: the median age of the articles that were cited in Journal Citation Reports each year. For example, if a journal's half-life in 2005 is 5, that means the citations from 2001-2005 are half of all the citations from that journal in 2005, and the other half of the citations precede 2001.[28]
  • the aggregate impact factor for a subject category: it is calculated taking into account the number of citations to all journals in the subject category and the number of articles from all the journals in the subject category.

These measures apply only to journals, not individual articles or individual scientists (unlike the H-index). The relative number of citations an individual article receives is better viewed as citation impact.

It is, however, possible to examine the impact factor of the journals in which a particular person has published articles. This use is widespread, but controversial. Garfield warns about the "misuse in evaluating individuals" because there is "a wide variation from article to article within a single journal".[29] Impact factors have a large, but controversial, influence on the way published scientific research is perceived and evaluated.

PageRank algorithm[edit]

In 1976 a recursive impact factor that gives citations from journals with high impact greater weight than citations from low-impact journals was proposed.[30] Such a recursive impact factor resembles the PageRank algorithm of the Google search engine, though the original Pinski and Narin paper uses a "trade balance" approach in which journals score highest when they are often cited but rarely cite other journals. A number of subsequent authors have proposed related approaches to ranking scholarly journals.[31][32][33] In 2006, Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel also proposed using the PageRank algorithm.[34] From their paper (based on 2003 data):

Impact Factor (ISI IF) PageRank (PRw x 103) Combined (Y-factor x 102)
1 52.28 Annual Review of Immunology 17.46 Journal of Biological Chemistry 51.15 Nature
2 37.65 Annual Review of Biochemistry 16.51 Nature 47.72 Science
3 36.83 Physiological Reviews 16.38 Science 19.92 The New England Journal of Medicine
4 35.04 Nature Reviews Molecular Cell Biology 13.77 Proceedings of the National Academy of Sciences 14.36 Cell
5 34.83 The New England Journal of Medicine 8.90 Physical Review Letters 14.14 Proceedings of the National Academy of Sciences
6 33.95 Nature Reviews Cancer 5.93 Physical Review B 11.32 Journal of Biological Chemistry
7 33.06 CA – A Cancer Journal for Clinicians 5.72 The New England Journal of Medicine 8.73 Journal of the American Medical Association
8 30.98 Nature 5.40 The Astrophysical Journal 7.83 The Lancet
9 30.55 Nature Medicine 5.39 Cell 7.22 Nature Genetics
10 30.17 Annual Review of Neuroscience 4.90 Journal of the American Chemical Society 6.26 Physical Review Letters

The table shows the top 10 journals by Impact Factor, PageRank, and a modified system that combines the two, all based on 2003 data.[34]

The Eigenfactor is another PageRank-type measure of journal influence,[35] with rankings freely available online.[36]

Article level metrics and altmetrics[edit]

Alternative metrics (often on an article-level) are sometimes called altmetrics. Altmetrics include metrics of use such as views or mentionings in social media. For example, the BMJ published as early as in 2004 the number of views for the articles it published – a metric that is somewhat correlated to subsequent citations.[37] In 2008, the Journal of Medical Internet Research started publishing article-level metrics such as views and tweets ("tweetations") – the latter were found to be predictive for highly cited articles, leading the author to propose the "twimpact factor", which is defined as the number of tweets within the first 7 days of publication, as well as the twindex, which is the rank percentile of the twimpact factor of an article compared to similar articles within the same journal.[38] Starting in March 2009, the Public Library of Science also introduced article level metrics[39] on every article in all of their titles.

See also[edit]

References[edit]

  1. ^ "Introducing the Impact Factor". Retrieved 2009-08-26. 
  2. ^ "JCR with Eigenfactor". Retrieved 2009-08-26. 
  3. ^ "Web of Knowledge > Real Facts > Quality and Quantity". Retrieved 2010-05-05. 
  4. ^ Rossner M, Van Epps H, Hill E (17 December 2007). "Show me the data". Journal of Cell Biology 179 (6): 1091–2. doi:10.1083/jcb.200711140. PMC 2140038. PMID 18086910. 
  5. ^ a b "European Association of Science Editors (EASE) Statement on Inappropriate Use of Impact Factors". Retrieved 2012-07-23. 
  6. ^ Erjen van Nierop (2009). "Why do statistics journals have low impact factors?". Statistica Neerlandica 63 (1): 52–62. doi:10.1111/j.1467-9574.2008.00408.x. 
  7. ^ Joint Committee on Quantitative Assessment of Research (12 June 2008). "Citation Statistics" (PDF). International Mathematical Union. 
  8. ^ "Not-so-deep impact". Nature 435 (7045): 1003–1004. 23 June 2005. doi:10.1038/4351003b. PMID 15973362. 
  9. ^ Lozano, George A.; Larivière, Vincent; Gingras, Yves (2012). "The weakening relationship between the impact factor and papers' citations in the digital age". Journal of the American Society for Information Science and Technology 63 (11): 2140. doi:10.1002/asi.22731. 
  10. ^ "House of Commons – Science and Technology – Tenth Report". 2004-07-07. Retrieved 2008-07-28. 
  11. ^ Grant, Bob (21 June 2010). "New impact factors yield surprises". The Scientist. Retrieved 31 March 2011. 
  12. ^ What does it mean to be #2 in Impact? – Thomson Reuters Community
  13. ^ Serenko A, Dohan M (2011). "Comparing the expert survey and citation impact journal ranking methods: Example from the field of Artificial Intelligence". Journal of Informetrics 5 (4): 629–648. doi:10.1016/j.joi.2011.06.002. 
  14. ^ Seglen PO (1997). "Why the impact factor of journals should not be used for evaluating research". BMJ 314 (7079): 498–502. doi:10.1136/bmj.314.7079.497. PMC 2126010. PMID 9056804. 
  15. ^ EASE Statement on Inappropriate Use of Impact Factors, European Association of Science Editors, November 2007, retrieved 2013-04-13 .
  16. ^ Monastersky, Richard (14 October 2005). "The Number That's Devouring Science". The Chronicle of Higher Education. 
  17. ^ Arnold, Douglas N.; Fowler, Kristine K. (2011). "Nefarious Numbers". Notices of the American Mathematical Society 58 (3): 434–437. arXiv:1010.0278. Bibcode:2010arXiv1010.0278A. 
  18. ^ Garfield, Eugene (20 June 1994). The Thomson Reuters Impact Factor. Thomson Reuters. 
  19. ^ PLoS Medicine Editors (6 June 2006). "The Impact Factor Game". PLoS Medicine 3 (6): e291. doi:10.1371/journal.pmed.0030291. PMC 1475651. PMID 16749869. 
  20. ^ Agrawal A (2005). "Corruption of Journal Impact Factors". Trends in Ecology and Evolution 20 (4): 157. doi:10.1016/j.tree.2005.02.002. PMID 16701362. 
  21. ^ Fassoulaki A, Papilas K, Paraskeva A, Patris K (2002). "Impact factor bias and proposed adjustments for its determination". Acta Anaesthesiologica Scandinavica 46 (7): 902–5. doi:10.1034/j.1399-6576.2002.460723.x. PMID 12139549. 
  22. ^ Schuttea HK, Svec JG (2007). "Reaction of Folia Phoniatrica et Logopaedica on the Current Trend of Impact Factor Measures". Folia Phoniatrica et Logopaedica 59 (6): 281–285. doi:10.1159/000108334. PMID 17965570. 
  23. ^ "Journal Citation Reports – Notices". Archived from the original on 2010-05-15. Retrieved 2009-09-24. 
  24. ^ Wilhite, A. W.; Fong, E. A. (2012). "Coercive Citation in Academic Publishing". Science 335 (6068): 542–3. doi:10.1126/science.1212540. PMID 22301307. 
  25. ^ Smith, R. (1997). "Journal accused of manipulating impact factor". BMJ 314 (7079): 461. doi:10.1136/bmj.314.7079.461d. PMC 2125988. PMID 9056791. 
  26. ^ International Council for Science statement
  27. ^ DFG press release <http://www.dfg.de/en/service/press/press_releases/2010/pressemitteilung_nr_07/index.html>
  28. ^ Impact Factor, Immediacy Index, Cited Half-life
  29. ^ Eugene Garfield (June 1998). "The Impact Factor and Using It Correctly". Der Unfallchirurg 101 (6): 413–414. PMID 9677838. 
  30. ^ Gabriel Pinski and Francis Narin (1976). "Citation influence for journal aggregates of scientific publications: Theory with application to literature of physics". Information Processing & Management 12 (5): 297–312. doi:10.1016/0306-4573(76)90048-0. 
  31. ^ S. J. Liebowitz and J. P. Palmer. (1984). "Assessing the relative impacts of economics journals" (PDF). Journal of Economic Literature (American Economic Association) 22 (1): 77–88. JSTOR 2725228. 
  32. ^ I. Palacios-Huerta and O. Volij (2004). "The measurement of intellectual influence". Econometrica 72 (3): 963–977. doi:10.1111/j.1468-0262.2004.00519.x. 
  33. ^ Y. K. Kodrzycki and P. D. Yu (2006). "New approaches to ranking economics journals". B. E. Journal of Economics Analysis and Policy 5 (1). doi:10.2202/1538-0645.1520. 
  34. ^ a b Johan Bollen, Marko A. Rodriguez, and Herbert Van de Sompel.; Rodriguez; Van De Sompel (December 2006). "Journal Status". Scientometrics 69 (3): 1030. arXiv:cs.GL/0601030. Bibcode:2006cs........1030B. 
  35. ^ C. T. Bergstrom. (May 2007). "Eigenfactor: Measuring the value and prestige of scholarly journals". College & Research Libraries News 68 (5). 
  36. ^ eigenfactor.org
  37. ^ Perneger, T. V (2004). "Relation between online "hit counts" and subsequent citations: Prospective study of research papers in the BMJ". BMJ 329 (7465): 546–7. doi:10.1136/bmj.329.7465.546. PMC 516105. PMID 15345629. 
  38. ^ Eysenbach, Gunther (2011). "Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact". Journal of Medical Internet Research 13 (4): e123. doi:10.2196/jmir.2012. PMC 3278109. PMID 22173204. 
  39. ^ "Article-Level Metrics Information". PLoS ONE. 2005-07-01. Retrieved 2012-05-29. 

External links[edit]

Retrieved from "http://en.wikipedia.org/w/index.php?title=Impact_factor&oldid=559978718"