Citation analysis

From Wikipedia, the free encyclopedia
Jump to: navigation, search

Citation analysis is the examination of the frequency, patterns, and graphs of citations in articles and books.[1][2] It uses citations in scholarly works to establish links to other works or other researchers.[3] Citation analysis is one of the most widely used methods of bibliometrics. For example, bibliographic coupling and co-citation are association measures based on citation analysis (shared citations or shared references).

Automated citation indexing[4] has changed the nature of citation analysis research, allowing millions of citations to be analyzed for large-scale patterns and knowledge discovery. The first example of automated citation indexing was CiteSeer, later to be followed by Google Scholar.

Today citation analysis tools are easily available to compute various impact measures for scholars based on data from citation indices.[5][6][7] These have various applications, from the identification of expert referees to review papers and grant proposals, to providing transparent data in support of academic merit review, tenure, and promotion decisions. This competition for limited resources may lead to ethical questionable behavior to increase citations.[8][9]

A great deal of criticism has been made of the practice of naively using citation analyses to compare the impact of different scholarly articles without taking into account other factors which may affect citation patterns.[10] Among these criticisms, a recurrent one focuses on “field-dependent factors”, which refers to the fact that citation practices vary from one area of science to another, and even between fields of research within a discipline.[11]


While citation indexes were originally designed for information retrieval, they are increasingly used for bibliometrics and other studies involving research evaluation. Citation data is also the basis of the popular journal impact factor.

There is a large body of literature on citation analysis, sometimes called scientometrics, a term invented by Vasily Nalimov, or more specifically bibliometrics. The field blossomed with the advent of the Science Citation Index, which now covers source literature from 1900 on. The leading journals of the field are Scientometrics, Informetrics, and the Journal of the American Society of Information Science and Technology. ASIST also hosts an electronic mailing list called SIGMETRICS at ASIST.[12] This method is undergoing a resurgence based on the wide dissemination of the Web of Science and Scopus subscription databases in many universities, and the universally available free citation tools such as CiteBase, CiteSeerX, Google Scholar, and the former Windows Live Academic (now available with extra features as Microsoft Academic Search).

Legal citation analysis is a citation analysis technique for analyzing legal documents to facilitate the understanding of the inter-related regulatory compliance documents by the exploration the citations that connect provisions to other provisions within the same document or between different documents. Legal citation analysis uses a citation graph extracted from a regulatory document, which could supplement E-discovery - a process that leverages on technological innovations in big data analytics.[13][14][15][16]


In a 1965 paper, Derek J. de Solla Price described the inherent linking characteristic of the SCI as "Networks of Scientific Papers".[17] The links between citing and cited papers became dynamic when the SCI began to be published online. The Social Sciences Citation Index became one of the first databases to be mounted on the Dialog system[18] in 1972. With the advent of the CD-ROM edition, linking became even easier and enabled the use of bibliographic coupling for finding related records. In 1973, Henry Small published his classic work on Co-Citation analysis which became a self-organizing classification system that led to document clustering experiments and eventually an "Atlas of Science" later called "Research Reviews".

The inherent topological and graphical nature of the worldwide citation network which is an inherent property of the scientific literature was described by Ralph Garner (Drexel University) in 1965.[19]

The use of citation counts to rank journals was a technique used in the early part of the nineteenth century but the systematic ongoing measurement of these counts for scientific journals was initiated by Eugene Garfield at the Institute for Scientific Information who also pioneered the use of these counts to rank authors and papers. In a landmark paper of 1965 he and Irving Sher showed the correlation between citation frequency and eminence in demonstrating that Nobel Prize winners published five times the average number of papers while their work was cited 30 to 50 times the average. In a long series of essays on the Nobel and other prizes Garfield reported this phenomenon. The usual summary measure is known as impact factor, the number of citations to a journal for the previous two years, divided by the number of articles published in those years. It is widely used, both for appropriate and inappropriate purposes—in particular, the use of this measure alone for ranking authors and papers is therefore quite controversial.

In an early study in 1964 of the use of Citation Analysis in writing the history of DNA, Garfield and Sher demonstrated the potential for generating historiographs, topological maps of the most important steps in the history of scientific topics. This work was later automated by E. Garfield, A. I. Pudovkin of the Institute of Marine Biology, Russian Academy of Sciences and V. S. Istomin of Center for Teaching, Learning, and Technology, Washington State University and led to the creation of the HistCite [20] software around 2002.

Automatic citation indexing was introduced in 1998 by Lee Giles, Steve Lawrence and Kurt Bollacker [21] and enabled automatic algorithmic extraction and grouping of citations for any digital academic and scientific document. Where previous citation extraction was a manual process, citation measures could now scale up and be computed for any scholarly and scientific field and document venue, not just those selected by organizations such as ISI. This led to the creation of new systems for public and automated citation indexing, the first being CiteSeer (now CiteSeerX, soon followed by Cora, which focused primarily on the field of computer science and information science. These were later followed by large scale academic domain citation systems such as the Google Scholar and Microsoft Academic. Such autonomous citation indexing is not yet perfect in citation extraction or citation clustering with an error rate estimated by some at 10% though a careful statistical sampling has yet to be done. This has resulted in such authors as Ann Arbor, Milton Keynes, and Walton Hall being credited with extensive academic output.[22] SCI claims to create automatic citation indexing through purely programmatic methods. Even the older records have a similar magnitude of error.

Citation analysis for legal documents[edit]

Citation analysis for legal documents is an approach to facilitate the understanding and analysis of inter-related regulatory compliance documents by exploration of the citations that connect provisions to other provisions within the same document or between different documents. Citation analysis uses a citation graph extracted from a regulatory document, which could supplement E-discovery - a process that leverages on technological innovations in big data analytics.[23][15][16]

Issues raised by electronic publishing[edit]

Due to the unprecedented growth of electronic resource (e-resource) availability, one of the questions currently being explored is, "how often are e-resources being cited in my field?"[24] For instance, there are claims that on-line access to computer science literature leads to higher citation rates,[25] however, humanities articles may suffer if not in print.

See also[edit]

Methods of citation analysis for document similarity computation[edit]


  1. ^ Rubin, Richard (2010). Foundations of library and information science (3rd ed.). New York: Neal-Schuman Publishers. ISBN 978-1-55570-690-6. 
  2. ^ Garfield, E. Citation Indexing - Its Theory and Application in Science, Technology and Humanities Philadelphia:ISI Press, 1983.
  3. ^ "Dimension of Citation Analysis". Retrieved 1 July 2012.  by Loet Leydesdorff and Olga Amsterdamska
  4. ^ Giles, C. Lee; Bollacker, Kurt D.; Lawrence, Steve (1998), "CiteSeer: an automatic citation indexing system.", Digital libraries 98 : the Third ACM Conference on Digital Libraries, June 23–26, 1998, Pittsburgh, PA (New York: Association for Computing Machinery): 89–98, doi:10.1145/276675.276685, ISBN 0-89791-965-3, retrieved July 7, 2011 
  5. ^ Examples include subscription-based tools based on proprietary data, such as Web of Science and Scopus, and free tools based on open data, such as Scholarometer by Filippo Menczer and his team.
  6. ^ Kaur, Jasleen; Diep Thi Hoang; Xiaoling Sun; Lino Possamai; Mohsen JafariAsbagh; Snehal Patil; Filippo Menczer (2012). "Scholarometer: A Social Framework for Analyzing Impact across Disciplines". PLOS ONE 7 (9): e43235. doi:10.1371/journal.pone.0043235. 
  7. ^ Hoang, D.; Kaur, J. and Menczer, F. (2010), "Crowdsourcing Scholarly Data", Proceedings of the WebSci10: Extending the Frontiers of Society On-Line, April 26-27th, 2010, Raleigh, NC: US 
  8. ^ Anderson, M.S. van; Ronning, E.A. van; de Vries, R.; Martison, B.C. (2007). "The perverse effects of competition on scientists’ work and relationship". Science and Engineering Ethics 4 (13): 437–461. doi:10.1007/s11948-007-9042-5. 
  9. ^ Wesel, M. van (2015). "Evaluation by Citation: Trends in Publication Behavior, Evaluation Criteria, and the Strive for High Impact Publications". Science and Engineering Ethics. doi:10.1007/s11948-015-9638-0. 
  10. ^ Bornmann, L., & Daniel, H. D. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45-80.
  11. ^ Anauati, Maria Victoria and Galiani, Sebastian and Gálvez, Ramiro H., Quantifying the Life Cycle of Scholarly Articles Across Fields of Economic Research (November 11, 2014). Available at SSRN:
  12. ^ "The American Society for Information Science & Technology". The Information Society for the Information Age. Retrieved 2006-05-21. 
  13. ^ [1][dead link]
  14. ^ Mohammad Hamdaqa and A. Hamou-Lhadj, "Citation Analysis: An Approach for Facilitating the Understanding and the Analysis of Regulatory Compliance Documents", In Proc. of the 6th International Conference on Information Technology, Las Vegas, USA
  15. ^ a b "E-Discovery Special Report: The Rising Tide of Nonlinear Review". Hudson Global. Retrieved 1 July 2012.  by Cat Casey and Alejandra Perez
  16. ^ a b "What Technology-Assisted Electronic Discovery Teaches Us About The Role Of Humans In Technology - Re-Humanizing Technology-Assisted Review". Forbes. Retrieved 1 July 2012. 
  17. ^ Derek J. de Solla Price (July 30, 1965). "Networks of Scientific Papers" (PDF). SCIENCE 149 (3683): 510–515. doi:10.1126/science.149.3683.510. PMID 14325149. 
  18. ^ "Dialog, A Thomson Business". "Dialog invented online information services". Retrieved 2006-05-21. 
  19. ^
  20. ^ Eugene Garfield, A. I. Pudovkin, V. S. Istomin (2002). "Algorithmic Citation-Linked Historiography—Mapping the Literature of Science". Presented the ASIS&T 2002: Information, Connections and Community. 65th Annual Meeting of ASIST in Philadelphia, PA. November 18–21, 2002. Retrieved 2006-05-21. 
  21. ^ C.L. Giles, K. Bollacker, S. Lawrence, "CiteSeer: An Automatic Citation Indexing System," DL'98 Digital Libraries, 3rd ACM Conference on Digital Libraries, pp. 89-98, 1998.
  22. ^ Postellon DC (March 2008). "Hall and Keynes join Arbor in the citation indexes". Nature 452 (7185): 282. doi:10.1038/452282b. PMID 18354457. 
  23. ^ Hamdaqa, M.; A Hamou-Lhadj (2009). Citation Analysis: An Approach for Facilitating the Understanding and the Analysis of Regulatory Compliance Documents. Las Vegas, NV: IEEE. pp. 278–283. doi:10.1109/ITNG.2009.161. ISBN 978-1-4244-3770-2. 
  24. ^ Zhao, Lisa. "How Librarian Used E-Resources--An Analysis of Citations in CCQ." Cataloging & Classification Quarterly 42(1) (2006): 117-131.
  25. ^ Lawrence, Steve. Free online availability substantially increases a paper's impact. Nature volume 411 (number 6837) (2001): 521. Also online at