Altmetrics are new metrics proposed as an alternative to the widely used journal impact factor and personal citation indices like the h-index. The term altmetrics was proposed in 2010, as a generalization of article level metrics, and has its roots in the twitter #altmetrics hashtag. Although altmetrics are often thought of as metrics about articles, they can be applied to people, journals, books, data sets, presentations, videos, source code repositories, web pages, etc. Altmetrics cover not just citation counts, but also other aspects of the impact of a work, such as how many data and knowledge bases refer to it, article views, downloads, or mentions in social media and news media.
Various websites and projects are calculating altmetrics, including ImpactStory, Altmetric.com, Plum Analytics, and CitedIn. Several publishers have started providing such information to readers, including BioMed Central, Public Library of Science, Frontiers, Nature Publishing Group, and Elsevier. For example, Elsevier announced in a press release to be "increasingly looking at additional metrics, including so called Altmetrics, as a measure of influence of journals and authors". Starting in March 2009, the Public Library of Science also introduced article-level metrics for all articles. Funders have started showing interest in alternative metrics, including the UK Medical Research Council. Altmetrics have been used in applications for promotion review by researchers. Furthermore, several universities, including the University of Pittsburgh are experimenting with altmetrics at an institute level.
However, it is also observed that an article needs little attention to jump to the upper quartile of ranked papers, suggesting that not enough sources of altmetrics are currently available to give a balanced picture of impact for the majority of papers.
Important in determining the relative impact of a paper, a service that calculates altmetrics statistics needs a considerably sized knowledge base. The following table shows the number of papers covered by services:
|Website||Number of papers|
Altmetrics are a very broad group of metrics, capturing various parts of impact a paper or work can have. A classification of altmetrics was proposed by ImpactStory in September 2012, and a very similar classification is used by the Public Library of Science:
- Viewed - HTML views and PDF downloads
- Discussed - journal comments, science blogs, Wikipedia, Twitter, Facebook and other social media
- Saved - Mendeley, CiteULike and other social bookmarks
- Cited - citations in the scholarly literature, tracked by Web of Science, Scopus, CrossRef and others
- Recommended - for example used by F1000Prime
One of the first alternative metrics to be used was the number of views of a paper. Traditionally, an author would wish to publish in a journal with a high subscription rate, so many people would have access to the research. With the introduction of web technologies it became possible to actually count how often a single paper was looked at. Typically, publishers count the number of HTML views and PDF views. As early as 2004, the BMJ published the number of views for its articles, which was found to be somewhat correlated to citations.
The discussion of a paper can be seen as a metric that captures the potential impact of a paper. Typical sources of data to calculate this metric include Facebook, Google+, Twitter, Science Blogs, and Wikipedia pages. The correlation between the mentions and likes and citation by primary scientific literature has been studied, and a slight correlation at best was found, e.g. for articles in PubMed. In 2008 the Journal of Medical Internet Research began publishing views and Tweets. These "tweetations" proved to be a good indicator of highly cited articles, leading the author to propose a "Twimpact factor", which is the number of Tweets it receives in the first seven days of publication, as well as a Twindex, which is the rank percentile of an article's Twimpact factor.
Besides Twitter and other streams, blogging has shown to be a powerful platform to discuss literature. Various platforms exist that keep track of which papers are being blogged about. Almetric.com is one that uses this information for calculating metrics, while other tools just report where discussion is happening, such as ResearchBlogging and Chemical blogspace. Moreover, platforms may even provide a formal way of ranking papers or recommending papers otherwise, such as Faculty of 1000 does.
Even more informative is the number of people that bookmark a paper. The idea behind this metric is that someone would not bookmark a paper of little influence to their own work. Providers of such information include science specific social bookmarking services such as CiteULike and Mendeley.
Besides the traditional metrics based on citations in scientific literature, for example as obtained from Google Scholar, CrossRef, PubMed Central, and Scopus, altmetrics also adopts citations in secondary and other knowledge sources. For example, ImpactStory counts the number of times a paper has been referenced by Wikipedia.
While the concept of altmetrics is questioned, the interpretation of altmetrics in particular is discussed. Proponents of altmetrics make clear that many of the metrics show influence or engagement, rather than impact on the progress of science. It should be noted that even citation-based metrics do not indicate if a high score implies a positive impact on science; that is, papers are also cited in papers that disagree with the cited paper, an issue for example addressed by the Citation Typing Ontology project.
The usefulness of metrics for estimating impact is controversial, but the community shows a clear need: funders demand measurables on the impact of their spending. Like other metrics, altmetrics are prone to self-citation, gaming, and other mechanisms to boost one's apparent impact. Additionally, it has been argued that the currently adopted metrics are suggestive of positive impact, while negative metrics are equally important.
However, it should be kept in mind that the metrics are only one of the outcomes of tracking how research is used. Even more informative than knowing how often a paper is cited, is which papers are citing it. That information allows researchers to see how their work is impacting the field (or not). Providers of metrics also typically provide access to the information from which the metrics were calculated. For example, Web of Science shows which are the citing papers, ImpactStory shows which Wikipedia pages are referencing the paper, and CitedIn shows which databases extracted data from the paper.
Altmetrics can be gamed: for example, likes and mentions can be bought. Altmetrics can be more difficult to standardize than citations. One example is the number of tweets linking to a paper where the number can vary widely depending on how the tweets are collected.
Another source of objections against altmetrics, or any metrics, is how universities are using metrics to rank their employees.
The score on each field does not directly tell you anything about the quality or impact of the paper. For example, a much discussed paper may merely be very controversial: papers discussed on Retraction Watch will typically get high altmetrics score, despite being retracted from the literature.
Altmetrics for more recent articles may be higher because of the increasing uptake of the social web and because articles may be mentioned mainly when they are published. As a result it is not fair to compare the altmetric scores of articles unless they have been published in the same year and, especially for fast increasing social web sites, at similar times in the same year.
The specific use cases and characteristics is an active research field in bibliometrics, providing the much needed data to measure the impact of altmetrics itself. Public Library of Science has an Altmetrics Collection and the Information Standards Quarterly recently published a special issue on altmetrics.
- J. Priem (@jasonpriem), I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures. Lately, I'm liking #altmetrics., 4:28 AM - 29 Sep 10, Tweet
- J. Priem, D. Taraborelli, P. Groth, C. Neylon, altmetrics: a manifesto, 2010, http://altmetrics.org/manifesto/
- Peter Binfield Article-Level Metrics at PLoS - what are they, and why should you care on YouTube, 2009
- E. Adie (@stew), @McDawg altmetrics could cover person, uni or journal level stats, non-trad contributions & items. ALMs a subset, fixes article as focus, 5:45 PM - 10 Aug 13, Tweet
- McFedries, P. (2012). "Measuring the impact of altmetrics [Technically Speaking]". IEEE Spectrum 49 (8): 28. doi:10.1109/MSPEC.2012.6247557.
- Galligan, F.; Dyas-Correia, S. (2013). "Altmetrics: Rethinking the Way We Measure". Serials Review 39: 56. doi:10.1016/j.serrev.2013.01.003.
- Liu, Jean; Euan Adie (8 July 2013). "New perspectives on article-level metrics: developing ways to assess research uptake and impact online". Insights 26 (2): 153. doi:10.1629/2048-7754.79. Retrieved 13 August 2013.
- "FAQ". ImpactStory. Retrieved 13 August 2013.
- "About us". Altmetric. Retrieved 13 August 2013.
- "About Us". Plum Analytics. Retrieved 19 August 2013.
- "Plum Analytics Metrics Overview". Plum Analytics. Retrieved 19 August 2013.
- "Included resources". CitedIn. Retrieved 16 August 2013.
- "Article-Level Metrics Information". PLoS ONE. 2005-07-01. Retrieved 2012-05-29.
- Press release: Article level metrics on nature.com, 2012, http://www.nature.com/press_releases/article-metrics.html
- Press release: Elsevier Announces 2012 Journal Impact Factor Highlights, 2012, http://www.marketwatch.com/story/elsevier-announces-2012-journal-impact-factor-highlights-2013-07-15
- Piwowar, Heather (2013). "Altmetrics: Value all research products". Nature. doi:10.1038/493159a.
- Viney, I. (2013). "Altmetrics: Research council responds". Nature 494 (7436): 176. doi:10.1038/494176c.
- Kwok, Roberta (2013). "Research impact: Altmetrics make their mark". Nature 500 (7463): 491. doi:10.1038/nj7463-491a.
- J. Kelly, Altmetric rankings, 22 August 2013, http://infiniflux.blogspot.ca/2013/08/altmetric-rankings.html
- Determined on 31 March 2014 using the altmetrics record for a specific paper.
- ImpactStory (@ImpactStory). "@egonwillighagen Sorry, we've got no automated reporting tool for the db size yet. But I just checked and it's just over 364k items." 2013-10-20, 8:10. Tweet.
- "A new framework for altmetrics". ImpactStory Blog. 2012-09-14. Retrieved 2013-08-13.
- Lin, J.; Fenner, M. (2013). "Altmetrics in Evolution: Defining and Redefining the Ontology of Article-Level Metrics". Information Standards Quarterly 25 (2): 20. doi:10.3789/isqv25no2.2013.04.
- Perneger, T. V (2004). "Relation between online "hit counts" and subsequent citations: Prospective study of research papers in the BMJ". BMJ 329 (7465): 546–7. doi:10.1136/bmj.329.7465.546. PMC 516105. PMID 15345629.
- Stefanie Haustein; Isabella Peters; Sugimoto, Cassidy R.; Mike Thelwall; Vincent Larivière (2013). "Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature". arXiv:1308.1838 [cs.DL].
- Eysenbach, Gunther (2011). "Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact". Journal of Medical Internet Research 13 (4): e123. doi:10.2196/jmir.2012. PMC 3278109. PMID 22173204.
- "FAQ: which metrics are measured?". ImpactStory. Retrieved 13 August 2013.
- Jump, Paul (23 August 2012). "Research Intelligence - Alt-metrics: fairer, faster impact data?". Times Higher Education. Retrieved 13 August 2013.
- Shotton, D. (2010). "CiTO, the Citation Typing Ontology". Journal of Biomedical Semantics 1 (Suppl 1): S6–S1. doi:10.1186/2041-1480-1-S1-S6. PMC 2903725. PMID 20626926.
- Mike Buschman; Andrea Michalek (April–May 2013). "Are Alternative Metrics Still Alternative?". asis&t Bulletin. Retrieved 21 August 2013.
- Cheung, M. K. (2013). "Altmetrics: Too soon for use in assessment". Nature 494 (7436): 176. doi:10.1038/494176d.
- J. Beall, Article-Level Metrics: An Ill-Conceived and Meretricious Idea, 2013, http://scholarlyoa.com/2013/08/01/article-level-metrics/
- Holbrook, J. B.; Barr, K. R.; Brown, K. W. (2013). "Research impact: We need negative metrics too". Nature 497 (7450): 439. doi:10.1038/497439a.
- Waagmeester, A.; Evelo, C. (2011). "Measuring impact in online resources with the CInumber (the CitedIn Number for online impact)". Nature Precedings. doi:10.1038/npre.2011.6037.1.
- Chamberlain, S. (2013). "Consuming Article-Level Metrics: Observations and Lessons". Information Standards Quarterly 25 (2): 4–2. doi:10.3789/isqv25no2.2013.02.
- David Colquhoun, How should universities be run to get the best out of people?, 2007
- Thelwall, M.; Haustein, S.; Larivière, V.; Sugimoto, C. R. (2013). "Do Altmetrics Work? Twitter and Ten Other Social Web Services". PLoS ONE 8 (5): e64841. doi:10.1371/journal.pone.0064841. PMC 3665624. PMID 23724101.
- Priem, J.; Groth, P.; Taraborelli, D. (2012). "The Altmetrics Collection". In Ouzounis, Christos A. PLoS ONE 7 (11): e48753. doi:10.1371/journal.pone.0048753. PMC 3486795. PMID 23133655.
- "Summer 2013 Volume 25, no. 2". Information Standards Quarterly. Retrieved 14 August 2013.