Jump to content

Google Books Ngram Viewer

From Wikipedia, the free encyclopedia

This is an old revision of this page, as edited by 72.14.228.1 (talk) at 08:09, 17 January 2014. The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

The Google Ngram Viewer is a phrase-usage graphing tool developed by Jon Orwant and Will Brockman of Google, and charts the yearly count of selected n-grams (letter combinations)[n] or words and phrases,[1][2] as found in over 5.2 million books digitized by Google Inc (up to 2008).[3][4] The words or phrases (or ngrams) are matched by case-sensitive spelling, comparing exact uppercase letters,[2] and plotted on the graph if found in 40 or more books during each year (of the requested year-range).[5] The Ngram tool was released in mid-December 2010.[1][3]

The word-search database was created by Google Labs, based originally on 5.2 million books, published between 1500 and 2008, containing 500 billion words[6] in American English, British English, French, German, Spanish, Russian, Hebrew, and Chinese.[1] Italian words are counted by their use in other languages. A user of the Ngram tool has the option to select among the source languages for the word-search operations.[7]

Researchers have analysed the Google Ngram database of books written in American or British English discovering interesting results. Amongst them, they found correlations between the emotional output and significant events in the 20th century such as the World War II.[8]

Operation and restrictions

Commas delimit user-entered search-terms, indicating each separate word or phrase to find.[5] The Ngram Viewer returns a plotted line chart within seconds of the user pressing the Enter key or the "Search" button on the screen.

As an adjustment for more books having been published during some years, the data is normalized, as a relative level, by the number of books published in each year.[5]

Google populated the database from over 5 million books published up to 2008. Accordingly, as of May  2012, no data will match beyond the year 2008. Due to limitations on the size of the Ngram database, only matches found in over 40 books are indexed in the database; otherwise the database could not have stored all possible combinations.[5]

Typically, search-terms cannot end with punctuation, although a separate full stop, or period, can be searched.[5] Also, an ending question mark (as in "Why?") will cause a 2nd search for the question mark separately.[5]

Omitting the periods in abbreviations will allow a form of matching, such as using "R M S" to search for "R.M.S." versus "RMS".

Corpora

The corpora used for the search are composed of total_counts, 1-grams, 2-grams, 3-grams, 4-grams, and 5-grams files for each language. The file format of each of the files is tab-separated data. Each line has the following format: [9]

  • total_counts file
year TAB match_count TAB page_count TAB volume_count NEWLINE
  • Version 1 ngram file (generated in July 2009)
ngram TAB year TAB match_count TAB page_count TAB volume_count NEWLINE
  • Version 2 ngram file (generated in July 2012)
ngram TAB year TAB match_count TAB volume_count NEWLINE

The Google Ngram Viewer uses match_count to plot the graph.

As an example, a word "Wikipedia" from the Version 2 file of the English 1-grams is stored as follows:[10]

ngram year match_count volume_count
Wikipedia 1904 1 1
Wikipedia 1912 11 1
Wikipedia 1924 1 1
Wikipedia 1925 11 1
Wikipedia 1929 11 1
Wikipedia 1943 11 1
Wikipedia 1946 11 1
Wikipedia 1947 11 1
Wikipedia 1949 11 1
Wikipedia 1951 11 1
Wikipedia 1953 22 2
Wikipedia 1955 11 1
Wikipedia 1958 1 1
Wikipedia 1961 22 2
Wikipedia 1964 22 2
Wikipedia 1965 11 1
Wikipedia 1966 15 2
Wikipedia 1969 33 3
Wikipedia 1970 129 4
Wikipedia 1971 44 4
Wikipedia 1972 22 2
Wikipedia 1973 1 1
Wikipedia 1974 2 1
Wikipedia 1975 33 3
Wikipedia 1976 11 1
Wikipedia 1977 13 3
Wikipedia 1978 11 1
Wikipedia 1979 112 12
Wikipedia 1980 13 4
Wikipedia 1982 11 1
Wikipedia 1983 3 2
Wikipedia 1984 48 3
Wikipedia 1985 37 3
Wikipedia 1986 6 4
Wikipedia 1987 13 2
Wikipedia 1988 14 3
Wikipedia 1990 12 2
Wikipedia 1991 8 5
Wikipedia 1992 1 1
Wikipedia 1993 1 1
Wikipedia 1994 23 3
Wikipedia 1995 4 1
Wikipedia 1996 23 3
Wikipedia 1997 6 1
Wikipedia 1998 32 10
Wikipedia 1999 39 11
Wikipedia 2000 43 12
Wikipedia 2001 59 14
Wikipedia 2002 105 19
Wikipedia 2003 149 53
Wikipedia 2004 803 285
Wikipedia 2005 2964 911
Wikipedia 2006 9818 2655
Wikipedia 2007 20017 5400
Wikipedia 2008 33722 6825

The graph plotted by the Google Ngram Viewer using this data is here.

See also

References

[n] - An "ngram" is a sequence of letters of any length, which could be a word, a misspelling, a phrase or gibberish.[6]
  1. ^ a b c "Google Ngram Database Tracks Popularity Of 500 Billion Words" Huffington Post, 17 December 2010, webpage: HP8150.
  2. ^ a b "Google Ngram Viewer - Google Books", Books.Google.com, May 2012, webpage: G-Ngrams.
  3. ^ a b "Google's Ngram Viewer: A time machine for wordplay", Cnet.com, 17 December 2010, webpage: CN93.
  4. ^ "A Picture is Worth 500 Billion Words – By Rusty S. Thompson", HarrisburgMagazine.com, 20 September 2011, webpage: HBMag20.
  5. ^ a b c d e f "Google Ngram Viewer - Google Books" (Information), Books.Google.com, December 16, 2010, webpage: G-Ngrams-info: notes bigrams and use of quotes for words with apostrophes.
  6. ^ a b "Google Books Ngram Viewer - University at Buffalo Libraries", Lib.Buffalo.edu, 22 August 2011, webpage: Buf497.
  7. ^ "Google NGrams: What We Learned From 5 Million Books", ComputationalLegalStudies.com, 25 September 2011, webpage: CLS25.
  8. ^ Acerbi A, Lampos V, Garnett P, Bentley RA (2013) The Expression of Emotions in 20th Century Books. PLoS ONE 8(3): e59030. doi: 10.1371/journal.pone.0059030
  9. ^ "Google Books Ngram Viewer". Google.
  10. ^ googlebooks-eng-all-1gram-20120701-w.gz at http://storage.googleapis.com/books/ngrams/books/datasetsv2.html
Retrieved from "https://en.wikipedia.org/w/index.php?title=Google_Books_Ngram_Viewer&oldid=591091985"