Google Books Ngram Viewer: Difference between revisions
Giraffedata (talk | contribs) |
No edit summary |
||
Line 1: | Line 1: | ||
The '''Google Ngram Viewer''' is a phrase-usage graphing tool developed by Jon Orwant and Will Brockman of Google, and charts the yearly count of selected [[n-gram|''n''-grams]] (letter combinations)[[#References|<sup>[n]</sup>]] or words and phrases,<ref name=Huf/><ref name=Goog/> as found in over 5.2 million books [[digitized]] by [[Google Inc]] (up to 2008).<ref name=Cnet/><ref name=SThom/> The words or phrases (or ngrams) are matched by [[case-sensitive]] spelling, comparing exact uppercase letters,<ref name=Goog/> and plotted on the graph if found in 40 or more books during each year (of the requested year-range).<ref name=GInf/> The Ngram tool was released in mid-December 2010.<ref name=Huf/><ref name=Cnet/> |
The '''Google Ngram Viewer''' is a phrase-usage graphing tool developed by Jon Orwant and Will Brockman of Google, and charts the yearly count of selected [[n-gram|''n''-grams]] (letter combinations)[[#References|<sup>[n]</sup>]] or words and phrases,<ref name=Huf/><ref name=Goog/> as found in over 5.2 million books [[digitized]] by [[Google Inc]] (up to 2008).<ref name=Cnet/><ref name=SThom/> The words or phrases (or ngrams) are matched by [[case-sensitive]] spelling, comparing exact uppercase letters,<ref name=Goog/> and plotted on the graph if found in 40 or more books during each year (of the requested year-range).<ref name=GInf/> The Ngram tool was released in mid-December 2010.<ref name=Huf/><ref name=Cnet/> |
||
The word-search database was created by [[Google Labs]], based originally on 5.2 million books, published between 1500 and 2008, containing 500 billion words<ref name=BufLib/> in American English, British English, French, German, Spanish, Russian, |
The word-search database was created by [[Google Labs]], based originally on 5.2 million books, published between 1500 and 2008, containing 500 billion words<ref name=BufLib/> in American English, British English, French, German, Spanish, Russian, Hebrew, and Chinese.<ref name=Huf/> Italian words are counted by their use in other languages. A user of the Ngram tool has the option to select among the source languages for the word-search operations.<ref name=CLStud/> |
||
Researchers have analysed the Google Ngram database of books written in American or British English discovering interesting results. Amongst them, they found correlations between the emotional output and significant events in the 20th century such as the World War II.<ref name=EmotBooks/> |
Researchers have analysed the Google Ngram database of books written in American or British English discovering interesting results. Amongst them, they found correlations between the emotional output and significant events in the 20th century such as the World War II.<ref name=EmotBooks/> |
Revision as of 08:09, 17 January 2014
The Google Ngram Viewer is a phrase-usage graphing tool developed by Jon Orwant and Will Brockman of Google, and charts the yearly count of selected n-grams (letter combinations)[n] or words and phrases,[1][2] as found in over 5.2 million books digitized by Google Inc (up to 2008).[3][4] The words or phrases (or ngrams) are matched by case-sensitive spelling, comparing exact uppercase letters,[2] and plotted on the graph if found in 40 or more books during each year (of the requested year-range).[5] The Ngram tool was released in mid-December 2010.[1][3]
The word-search database was created by Google Labs, based originally on 5.2 million books, published between 1500 and 2008, containing 500 billion words[6] in American English, British English, French, German, Spanish, Russian, Hebrew, and Chinese.[1] Italian words are counted by their use in other languages. A user of the Ngram tool has the option to select among the source languages for the word-search operations.[7]
Researchers have analysed the Google Ngram database of books written in American or British English discovering interesting results. Amongst them, they found correlations between the emotional output and significant events in the 20th century such as the World War II.[8]
Operation and restrictions
Commas delimit user-entered search-terms, indicating each separate word or phrase to find.[5] The Ngram Viewer returns a plotted line chart within seconds of the user pressing the Enter key or the "Search" button on the screen.
As an adjustment for more books having been published during some years, the data is normalized, as a relative level, by the number of books published in each year.[5]
Google populated the database from over 5 million books published up to 2008. Accordingly, as of May 2012[update], no data will match beyond the year 2008. Due to limitations on the size of the Ngram database, only matches found in over 40 books are indexed in the database; otherwise the database could not have stored all possible combinations.[5]
Typically, search-terms cannot end with punctuation, although a separate full stop, or period, can be searched.[5] Also, an ending question mark (as in "Why?") will cause a 2nd search for the question mark separately.[5]
Omitting the periods in abbreviations will allow a form of matching, such as using "R M S" to search for "R.M.S." versus "RMS".
Corpora
The corpora used for the search are composed of total_counts, 1-grams, 2-grams, 3-grams, 4-grams, and 5-grams files for each language. The file format of each of the files is tab-separated data. Each line has the following format: [9]
- total_counts file
- year TAB match_count TAB page_count TAB volume_count NEWLINE
- Version 1 ngram file (generated in July 2009)
- ngram TAB year TAB match_count TAB page_count TAB volume_count NEWLINE
- Version 2 ngram file (generated in July 2012)
- ngram TAB year TAB match_count TAB volume_count NEWLINE
The Google Ngram Viewer uses match_count to plot the graph.
As an example, a word "Wikipedia" from the Version 2 file of the English 1-grams is stored as follows:[10]
ngram | year | match_count | volume_count |
Wikipedia | 1904 | 1 | 1 |
Wikipedia | 1912 | 11 | 1 |
Wikipedia | 1924 | 1 | 1 |
Wikipedia | 1925 | 11 | 1 |
Wikipedia | 1929 | 11 | 1 |
Wikipedia | 1943 | 11 | 1 |
Wikipedia | 1946 | 11 | 1 |
Wikipedia | 1947 | 11 | 1 |
Wikipedia | 1949 | 11 | 1 |
Wikipedia | 1951 | 11 | 1 |
Wikipedia | 1953 | 22 | 2 |
Wikipedia | 1955 | 11 | 1 |
Wikipedia | 1958 | 1 | 1 |
Wikipedia | 1961 | 22 | 2 |
Wikipedia | 1964 | 22 | 2 |
Wikipedia | 1965 | 11 | 1 |
Wikipedia | 1966 | 15 | 2 |
Wikipedia | 1969 | 33 | 3 |
Wikipedia | 1970 | 129 | 4 |
Wikipedia | 1971 | 44 | 4 |
Wikipedia | 1972 | 22 | 2 |
Wikipedia | 1973 | 1 | 1 |
Wikipedia | 1974 | 2 | 1 |
Wikipedia | 1975 | 33 | 3 |
Wikipedia | 1976 | 11 | 1 |
Wikipedia | 1977 | 13 | 3 |
Wikipedia | 1978 | 11 | 1 |
Wikipedia | 1979 | 112 | 12 |
Wikipedia | 1980 | 13 | 4 |
Wikipedia | 1982 | 11 | 1 |
Wikipedia | 1983 | 3 | 2 |
Wikipedia | 1984 | 48 | 3 |
Wikipedia | 1985 | 37 | 3 |
Wikipedia | 1986 | 6 | 4 |
Wikipedia | 1987 | 13 | 2 |
Wikipedia | 1988 | 14 | 3 |
Wikipedia | 1990 | 12 | 2 |
Wikipedia | 1991 | 8 | 5 |
Wikipedia | 1992 | 1 | 1 |
Wikipedia | 1993 | 1 | 1 |
Wikipedia | 1994 | 23 | 3 |
Wikipedia | 1995 | 4 | 1 |
Wikipedia | 1996 | 23 | 3 |
Wikipedia | 1997 | 6 | 1 |
Wikipedia | 1998 | 32 | 10 |
Wikipedia | 1999 | 39 | 11 |
Wikipedia | 2000 | 43 | 12 |
Wikipedia | 2001 | 59 | 14 |
Wikipedia | 2002 | 105 | 19 |
Wikipedia | 2003 | 149 | 53 |
Wikipedia | 2004 | 803 | 285 |
Wikipedia | 2005 | 2964 | 911 |
Wikipedia | 2006 | 9818 | 2655 |
Wikipedia | 2007 | 20017 | 5400 |
Wikipedia | 2008 | 33722 | 6825 |
The graph plotted by the Google Ngram Viewer using this data is here.
See also
References
- [n] - An "ngram" is a sequence of letters of any length, which could be a word, a misspelling, a phrase or gibberish.[6]
- ^ a b c "Google Ngram Database Tracks Popularity Of 500 Billion Words" Huffington Post, 17 December 2010, webpage: HP8150.
- ^ a b "Google Ngram Viewer - Google Books", Books.Google.com, May 2012, webpage: G-Ngrams.
- ^ a b "Google's Ngram Viewer: A time machine for wordplay", Cnet.com, 17 December 2010, webpage: CN93.
- ^ "A Picture is Worth 500 Billion Words – By Rusty S. Thompson", HarrisburgMagazine.com, 20 September 2011, webpage: HBMag20.
- ^ a b c d e f "Google Ngram Viewer - Google Books" (Information), Books.Google.com, December 16, 2010, webpage: G-Ngrams-info: notes bigrams and use of quotes for words with apostrophes.
- ^ a b "Google Books Ngram Viewer - University at Buffalo Libraries", Lib.Buffalo.edu, 22 August 2011, webpage: Buf497.
- ^ "Google NGrams: What We Learned From 5 Million Books", ComputationalLegalStudies.com, 25 September 2011, webpage: CLS25.
- ^ Acerbi A, Lampos V, Garnett P, Bentley RA (2013) The Expression of Emotions in 20th Century Books. PLoS ONE 8(3): e59030. doi: 10.1371/journal.pone.0059030
- ^ "Google Books Ngram Viewer". Google.
- ^ googlebooks-eng-all-1gram-20120701-w.gz at http://storage.googleapis.com/books/ngrams/books/datasetsv2.html