||It has been suggested that this article be merged into Word lists by frequency. (Discuss) Proposed since May 2012.|
||This article includes a list of references, related reading or external links, but its sources remain unclear because it lacks inline citations. (May 2010)|
In computational linguistics, a frequency list is a sorted list of words (word types) together with their frequency, where frequency here usually means the number of occurrences in a given corpus. A short example could be:
It seems that Zipf's law holds for frequency lists drawn from longer texts of any natural language. Frequency lists are a useful tool when building an electronic dictionary, which is a prerequisite for a wide range of applications in computational linguistics.
German linguists define the Häufigkeitsklasse (frequency class) of an item in the list using the base 2 logarithm of the ratio between its frequency and the frequency of the most frequent item. The most common item belongs to frequency class 0 (zero) and any item that is approximately half as frequent belongs in class 1. In the example list above, the misspelled word outragious has a ratio of 76/3789654 and belongs in class 16.
where is the floor function.
- Helmut Meier: Deutsche Sprachstatistik. Hildesheim: Olms 1967. (frequency list of German words)