Percentile

From Wikipedia, the free encyclopedia
Jump to: navigation, search

A percentile (or a centile) is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations fall. For example, the 20th percentile is the value (or score) below which 20 percent of the observations may be found.

The term percentile and the related term percentile rank are often used in the reporting of scores from norm-referenced tests. For example, if a score is in the 86th percentile, it is higher than 86% of the other scores. The 25th percentile is also known as the first quartile (Q1), the 50th percentile as the median or second quartile (Q2), and the 75th percentile as the third quartile (Q3). In general, percentiles and quartiles are specific types of quantiles.

Applications[edit]

When ISPs bill "burstable" internet bandwidth, the 95th or 98th percentile usually cuts off the top 5% or 2% of bandwidth peaks in each month, and then bills at the nearest rate. In this way infrequent peaks are ignored, and the customer is charged in a fairer way. The reason this statistic is so useful in measuring data throughput is that it gives a very accurate picture of the cost of the bandwidth. The 95th percentile says that 95% of the time, the usage is below this amount. Just the same, the remaining 5% of the time, the usage is above that amount.

Physicians will often use infant and children's weight and height to assess their growth in comparison to national averages and percentiles which are found in growth charts.

The 85th percentile speed of traffic on a road is often used as a guideline in setting speed limits and assessing whether such a limits is too high or low.[1]

The normal distribution and percentiles[edit]

Representation of the three-sigma rule. The dark blue zone represents observations within one standard deviation (σ) to either side of the mean (μ), which accounts for about 68.2% of the population. Two standard deviations from the mean (dark and medium blue) account for about 95.4%, and three standard deviations (dark, medium, and light blue) for about 99.7%.

The methods given in the Definitions section are approximations for use in small-sample statistics. In general terms, for very large populations following a normal distribution percentiles may often be represented by reference to a normal curve plot. The normal distribution is plotted along an axis scaled to standard deviations, or sigma units. Mathematically, the normal distribution extends to negative infinity on the left and positive infinity on the right. Note, however, that only a very small proportion of individuals in a population will fall outside the −3 to +3 range. For example, with human heights very few people are above the +3 sigma height level.

Percentiles represent the area under the normal curve, increasing from left to right. Each standard deviation represents a fixed percentile. Thus, rounding to two decimal places, −3 \sigma is the 0.13th percentile, −2 the 2.28th percentile, −1 the 15.87th percentile, 0 the 50th percentile (both the mean and median of the distribution), +1 the 84.13th percentile, +2 the 97.72nd percentile, and +3 the 99.87th percentile. This is known as the 68–95–99.7 rule or the three-sigma rule.[dubious ] Note that in theory the 0th percentile falls at negative infinity and the 100th percentile at positive infinity, although in many practical applications, such as test results, natural lower and/or upper limits are enforced.

Definitions[edit]

There is no standard definition of percentile,[2][3][4] however all definitions yield similar results when the number of observations is very large.[5] Some methods for calculating the percentiles are given below.

The Nearest Rank method[edit]

Definition of the Nearest Rank method[edit]

One definition of percentile, often given in texts, is that the P-th percentile (0 \le P \le 100) of a list of N ordered values (sorted from least to greatest) is the smallest value in the list such that P percent of the data is less than or equal to that value. This is obtained by first calculating the ordinal rank and then taking the value from the ordered list that corresponds to that rank. The ordinal rank n is calculated using this formula

 n =  \left \lceil \frac{P}{100} \times N  \right \rceil

Note the following:

  • Using the Nearest Rank method on lists with fewer than 100 values can result in the same number being used for more than one percentile.
  • A percentile calculated using the Nearest Rank method will always be a member of the original ordered list.
  • The 100th percentile is defined to be the largest value in the ordered list.

First worked example of the Nearest Rank method[edit]

Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What are the 30th, 40th, 50th and 100th percentiles of this list using the Nearest Rank method?

Percentile
P
Number in list
N
Ordinal rank
n
Number from the ordered list
that has that rank
Percentile
value
Notes
30th 5 n = \left\lceil \frac{30}{100} \times 5 \right \rceil = \lceil 1.5 \rceil = 2. the second number in the ordered list, which is 20 20 20 is an element of the list
40th 5 n = \left\lceil \frac{40}{100} \times 5 \right \rceil = \lceil 2.0 \rceil = 2. the second number in the ordered list, which is 20 20 In this example it is the same as the 30th percentile.
50th 5 n = \left\lceil \frac{50}{100} \times 5 \right \rceil = \lceil 2.5 \rceil = 3. the third number in the ordered list, which is 35 35 35 is an element of the ordered list.
100th 5 Last 50, which is the last number in the ordered list 50 The 100th percentile is defined to be the largest value in the list, which is 50.

So the 30th, 40th, 50th and 100th percentiles of the ordered list {15, 20, 35, 40, 50} using the Nearest Rank method are {20, 20, 35, 50}

Second worked example of the Nearest Rank method[edit]

Consider an ordered population of 10 data values {3, 6, 7, 8, 8, 10, 13, 15, 16, 20}. What are the 25th, 50th, 75th and 100th percentiles of this list using the Nearest Rank method?

Percentile
P
Number in list
N
Ordinal rank
n
Number from the ordered list
that has that rank
Percentile
value
Notes
25th 10 n = \left\lceil \frac{25}{100} \times 10 \right \rceil = \lceil 2.5 \rceil = 3. the third number in the ordered list, which is 7 7 7 is an element of the list
50th 10 n = \left\lceil \frac{50}{100} \times 10 \right \rceil = \lceil 5.0 \rceil = 5. the fifth number in the ordered list, which is 8 8 8 is an element of the list.
75th 10 n = \left\lceil \frac{75}{100} \times 10 \right \rceil = \lceil 7.5 \rceil = 8. the eighth number in the ordered list, which is 15 15 15 is an element of the list.
100th 10 Last 20, which is the last number in the ordered list 20 The 100th percentile is defined to be the largest value in the list, which is 20.

So the 25th, 50th, 75th and 100th percentiles of the ordered list {3, 6, 7, 8, 8, 10, 13, 15, 16, 20} using the Nearest Rank method are {7, 8, 15, 20}

Third worked example of the Nearest Rank method[edit]

Consider an ordered population of 11 data values {3, 6, 7, 8, 8, 9, 10, 13, 15, 16, 20}. What are the 25th, 50th, 75th and 100th percentiles of this list using the Nearest Rank method?

Percentile
P
Number in list
N
Ordinal rank
n
Number from the ordered list
that has that rank
Percentile
value
Notes
25th 11 n = \left\lceil \frac{25}{100} \times 11 \right \rceil = \lceil 2.75 \rceil = 3. the third number in the ordered list, which is 7 7 7 is an element of the list
50th 11 n = \left\lceil \frac{50}{100} \times 11 \right \rceil = \lceil 5.50 \rceil = 6. the sixth number in the ordered list, which is 9 9 9 is an element of the list.
75th 11 n = \left\lceil \frac{75}{100} \times 11 \right \rceil = \lceil 8.25 \rceil = 9. the ninth number in the ordered list, which is 15 15 15 is an element of the list.
100th 11 Last 20, which is the last number in the ordered list 20 The 100th percentile is defined to be the largest value in the list, which is 20.

So the 25th, 50th, 75th and 100th percentiles of the ordered list {3, 6, 7, 8, 8, 9, 10, 13, 15, 16, 20} using the Nearest Rank method are {7, 9, 15, 20}

The Linear Interpolation Between Closest Ranks method[edit]

Definition of the Linear Interpolation Between Closest Ranks method[edit]

An alternative to rounding used in many applications is to use linear interpolation between the two nearest ranks. In particular, given the N sorted values v_1 \le v_2 \le v_3 \le \dots \le v_N, we define the percent rank corresponding to the nth value as:

p_n=\frac{100}{N}\left(n-\frac{1}{2}\right).

The value v of the P-th percentile may now be calculated as follows:[6]

  • If P<p_1 then we take v=v_1
  • If P>p_N, then we take v=v_N
  • If P=p_k for some integer k, then we take v = v_k.
  • Otherwise, we find the integer k for which p_k < P < p_{k+1}, and take v=v_k + \frac{P-p_k}{p_{k+1}-p_k}(v_{k+1}-v_k)=v_k+N\times\frac{P-p_k}{100}(v_{k+1}-v_k).

Note the following:

  • A percentile calculated using the Linear Interpolation Between Closest Ranks method will not always be a member of the original ordered list.
  • The 50th percentile calculated using the Linear Interpolation Between Closest Ranks method is just the sample median.
  • When N is even, the 25th percentile calculated using the Linear Interpolation Between Closest Ranks method is the median of the first \frac{N}{2} values (i.e. the median of the lower half of the data).

First worked example of the Linear Interpolation Between Closest Ranks method[edit]

Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What are the 5th, 30th, 40th and 95th percentiles of this list using the Linear Interpolation Between Closest Ranks method? First, we calculate the percent rank for each list value.

List value
vn
Position of that value
in the ordered list
n
Number of values
N
Calculation of
percent rank
Percent rank
pn
Notes
15 1 5 p_1=\frac{100}{5}\left(1-\frac{1}{2}\right)=10. 10
20 2 5 p_2=\frac{100}{5}\left(2-\frac{1}{2}\right)=30. 30
35 3 5 p_3=\frac{100}{5}\left(3-\frac{1}{2}\right)=50. 50
40 4 5 p_4=\frac{100}{5}\left(4-\frac{1}{2}\right)=70. 70
50 5 5 p_5=\frac{100}{5}\left(5-\frac{1}{2}\right)=90. 90

Then we take those percent ranks and calculate the percentile values as follows:

Percentile
P
Number of values
N
Is P less
than p1?
Is P greater
than pN?
Is there a
percent rank
equal to P?
What do we use for percentile value? Percentile value
v
Notes
5 5 Yes No No We see that P=5, which is less than the first percent rank p1=10, so use the first list value v1, which is 10 10 10 is a member of the ordered list
30 5 No No Yes We see that P=30 is the same as the second percent rank p2=30, so use the second list value v2, which is 20 20 20 is a member of the ordered list
40 5 No No No We see that P=40 is between percent rank p2=30 and p3=50, so we take k=2, k+1=3, P=40, pk=p2=30, vk=v2=20, vk+1=v3=35, N=5.
Given those values we can then calculate v as follows:
v=20+5\times\frac{40-30}{100}(35-20)=27.5
27.5 27.5 is not a member of the ordered list
95 5 No Yes No We see that P=95, which is greater than the last percent rank pN=90, so use the last list value, which is 50 50 50 is a member of the ordered list

So the 5th, 30th, 40th and 95th percentiles of the ordered list {15, 20, 35, 40, 50} using the Linear Interpolation Between Closest Ranks method are {10, 20, 27.5, 50}

The Weighted Percentile method[edit]

Definition of the Weighted Percentile method[edit]

In addition to the percentile function, there is also a weighted percentile, where the percentage in the total weight is counted instead of the total number. There is no standard function for a weighted percentile. One method extends the above approach in a natural way.

Suppose we have positive weights w_1, w_2, w_3, \dots, w_N associated, respectively, with our N sorted sample values. Let

S_n=\sum_{k=1}^n w_k,

the n-th partial sum of the weights. Then the formulas above are generalized by taking

p_n=\frac{100}{S_N}\left(S_n-\frac{w_n}{2}\right)

and

v=v_k+\frac{p-p_k}{p_{k+1}-p_k}(v_{k+1}-v_k).

The 50% weighted percentile is known as the weighted median.

Microsoft Excel method[edit]

Definition of the Microsoft Excel method[edit]

Some software packages, including Microsoft Excel[4] (up to and including version 2010, 2013 unverified) use the following method, noted as an alternative by NIST[7] to estimate v_P. Here, v_P is the value of the P-th percentile of an ascending ordered dataset containing N elements with values v_1 \le v_2 \le \dots \le v_N.

Firstly, the rank is calculated:

n = \frac{P}{100}(N-1)+1

Then the rank is split into its integer component k and decimal component d, such that n = k + d. Then v_P is calculated as:

 v_P = \begin{cases}
  v_1, & \text{for }k=0 \\
  v_N, & \text{for }k=N \\
  v_k+d(v_{k+1}-v_k), & \text{for }0 < k < N
  \end{cases}

First worked example of the Microsoft Excel method[edit]

Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What is the 40th percentile of this list using the Microsoft Excel method?

First we calculate the rank of the 40th percentile as follows:

n = \frac{40}{100}(5-1)+1=2.6

So n=2.6, which gives us k=2 and d=0.6. So the value of the 40th percentile is calculated as:

v_k+d(v_{k+1}-v_k) = v_2+0.6(v_3-v_2) = 20+0.6(35-20) = 29

So the value of the 40th percentile of the ordered list {15, 20, 35, 40, 50} using the Microsoft Excel method is 29

Second worked example of the Microsoft Excel method[edit]

Consider the ordered list {1,2,3,4} which contains four data values. What is the 75th percentile of this list using the Microsoft Excel method?

First we calculate the rank of the 40th percentile as follows:

n = \frac{75}{100}(4-1)+1=3.25

So n=3.25, which gives us k=3 and d=0.25. So the value of the 75th percentile is calculated as:

v_k+d(v_{k+1}-v_k) = v_3+0.25(v_4-v_3) = 3+0.25(4-3) = 3.25

So the value of the 75th percentile of the ordered list {1,2,3,4} using the Microsoft Excel method is 3.25

NIST method[edit]

Definition of the NIST method[edit]

The primary method recommended by NIST[7] to estimate v_P is given below. Here, v_P is the value of the P-th percentile of an ascending ordered dataset containing N elements with values v_1 \le v_2 \le \dots \le v_N.

Firstly, the rank is calculated:

n = \frac{P}{100}(N+1)

Then the rank is split into its integer component k and decimal component d, such that n = k + d. Then v_P is calculated as:

 v_P = \begin{cases}
  v_1, & \text{for }k=0 \\
  v_N, & \text{for }k=N \\
  v_k+d(v_{k+1}-v_k), & \text{for }0 < k < N
  \end{cases}

First worked example of the NIST method[edit]

Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What is the 40th percentile of this list using the NIST method?

First we calculate the rank of the 40th percentile as follows:

n = \frac{40}{100}(5+1)=2.4

So n=2.4, which gives us k=2 and d=0.4. So the value of the 40th percentile is calculated as:

v_k+d(v_{k+1}-v_k) = v_2+0.4(v_3-v_2) = 20+0.4(35-20) = 26

So the value of the 40th percentile of the ordered list {15, 20, 35, 40, 50} using the NIST method is 26

See also[edit]

References[edit]

  1. ^ Johnson, Robert; Kuby, Patricia (2007), "Applied Example 2.15, The 85th Percentile Speed Limit: Going With 85% of the Flow", Elementary Statistics (10th ed.), Cengage Learning, p. 102, ISBN 9781111802493 .
  2. ^ Hyndman RH, Fan Y (1996). "Sample quantiles in statistical packages". The American Statistician 50 (4): 361–365. doi:10.2307/2684934. JSTOR 2684934. 
  3. ^ Lane, David. "Percentiles". Retrieved 2007-09-15. 
  4. ^ a b Pottel, Hans. "Statistical flaws in Excel". Retrieved 2013-03-25. 
  5. ^ Schoonjans F, De Bacquer D, Schmid P (2011). "Estimation of population percentiles". Epidemiology 22 (5): 750–751. doi:10.1097/EDE.0b013e318225c1de. 
  6. ^ "Matlab Statistics Toolbox – Percentiles". Retrieved 2006-09-15. , This is equivalent to Method 5 discussed here
  7. ^ a b "Engineering Statistics Handbook: Percentile". NIST. Retrieved 2009-02-18. 

External links[edit]