Amortized analysis

"Amortized" redirects here. For other uses, see Amortization.

In computer science, amortized analysis is a method for analyzing a given algorithm's time complexity, or how much of a resource, especially time or memory in the context of computer programs, it takes to execute. Unlike other analyses that focus on an algorithm's run time in a worst case scenario, known as asymptotic analysis, amortized analysis examines how an algorithm will perform in practice or on average.[1][2]

While certain operations for a given algorithm may have a significant cost in resources, other operations may not be as costly. Amortized analysis considers both the costly and less costly operations together over the whole series of operations of the algorithm. This may include accounting for different types of input, length of the input, and other factors that affect its performance.[3]

History

Amortized analysis initially emerged from a method called aggregate analysis, which is now subsumed by amortized analysis. However, the technique was first formally introduced by Robert Tarjan in his 1985 paper Amortized Computational Complexity, which addressed the need for more useful form of analysis than the common probabilistic methods used. Amortization was initially used for very specific types of algorithms, particularly those involving binary trees and union operations. However, it is now ubiquitous and comes into play when analyzing many other algorithms as well.[3]

Method

The method requires knowledge of which series of operations are possible. This is most commonly the case with data structures, which have state that persists between operations. The basic idea is that a worst case operation can alter the state in such a way that the worst case cannot occur again for a long time, thus "amortizing" its cost.

There are generally three methods for performing amortized analysis: the aggregate method, the accounting method, and the potential method. All of these give the same answers, and their usage difference is primarily circumstantial and due to individual preference.[4]

• Aggregate analysis determines the upper bound T(n) on the total cost of a sequence of n operations, then calculates the amortized cost to be T(n) / n.[4]
• The accounting method determines the individual cost of each operation, combining its immediate execution time and its influence on the running time of future operations. Usually, many short-running operations accumulate a "debt" of unfavorable state in small increments, while rare long-running operations decrease it drastically.[4]
• The potential method is like the accounting method, but overcharges operations early to compensate for undercharges later.[4]

Examples

Dynamic Array

Amortized Analysis of the Push operation for a Dynamic Array

Consider a dynamic array that grows in size as more elements are added to it such as an ArrayList in Java. If we started out with a dynamic array of size 4, it would take constant time to push four elements onto it. Yet pushing a fifth element onto that array would take longer as the array would have to create a new array of double the current size (8), copy the old elements onto the new array, and then add the new element. The next three push operations would similarly take constant time, and then the subsequent addition would require another slow doubling of the array size.

In general if we consider an arbitrary number of pushes n to an array of size n, we notice that push operations take constant time except for the last one which takes O(n) time to perform the size doubling operation. Since there were n operations total we can take the average of this and find that for pushing elements onto the dynamic array takes :$O(\tfrac{n}{n}) = O(1)$, constant time.[4]

Queue

Let's look at a Ruby implementation of a Queue, a FIFO data structure:

```class Queue
def initialize
@input = []
@output = []
end

def enqueue(element)
@input << element
end

def dequeue
if @output.empty?
while @input.any?
@output << @input.pop
end
end

@output.pop
end
end
```

The amortized enqueue operation is always constant time because it just pushes an element onto the input array. This operation does not depend on the lengths of either input or output and so takes just constant time.

However the dequeue operation is more complicated. If the output array already has some elements in it, then it takes constant time to perform dequeue. Otherwise if output is empty, it will take O(n) time to add all the elements onto the output array from the input array, with n being the length of the input array. Now if we have just copied n elements from input, then we can perform n dequeue operations each taking constant time before we have to perform another costly operation of copying elements from input again. Therefore in practice the amortized run time of dequeue is O(1).[5]

Common use

• In common usage, an "amortized algorithm" is one that an amortized analysis has shown to perform well.
• Online algorithms commonly use amortized analysis.

References

1. ^ "Lecture 7: Amortized Analysis". https://www.cs.cmu.edu/. Retrieved 14 March 2015.
2. ^ "Lecture 16: Introduction to Asymptotic Analysis". cs.cornell.edu. Retrieved 14 March 2015.
3. ^ a b Rebecca Fiebrink (2007), Amortized Analysis Explained, retrieved 2011-05-03
4. "Lecture 20: Amortized Analysis". http://www.cs.cornell.edu/. Cornell University. Retrieved 14 March 2015.
5. ^ Grossman, Dan. "CSE332: Data Abstractions". cs.washington.edu. Retrieved 14 March 2015.