Farkas' lemma
Farkas' lemma is a solvability theorem for a finite system of linear inequalities in mathematics. It was originally proven by the Hungarian mathematician Gyula Farkas.[1] Farkas' lemma is the key result underpinning the linear programming duality and has played a central role in the development of mathematical optimization (alternatively, mathematical programming). It is used amongst other things in the proof of the Karush–Kuhn–Tucker theorem in nonlinear programming.[2]
Generalizations of the Farkas' lemma are about the solvability theorem for convex inequalities,[3] i.e., infinite system of linear inequalities. Farkas' lemma belongs to a class of statements called "theorems of the alternative": a theorem stating that exactly one of two systems has a solution.
Statement of the lemma
There are a number of slightly different (but equivalent) formulations of the lemma in the literature. The one given here is due to Gale, Kuhn and Tucker (1951).[4]
Farkas' lemma — Let and . Then exactly one of the following two assertions is true:
- There exists an such that and .
- There exists a such that and .
Here, the notation means that all components of the vector are nonnegative.
Example
Let m, n = 2, , and . The lemma says that exactly one of the following two statements must be true (depending on b1 and b2):
- There exist x1 ≥ 0, x2 ≥ 0 such that 6 x1 + 4 x2 = b1 and 3 x1 = b2, or
- There exist y1, y2 such that 6 y1 + 3 y2 ≥ 0, 4 y1 ≥ 0, and b1 y1 + b2 y2 < 0.
Here is a proof of the lemma in this special case:
- If b2 ≥ 0 and b1 − 2b2 ≥ 0, then option 1 is true, since the solution of the linear equations is x1 = b2/3 and x2 = b1-2b2. Option 2 is false, since b1 y1 + b2 y2 ≥ b2 (2 y1 + y2) = b2 (6 y1 + 3 y2) / 3, so if the right-hand side is positive, the left-hand side must be positive too.
- Otherwise, option 1 is false, since the unique solution of the linear equations is not weakly positive. But in this case, option 2 is true:
- If b2 < 0, then we can take e.g. y1 = 0 and y2 = 1.
- If b1 − 2b2 < 0, then, for some number B > 0, b1 = 2b2 − B, so: b1 y1 + b2 y2 = 2 b2 y1 + b2 y2 − B y1 = b2 (6 y1 + 3 y2) / 3 − B y1. Thus we can take, for example, y1 = 1, y2 = −2.
Geometric interpretation
Consider the closed convex cone spanned by the columns of ; that is,
Observe that is the set of the vectors for which the first assertion in the statement of Farkas' lemma holds. On the other hand, the vector in the second assertion is orthogonal to a hyperplane that separates and . The lemma follows from the observation that belongs to if and only if there is no hyperplane that separates it from .
More precisely, let denote the columns of . In terms of these vectors, Farkas' lemma states that exactly one of the following two statements is true:
- There exist non-negative coefficients such that .
- There exists a vector such that for , and .
The sums with nonnegative coefficients form the cone spanned by the columns of . Therefore, the first statement tells that belongs to .
The second statement tells that there exists a vector such that the angle of with the vectors is at most 90°, while the angle of with the vector is more than 90°. The hyperplane normal to this vector has the vectors on one side and the vector on the other side. Hence, this hyperplane separates the cone spanned by from the vector .
For example, let n, m = 2, a1 = (1, 0)T, and a2 = (1, 1)T. The convex cone spanned by a1 and a2 can be seen as a wedge-shaped slice of the first quadrant in the xy plane. Now, suppose b = (0, 1). Certainly, b is not in the convex cone a1x1 + a2x2. Hence, there must be a separating hyperplane. Let y = (1, −1)T. We can see that a1 · y = 1, a2 · y = 0, and b · y = −1. Hence, the hyperplane with normal y indeed separates the convex cone a1x1 + a2x2 from b.
Logic interpretation
A particularly suggestive and easy-to-remember version is the following: if a set of inequalities has no solution, then a contradiction can be produced from it by linear combination with nonnegative coefficients. In formulas: if ≤ is unsolvable then , , ≥ has a solution.[5] Note that is a combination of the left-hand sides, a combination of the right-hand side of the inequalities. Since the positive combination produces a zero vector on the left and a −1 on the right, the contradiction is apparent.
Thus, Farkas' lemma can be viewed as a theorem of logical completeness: ≤ is a set of "axioms", the linear combinations are the "derivation rules", and the lemma says that, if the set of axioms is inconsistent, then it can be refuted using the derivation rules.[6]: 92–94
Variants
The Farkas Lemma has several variants with different sign-constraints (the first one is the original version):[6]: 92
- Either the system has a solution with , or the system has a solution with .
- Either the system has a solution with , or the system has a solution with and .
- Either the system has a solution with , or the system has a solution with and .
- Either the system has a solution with , or the system has a solution with .
The latter variant is mentioned for completeness; it is not actually a "Farkas lemma" since it contains only equalities. Its proof is a simple exercise in linear algebra.
Generalizations
Generalized Farkas' lemma — Let , , is a closed convex cone in , and the dual cone of is . If convex cone is closed, then exactly one of the following two statements is true:
- There exists an such that and .
- There exists a such that and .
Generalized Farkas' lemma can be interpreted geometrically as follows: either a vector is in a given closed convex cone, or there exists a hyperplane separating the vector from the cone; there are no other possibilities. The closedness condition is necessary, see Separation theorem I in Hyperplane separation theorem. For original Farkas' lemma, is the nonnegative orthant , hence the closedness condition holds automatically. Indeed, for polyhedral convex cone, i.e., there exists a such that , the closedness condition holds automatically. In convex optimization, various kinds of constraint qualification, e.g. Slater's condition, are responsible for closedness of the underlying convex cone .
By setting and in generalized Farkas' lemma, we obtain the following corollary about the solvability for a finite system of linear equalities:
Corollary — Let and . Then exactly one of the following two statements is true:
- There exists an such that .
- There exists a such that and .
Further implications
Farkas's lemma can be varied to many further theorems of alternative by simple modifications, such as Gordan's theorem: Either has a solution x, or has a nonzero solution y with y ≥ 0.
Common applications of Farkas' lemma include proving the strong duality theorem associated with linear programming, game theory at a basic level,[clarification needed] and the Kuhn–Tucker constraints. An extension of Farkas' lemma can be used to analyze the strong duality conditions for and construct the dual of a semidefinite program. It is sufficient to prove the existence of the Kuhn–Tucker constraints using the Fredholm alternative but for the condition to be necessary, one must apply Von Neumann's minimax theorem to show the equations derived by Cauchy are not violated.
See also
- Hyperplane separation theorem
- Dual linear program
- Fourier–Motzkin elimination – can be used to prove Farkas' lemma.
Notes
- ^ Farkas, Julius (Gyula) (1902), "Theorie der Einfachen Ungleichungen", Journal für die Reine und Angewandte Mathematik, 124 (124): 1–27, doi:10.1515/crll.1902.124.1
- ^ Takayama, Akira (1985). Mathematical Economics (2nd ed.). New York: Cambridge University Press. p. 48. ISBN 0-521-31498-4.
- ^ Dinh, N.; Jeyakumar, V. (2014), "Farkas' lemma three decades of generalizations for mathematical optimization", TOP, 22 (1): 1–22, doi:10.1007/s11750-014-0319-y
- ^ Gale, David; Kuhn, Harold; Tucker, Albert W. (1951), "Linear Programming and the Theory of Games - Chapter XII" (PDF), in Koopmans (ed.), Activity Analysis of Production and Allocation, Wiley. See Lemma 1 on page 318.
- ^ Boyd, Stephen P.; Vandenberghe, Lieven (2004), "Section 5.8.3" (pdf), Convex Optimization, Cambridge University Press, ISBN 978-0-521-83378-3, retrieved October 15, 2011.
- ^ a b Gärtner, Bernd; Matoušek, Jiří (2006). Understanding and Using Linear Programming. Berlin: Springer. ISBN 3-540-30697-8. Pages 81–104.
Further reading
- Goldman, A. J.; Tucker, A. W. (1956). "Polyhedral Convex Cones". In Kuhn, H. W.; Tucker, A. W. (eds.). Linear Inequalities and Related Systems. Princeton: Princeton University Press. pp. 19–40.
{{citation}}
: External link in
(help); Unknown parameter|chapterurl=
|chapterurl=
ignored (|chapter-url=
suggested) (help) - Rockafellar, R. T. (1979). Convex Analysis. Princeton University Press. p. 200.