Piecewise linear function

In mathematics and statistics, a piecewise linear, PL or segmented function is a real-valued function of a real variable, whose graph is composed of straight-line segments.^[1]

Definition

A piecewise linear function is a function defined on a (possibly unbounded) interval of real numbers, such that there is a collection of intervals on each of which the function is an affine function. (Thus "piecewise linear" is actually defined to mean "piecewise affine".) If the domain of the function is compact, there needs to be a finite collection of such intervals; if the domain is not compact, it may either be required to be finite or to be locally finite in the reals.

Examples

The function defined by

f(x)={\begin{cases}-x-3&{\text{if }}x\leq -3\\x+3&{\text{if }}-3<x<0\\-2x+3&{\text{if }}0\leq x<3\\0.5x-4.5&{\text{if }}x\geq 3\end{cases}}

is piecewise linear with four pieces. The graph of this function is shown to the right. Since the graph of an affine(*) function is a line, the graph of a piecewise linear function consists of line segments and rays. The x values (in the above example −3, 0, and 3) where the slope changes are typically called breakpoints, changepoints, threshold values or knots. As in many applications, this function is also continuous. The graph of a continuous piecewise linear function on a compact interval is a polygonal chain.

Other examples of piecewise linear functions include the absolute value function, the sawtooth function, and the floor function.

(*) A linear function satisfies by definition $f(\lambda x)=\lambda f(x)$ and therefore in particular $f(0)=0$ ; functions whose graph is a straight line are affine rather than linear.

Fitting to a curve

An approximation to a known curve can be found by sampling the curve and interpolating linearly between the points. An algorithm for computing the most significant points subject to a given error tolerance has been published.^[2]

Fitting to data

If partitions, and then breakpoints, are already known, linear regression can be performed independently on these partitions. However, continuity is not preserved in that case, and also there is no unique reference model underlying the observed data. A stable algorithm with this case has been derived.^[3]

If partitions are not known, the residual sum of squares can be used to choose optimal separation points.^[4] However efficient computation and joint estimation of all model parameters (including the breakpoints) may be obtained by an iterative procedure^[5] currently implemented in the package segmented^[6] for the R language.

A variant of decision tree learning called model trees learns piecewise linear functions.^[7]

Notation

The notion of a piecewise linear function makes sense in several different contexts. Piecewise linear functions may be defined on n-dimensional Euclidean space, or more generally any vector space or affine space, as well as on piecewise linear manifolds and simplicial complexes (see simplicial map). In each case, the function may be real-valued, or it may take values from a vector space, an affine space, a piecewise linear manifold, or a simplicial complex. (In these contexts, the term “linear” does not refer solely to linear transformations, but to more general affine linear functions.)

In dimensions higher than one, it is common to require the domain of each piece to be a polygon or polytope. This guarantees that the graph of the function will be composed of polygonal or polytopal pieces.

Important sub-classes of piecewise linear functions include the continuous piecewise linear functions and the convex piecewise linear functions. In general, for every n-dimensional continuous piecewise linear function $f:\mathbb {R} ^{n}\to \mathbb {R}$ , there is a

\Pi \in {\mathcal {P}}({\mathcal {P}}(\mathbb {R} ^{n+1}))

such that

f({\vec {x}})=\min _{\Sigma \in \Pi }\max _{({\vec {a}},b)\in \Sigma }{\vec {a}}\cdot {\vec {x}}+b.

^[8]

If $f$ is convex and continuous, then there is a

\Sigma \in {\mathcal {P}}(\mathbb {R} ^{n+1})

such that

f({\vec {x}})=\max _{({\vec {a}},b)\in \Sigma }{\vec {a}}\cdot {\vec {x}}+b.

Splines generalize piecewise linear functions to higher-order polynomials, which are in turn contained in the category of piecewise-differentiable functions, PDIFF.

Applications

Crop response to depth of the watertable^[9]

In agriculture piecewise regression analysis of measured data is used to detect the range over which growth factors affect the yield and the range over which the crop is not sensitive to changes in these factors.

The image on the left shows that at shallow watertables the yield declines, whereas at deeper (> 7 dm) watertables the yield is unaffected. The graph is made using the method of least squares to find the two segments with the best fit.

The graph on the right reveals that crop yields tolerate a soil salinity up to ECe = 8 dS/m (ECe is the electric conductivity of an extract of a saturated soil sample), while beyond that value the crop production reduces. The graph is made with the method of partial regression to find the longest range of "no effect", i.e. where the line is horizontal. The two segments need not join at the same point. Only for the second segment method of least squares is used.

References

^ Stanley, William D. (2004). Technical Analysis And Applications With Matlab. Cengage Learning. p. 143. ISBN 978-1401864811.
^ Hamann, B.; Chen, J. L. (1994). "Data point selection for piecewise linear curve approximation" (PDF). Computer Aided Geometric Design. 11 (3): 289. doi:10.1016/0167-8396(94)90004-3.
^ Golovchenko, Nikolai. "Least-squares Fit of a Continuous Piecewise Linear Function". Retrieved 6 Dec 2012.
^ Vieth, E. (1989). "Fitting piecewise linear regression functions to biological responses". Journal of Applied Physiology. 67 (1): 390–396. doi:10.1152/jappl.1989.67.1.390. PMID 2759968.
^ Muggeo, V. M. R. (2003). "Estimating regression models with unknown break‐points". Statistics in Medicine. 22 (19): 3055–3071. doi:10.1002/sim.1545. PMID 12973787. S2CID 36264047.
^ Muggeo, V. M. R. (2008). "Segmented: an R package to fit regression models with broken-line relationships" (PDF). R News. 8: 20–25.
^ Landwehr, N.; Hall, M.; Frank, E. (2005). "Logistic Model Trees" (PDF). Machine Learning. 59 (1–2): 161–205. doi:10.1007/s10994-005-0466-3. S2CID 6306536.
^ Ovchinnikov, Sergei (2002). "Max-min representation of piecewise linear functions". Beiträge zur Algebra und Geometrie. 43 (1): 297–302. arXiv:math/0009026. MR 1913786.
^ A calculator for piecewise regression.
^ A calculator for partial regression.

[1] Stanley, William D. (2004). Technical Analysis And Applications With Matlab. Cengage Learning. p. 143. ISBN 978-1401864811.

[2] Hamann, B.; Chen, J. L. (1994). "Data point selection for piecewise linear curve approximation" (PDF). Computer Aided Geometric Design. 11 (3): 289. doi:10.1016/0167-8396(94)90004-3.

[Golovchenko-3] Golovchenko, Nikolai. "Least-squares Fit of a Continuous Piecewise Linear Function". Retrieved 6 Dec 2012.

[4] Vieth, E. (1989). "Fitting piecewise linear regression functions to biological responses". Journal of Applied Physiology. 67 (1): 390–396. doi:10.1152/jappl.1989.67.1.390. PMID 2759968.

[5] Muggeo, V. M. R. (2003). "Estimating regression models with unknown break‐points". Statistics in Medicine. 22 (19): 3055–3071. doi:10.1002/sim.1545. PMID 12973787. S2CID 36264047.

[6] Muggeo, V. M. R. (2008). "Segmented: an R package to fit regression models with broken-line relationships" (PDF). R News. 8: 20–25.

[7] Landwehr, N.; Hall, M.; Frank, E. (2005). "Logistic Model Trees" (PDF). Machine Learning. 59 (1–2): 161–205. doi:10.1007/s10994-005-0466-3. S2CID 6306536.

[8] Ovchinnikov, Sergei (2002). "Max-min representation of piecewise linear functions". Beiträge zur Algebra und Geometrie. 43 (1): 297–302. arXiv:math/0009026. MR 1913786.

[9] A calculator for piecewise regression.

[10] A calculator for partial regression.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]