Talk:Logistic function

Logistic Equation Page

I have to edit / make a wiki page for a project. I proposed to make a page titled Logistic Equation The page will basically link to here if people are looking for it continuous version (Logistic Differential Equation) and to the logistic map page if people are looking for its discrete form. Unless of course there are major objections from people. I feel the Logistic Equation is worthy of a page on its own as if you type logistic equation in google the mathworld page comes up first on the search. —Preceding unsigned comment added by Rjor2 (talk • contribs) 12:51, 19 February 2010 (UTC)

Notice on restructuring

I have reorganized the first two sections of the article. Please note that no (substantial) material has been deleted.

I now propose to make small textual additions in order to make the material (a) more accessible and (b) self-explanatory as much as possible. For example,

It models the S-curve of growth of some set P.

As a "pure" mathematician, the phrase "some set P" is delightful. But it might as well be "some set T" or "some set A" or "some set Z". That "P" is used is also self-evident because we made it so in the body of the entire article. Later we will want to reference Pierre François Verhulst. In that Wikipedia article the logistic differential equation uses the variable N. Such variety in variable names is acceptable to mathematicians and others of that ilk. But for the majority of Wikipedia readers, some explanation is needed.

The new text will read

It models the S-curve of growth of some set P, where P might be thought of as population.

This minor addition I will make now. --Михал Орела (talk) 07:29, 19 October 2008 (UTC)

The next text needing a massage is

A logistic function is defined by the mathematical formula:

The well-versed mathematical reader will understand that we are dealing with just one of the many mathematical forms in which the logistic function may be presented.

In particular, we will need to explain this particular choice and then to say later how it relates the wider field so that our readers can make the transition with little extra work. To begin that process, I propose the simple text

A simple logistic function may be defined by the mathematical formula:

--Михал Орела (talk) 07:40, 19 October 2008 (UTC)

Now it is clear that we need to add explanatory information on the fact that the variable P may be read as population and variable t may be read as time. Will do that now.--Михал Орела (talk) 07:47, 19 October 2008 (UTC)

Massaging the math

I am thinking again about the Wikipedia reader who might find the "usual" Mathematicians' favorite form

$P(t)={\frac {1}{1+e^{-t}}}\!$

a little bit frightening. I am also thinking that a linearized form of the exponential function ought to be included as well:

$P(t)={\frac {1}{1+exp(-t)}}\!$

and ultimately

$P(t)=({1+exp(-t)})^{-1}\!$

Now the issue is appearance on the page: same line (which I favor) or 3 separate lines (which "waste" space). — Михал Орела (talk) 08:35, 11 September 2009 (UTC)

So! My proposed change is

$P(t)={\frac {1}{1+e^{-t}}}\equiv {\frac {1}{1+exp(-t)}}\equiv ({1+exp(-t)})^{-1}\!$

It looks good and gives all the info needed. — Михал Орела (talk) 08:40, 11 September 2009 (UTC)

I don't agree. It looks cluttered and does not lead to any information. Anyone who is interested in knowing logistic function would know that "e" and "exp" are the same. Similarly, the differential equation is currently written as:

${\frac {d}{dt}}P(t)=P(t)(1-P(t))\equiv {\frac {dP}{dt}}=P(1-P)\equiv P'=P(1-P)\!$

which looks ugly. Why do you want a reader to sift through all the unnecessary equalities. The following is sufficient and complete:

${\frac {d}{dt}}P(t)=P(t)(1-P(t))\!$

—Preceding unsigned comment added by 65.202.28.10 (talk) 00:11, 23 February 2010 (UTC) 

Use of footnotes

I foresee the need to use footnotes to explain certain relationships to other mathematical forms of the same things. Hence I have introduced the appropriate section entitle "Notes".

Logistic differential equation

The next major task is to present the simple differential equation into the wider mathematical context. In other words, from this starting point we ought to generalize and make connection with the original of Pierre François Verhulst. This is the task scheduled for today.--Михал Орела (talk) 08:14, 19 October 2008 (UTC)

I have already introduced a section for footnotes in preparation for the development of the logistic differential equation. It now makes good sense to introduce the logistic differential equation in a section of its own. --Михал Орела (talk) 10:12, 19 October 2008 (UTC)

The first concrete step to take to achieve this goal will be to move the material in the section "In Ecology: modeling population growth" as close as possible to the top of the article. After all, this is where it all began. I will do this now and add in the appropriate Wikipedia reference to Verhulst.--Михал Орела (talk) 14:23, 20 October 2008 (UTC)

To highlight this "ecology section" I have introduced a strong visual marker — the usual Wikipedia Commons image of Verhulst. This will draw the eye to the section and keep the reader on the page. No need to click away to the Verhulst Wikipedia page for the basic information on the original differential equation, which is already here.--Михал Орела (talk) 14:50, 20 October 2008 (UTC)

The next task is to put the following into an appropriate place.

"The Verhulst equation, (1), was first published by Pierre F. Verhulst in 1838 after he had read Thomas Malthus' An Essay on the Principle of Population. Verhulst derived his logistic equation to describe the self-limiting growth of a biological population. The equation is also sometimes called the Verhulst-Pearl equation following its rediscovery in 1920. Alfred J. Lotka derived the equation again in 1925, calling it the law of population growth."

It is very good text! But the information needs to be "spread around". It would not suprise me if it is not already on Verhulst's Wikipedia page. We want to keep it here. My proposal is to place it into the footnote section. Notice that the reference number (1) used, no longer applies. —Preceding unsigned comment added by Mihal Orela (talk • contribs) 15:03, 20 October 2008 (UTC)

request

can we please have instructions for how to fit a logistic curve's parameters to data? Is there an analytic way to do this; or do you just have to use numerical methods like gradient descent and pray that it converges?

The Levenberg-Marquardt algorithm is a very general method that may be applicable here, depending on what exactly it is that you want to do. --MarkSweep (call me collect) 08:50, 22 March 2006 (UTC)

Plot

The plot needs labels on the X and Y axis! Targa86 21:25, 28 June 2007 (UTC)

wishlist entry

Would be neat to have a 1-2 sentence explanation in laymen's terms what is it good for, and why does it fit better or worse a task than other, comparable things/methods/whatevers. See standard deviation for example, where the section Interpretation and application explains in simple terms what is that good for. (Many statistical terms could use such an "interpretation"... most of them mathematically defines the terms but forget to explain it to shoemakers and milkmen. :-) ) --grin ✎ 07:42, 2004 May 6 (UTC)

I agree. As someone researching the diffusion of innovations, which follows an s-curve, I'd like to know what an s-curve really means. I'm not a shoemaker or milkman, but I would like to have a better understanding of what an s-curve is. --Westendgirl 22:59, 3 Jan 2005 (UTC)

This is largely already there; it's just not explicitly labeled as the answer to that question. I'll return to this article soon. Michael Hardy 01:04, 4 Jan 2005 (UTC)

It might be there, but I don't have the math background to understand it. :) My husband, a mathematician, assures me that all I need to know (for my purposes) is that an s-curve is a curve shaped like an S, as a result of data that has been plotted. I recognize that I don't need to understand the formula, but is this s-shape all I (and other lay readers) need to know? --Westendgirl 05:37, 4 Jan 2005 (UTC)

Done.Ancheta Wis 10:29, 8 Apr 2005 (UTC) Cleanup note removed

In effect, the S-curve is the simplest kind of long-term transition you can have where both the starting point and ending point are bounded. The differential equation governing the process is a polynomial of the smallest order (quadratic) that will accomplish the trick. Polynomials of order 0 and 1 won't do the trick.

It also occurs naturally in the study of any chemical process where both sides of a reaction, e.g. 2A <-> B+C are of order 2 or less. The resulting differential equation governing the rate is then quadratic and the solution (if bounded) will be a logistic.

In the study of the world population curve, contrary to the article's assertion, there actually is a good fit -- provided one takes the lower asymptotic to be non-zero. The underlying equation generalizes the Verhulst equation (also called the Malthus-Verhulst equation) by allowing a zero order term on the right, x' = A + Bx + Cx^2 with non-zero A. In fact, what one finds is that for the 1974-2004 range, there is a near-exact symmetry about 1989 (if I recall, it's P(1989-x)+P(1989+x) = 10.386 billion +/- 5 million, according to the international database of the US Census Bureau) and the curve closely fits a logistic with an upper asymptotic of 7.8 billion. The fit is within 6 million over the entire 1974-2004 range with a root-mean-square deviation of around 3 million!

For earler segments of the curve, there is a poor fit to a logistic primarily because (a) there are natural phase boundaries c. 1950 and c. 1850 which are impossible to fit a logistic across, and (b) the 1850-1950 segment does not fit any logistic at all, but rather the limiting case: an exponential with a lower asymptotic of about 1 billion. -- Mark, 26 September 2006

Standard formula needed

The three pages sigmoid, sigmoid function, and logistic curve seemed rather confusing for readers. So I have moved all info on the "standard" logistic curve 1/(1+exp(-t)) to sigmoid function, and kept here only the stuff about more general logistic curves that are shifted and scaled copied of the latter. I have provided a general formula for these curves, but I am not familiar with this field so I do not know whether the formula and parameters that I picked are OK. If there is a "canonical" formula, would someone please provide it?
Also the plot shows the standard sigmoid function; here one should have a plot of the more general logistic curve, shifted and scaled.
Jorge Stolfi 07:37, 2 Jul 2004 (UTC)

It is still confusing that there are two different formulas presented right at the top of the article. They are the same, but one is forced to do a little bit of algebra to see that is the case, and I don't see why there needs to be a change in notation, nor the difference between the "logistic function" and the "logistic sigmoid function". —Preceding unsigned comment added by Aheilbut (talk • contribs) 01:59, 7 August 2008 (UTC)

I am proposing to replace the "two different formulas" with one and to show how the two are related by integration of logistic differential equation.--Михал Орела (talk) 19:48, 17 October 2008 (UTC)

Logistic curve vs. logistic function

Argh.. I moved logistic curve to logistic function because it seemed more logical, but now I see that most users call it "curve". Profuse apologies. If you can, could you please reverse the move, making logistic function into a redirect? Thanks...
Jorge Stolfi 08:01, 2 Jul 2004 (UTC)

Typo

I think there is a mistake in the line: "In 1924 during The Great Depression Professor Ray Pearl and Lowell J. Reed used Verhulst's model to predict an upper limit of 2 billion for the world population. This was passed in 1930..." The Roaring 20s were at their height in 1924. Black Tuesday wasn't until 1929.

Article has wrong focus

The phrasing and structure of the article mistakes the logistic function for one of its applications, namely modeling population growth. Instead, the logistic function should be introduced as a mathematical function. It surely it has mathematical properties. Other important applications are found, for example, in machine learning and statistics. -Pgan002 21:47, 7 February 2006 (UTC)

Another form of the logistic function

I've only taken one year of calculus, but we learned the logistic function as $P(t)={M \over {Ae^{-kt}+1}}$, not :$P(t)=a{\frac {1+me^{-t/\tau }}{1+ne^{-t/\tau }}}\!$. Can someone explain the differences to me? If possible, I'd like to incorporate the derivation of the logistics formula that I added [[1]] into this page. --Alex S

Your formula is a special case of the presented formula with $a=M,m=0,n=A,\tau =k^{-1}$

MisterSheik 15:19, 26 June 2006 (UTC)

It's only the same curve moved by translation : $P(t)=am/n+a(1-m/n){\frac {1}{1+ne^{-t/\tau }}}=am/n+{\frac {M}{1+Ae^{-kt}}}$. It's not an usual presentation but why not. My english is too bad. Sorry, I'm french - HB - 2 August 2007

Another pov

I came across this curve while examining the qustion: what does it mean to double a probability? Eventually I decided that a nice way to do it is to duble the ratio r = p/(1-p). In effect, this treats the probability as being expressed as gambling odds.

If you take the log of this value ln(p/(1-p)) and flip the x any y axes y=exp(x)/exp(x+1) you get the logistical curve. Pmurray bigpond.com 06:08, 5 June 2006 (UTC)

proposed merge with logit

The page on logit and logistic function appear to regard the exact same topic, the logistic function. As such, I propose that they be merged. The primary portion of the logit page that I think bears inclusion in this page is the comment at the top regarding logistic regression (in italics). Pdbailey 17:11, 13 June 2007 (UTC)

I am not against a merge, but let's note they are different topics, albeit each other's inverse. Logit would need its own section which would be a redirect target, right? Now that the logit page is cleaned up, it may be doable. Baccyak4H (Yak!) 17:37, 13 June 2007 (UTC)

The more I thought about it, these two articles focus quite clearly on the two functions and little ink (pixels?) would be spared merging. The fact that one is the others inverse doesn't appear to be relevant to either article's point. Pdbailey 23:45, 13 June 2007 (UTC)

no equation 1

The section on history suggests that equation 1 was first proposed by... but there is no equation 1. I'd link it if I knew which one it was. Pdbailey 17:09, 13 June 2007 (UTC)

Logistic sigmoid function

The following text is incomplete because it does not give the original equation in which the parameters a, m, n, and τ occur.

“The sigmoid function with a = 1, m = 0, n = 1, τ = 1, namely

$P(t)={\frac {1}{1+e^{-t}}}\!$

is called the logistic sigmoid function or sometimes the standard sigmoid curve.”

I propose to correct this as soon as possible.--Михал Орела (talk) 15:06, 17 October 2008 (UTC)

[1] The first task is to merge the two variants of the "definition" of the function:

"A logistic function is defined by the mathematical formula:

$Z(\theta )={\frac {e^{\theta }}{1+e^{\theta }}}\!$

..."

and

"The sigmoid function with a = 1, m = 0, n = 1, τ = 1, namely

$P(t)={\frac {1}{1+e^{-t}}}\!$

..."

Given that the function variable name $P$ is used consistently in most of the article, it seems to make good sense to keep it. --Михал Орела (talk) 18:07, 17 October 2008 (UTC)

[2] Before doing the merge it is a good idea to harmonize with what already exists in Wikipedia. For example, in Logistic regression the definition is given as

"...

$f(z)={\frac {1}{1+e^{-z}}}$

..."

My suggestion will be to keep the function variable name $P$ with parameter $t$.--Михал Орела (talk) 18:19, 17 October 2008 (UTC)

[3] If we look at the given logistic differential equation

"...

${\frac {dP}{dt}}=P(1-P),\quad {\mbox{(2)}}\!$

..."

we readily find the (symbolic) solution (using Mathematica, for example) to be

$P(t)={\frac {e^{t}}{e^{t}-e^{c}}}$

Choosing the constant of integration $e^{c}=-1$ gives the desired result: the original first definition of the logistic function with $P$ ==> $Z$ and $t$ ==> $\theta$.--Михал Орела (talk) 19:08, 17 October 2008 (UTC)

Here is the proposed solution for the article:

---

Standard logistic sigmoid function

A logistic function or logistic curve is the most common sigmoid curve. It models the S-curve of growth of some set P. The initial stage of growth is approximately exponential; then, as saturation begins, the growth slows, and at maturity, growth stops.

A logistic function is defined by the mathematical formula:

$P(t)={\frac {1}{1+e^{-t}}}\!$

The logistic function finds applications in a range of fields, including artificial neural networks, biology, biomathematics, economics, chemistry, mathematical psychology, probability and statistics.

The logistic function is the solution of the first-order non-linear differential equation

${\frac {dP}{dt}}=P(1-P)\!$

with boundary condition P(0) = 1/2. This equation is the continuous version of the logistic map. One may readily find the (symbolic) solution (using Mathematica, for example) to be

$P(t)={\frac {e^{t}}{e^{t}-e^{c}}}$

Choosing the constant of integration $e^{c}=-1$ gives the other well-known form of the definition of the logistic curve

$P(t)={\frac {e^{t}}{1+e^{t}}}\!$

The logistic curve shows early exponential growth for negative t, which slows to linear growth of slope 1/4 near t = 0, then approaches y = 1 with an exponentially decaying gap.

The logistic function is the inverse of the natural logit function and so can be used to convert the logarithm of odds into a probability; the conversion from the log-likelihood ratio of two alternatives also takes the form of a logistic curve.

The logistic sigmoid function is related to the hyperbolic tangent, A.p. by

$2\,P(t)=1+\tanh \left({\frac {t}{2}}\right).$

---

I propose to post this within 24 hours if there are no major objections. It is still not exact. For example, I still need to state that $t$ ranges over the Real numbers from $-\infty$ to $+\infty$. --Михал Орела (talk) 19:45, 17 October 2008 (UTC)

Added internal link to Mathematica--Михал Орела (talk) 04:47, 18 October 2008 (UTC)

The new restructured text has now been posted. I will add some minor additions later.--Михал Орела (talk) 05:09, 19 October 2008 (UTC)

Massaging the math

The original definition has now been expanded to read

$P(t)={\frac {1}{1+e^{-t}}}\equiv {\frac {1}{1+exp(-t)}}\equiv ({1+exp(-t)})^{-1}\!$

For the purposes of consistency it is necessary to do a little rewriting for the corresponding differential equation

${\frac {dP}{dt}}=P(1-P)\!$

and its solution

$P(t)={\frac {e^{t}}{e^{t}-e^{c}}}$

which is also written as

$P(t)={\frac {e^{t}}{1+e^{t}}}\!$

This needs to be done carefully. I will post the suggested rewrite here first. — Михал Орела (talk) 08:56, 11 September 2009 (UTC)

For the differential equation I have now included

${\frac {d}{dt}}P(t)=P(t)(1-P(t))\equiv {\frac {dP}{dt}}=P(1-P)\equiv P'=P(1-P)\!$

to replace the original equation

${\frac {dP}{dt}}=P(1-P)\!$

My reasoning is this. Those who are not already familiar with all the conventions of the mathematics of differential equations may need to see (and learn) about the notations used. I still need to mention the "dot" notation which is still in common usage for differentiation with respect to time. — Михал Орела (talk) 08:40, 12 September 2009 (UTC)

image problem

The first image on the page, the plot of the logistic function, appears to be broken to me. Anyone else having this problem? 128.97.68.15 (talk) 17:18, 30 October 2008 (UTC)

At first glance it looks OK to me - a red S-shaped curve passing through (0, 0.5) with asymptotes y=0 and y=1. I'm using IE6. What do you see?--A bit iffy (talk) 17:26, 30 October 2008 (UTC)

It looks fine on Firefox and Safari; I will check with Opera now. --Михал Орела (talk) 09:08, 31 October 2008 (UTC)

Just checked with Opera browser; everything is "beautiful" --Михал Орела (talk) 09:16, 31 October 2008 (UTC)

Solution of the Logistic Differential Equation

Can we say more about the solution to the logistic differential equation than to use Mathematica? The equation is a lovely example of a Bernoulli differential equation, and mentioning the fact gives an easy explicit solution given an initial value. No need to mask pretty mathematics. --Moly 04:24, 21 January 2009 (UTC) —Preceding unsigned comment added by Moly (talk • contribs)

Looked at the Bernoulli differential equation you mentioned! It is very raw Mathematics :-) In other words there is very little "English text" to suggest to the "uninitiated" where the equation comes from and why it is significant. Perhaps you might elaborate on its "beauty" there and then see how to fit the logistic differential equation example into place?--Михал Орела (talk) 14:17, 27 January 2009 (UTC)

Neural Networks

I am looking at the following section:

"Logistic functions are often used in neural networks to introduce nonlinearity in the model and/or to clamp signals to within a specified range. A popular neural net element computes a linear combination of its input signals, and applies a bounded logistic function to the result; this model can be seen as a "smoothed" variant of the classical threshold neuron.

A reason for its popularity in neural networks is because the logistic function satisfies the differential equation

$y'=y(1-y).\,$

The right hand side is a low-degree polynomial. Furthermore, the polynomial has factors y and 1 − y, both of which are simple to compute. Given y = sig(t) at a particular t, the derivative of the logistic function at that t can be obtained by multiplying the two factors together. These relationships result in simplified implementations of artificial neural networks with artificial neurons."

There are no citations to support the assertions in the text. I will look into remedying this now. In addition, to argue the case that it is popular in neural networks precisely because it satisfies the given differential equation is not a good argument, in my opinion. I will replace this with specific examples from the neural network literature (and thus "prove" the assertion by example :) )--Михал Орела (talk) 16:46, 29 January 2009 (UTC)

I propose to replace the text "A reason for its popularity ... artificial neural networks with artificial neurons" with the following:

A common choice for the activation or "squashing" functions, used to clip for large magnitudes to keep the response of the neural network bounded is

$g(h)={\frac {1}{1+e^{-2\beta h}}}\!$

which we recognize to be of the form of the logistic function.

The text is taken from Gershenfeld 1999, p.150--Михал Орела (talk) 19:05, 29 January 2009 (UTC)

Why is it called "logistic"?

The origin of the term "logistic" is obscure. This word is far from self-explanatory. Thus, unlike for most other scientific terms, trying to look at its latin/greek etymology (e.g. consulting wiktionary) provides no insight really to the meaning of the concept. Could someone throw some light on the logic (if any) behind the word "logistic", especially when used in the present context? Typofier 22:34, 13 February 2009 (UTC) —Preceding unsigned comment added by Typofier (talk • contribs)

Logistic function and chaos.

I miss some information about this... --Andrestand (talk) 10:26, 24 August 2009 (UTC)

You will find this information at Logistic map. — Михал Орела (talk) 08:23, 10 September 2009 (UTC)