Talk:Cross entropy

From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Statistics (Rated Start-class, Low-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

Start-Class article Start  This article has been rated as Start-Class on the quality scale.
 Low  This article has been rated as Low-importance on the importance scale.
 
WikiProject Mathematics (Rated Start-class, Low-importance)
WikiProject Mathematics
This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of Mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Mathematics rating:
Start Class
Low Importance
 Field: Probability and statistics
WikiProject Physics (Rated Start-class, Low-importance)
WikiProject icon This article is within the scope of WikiProject Physics, a collaborative effort to improve the coverage of Physics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Start-Class article Start  This article has been rated as Start-Class on the project's quality scale.
 Low  This article has been rated as Low-importance on the project's importance scale.
 


Untitled[edit]

This article uses the notation KL(p, q) and also DKL(p || m) when talking about Kullback-Leibler divergence. Are these notations two ways of expressing the same idea? If so, the article may want to indicate this equivalence.

The log-likelihood of the training data for a multinomial model is the same as the cross-entropy of the data. (Elements of Statistical Learning, page 32)

L(theta) = sum (all classes k) I(G=k) log Pr(G=k | X = x)

I guess "I(G=k)" is p and Pr(G=k | X=x) is q here.

Could somebody in the know please add this? Thanks!

WikiProject class rating[edit]

This article was automatically assessed because at least one WikiProject had rated the article as stub, and the rating on other projects was brought up to Stub class. BetacommandBot 09:46, 10 November 2007 (UTC)


To improve clarity, this stub should be merged into the "Kullback–Leibler divergence" article. "Cross entropy" and "relative entropy" refer to the same quantity in the literature, at least up to a sign convention. DRB (talk) 00:39, 15 January 2010 (UTC)

Hmmm... fair idea but not a cut-and-dry case. maybe put a merge tag on it, at least. Kevin Baastalk 14:07, 15 January 2010 (UTC)

What's missing[edit]

At least one analogy or example would be most helpful. I get no sense at all as to what this article is attempting to describe. Marc W. Abel (talk) 02:24, 7 May 2012 (UTC)