Talk:Information theory

From Wikipedia, the free encyclopedia
Jump to: navigation, search
          This article is of interest to the following WikiProjects:
WikiProject Statistics (Rated B-class, Top-importance)
WikiProject icon

This article is within the scope of the WikiProject Statistics, a collaborative effort to improve the coverage of statistics on Wikipedia. If you would like to participate, please visit the project page or join the discussion.

B-Class article B  This article has been rated as B-Class on the quality scale.
 Top  This article has been rated as Top-importance on the importance scale.
 
WikiProject Computing (Rated B-class, Mid-importance)
WikiProject icon This article is within the scope of WikiProject Computing, a collaborative effort to improve the coverage of computers, computing, and information technology on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
B-Class article B  This article has been rated as B-Class on the project's quality scale.
 Mid  This article has been rated as Mid-importance on the project's importance scale.
 
WikiProject Computer science (Rated B-class, Top-importance)
WikiProject icon This article is within the scope of WikiProject Computer science, a collaborative effort to improve the coverage of Computer science related articles on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
B-Class article B  This article has been rated as B-Class on the project's quality scale.
 Top  This article has been rated as Top-importance on the project's importance scale.
 
WikiProject Mathematics (Rated B+ class, Top-importance)
WikiProject Mathematics
This article is within the scope of WikiProject Mathematics, a collaborative effort to improve the coverage of Mathematics on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
Mathematics rating:
A-BB+ Class
Top Importance
 Field: Applied mathematics
One of the 500 most frequently viewed mathematics articles.
WikiProject Telecommunications (Rated B-class, Top-importance)
WikiProject icon This article is within the scope of WikiProject Telecommunications, a collaborative effort to improve the coverage of Telecommunications on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
B-Class article B  This article has been rated as B-Class on the quality scale.
 Top  This article has been rated as Top-importance on the importance scale.
 
WikiProject Cryptography / Computer science  (Rated B-class, Top-importance)
WikiProject icon This article is within the scope of WikiProject Cryptography, a collaborative effort to improve the coverage of Cryptography on Wikipedia. If you would like to participate, please visit the project page, where you can join the discussion and see a list of open tasks.
B-Class article B  This article has been rated as B-Class on the quality scale.
 Top  This article has been rated as Top-importance on the importance scale.
Taskforce icon
This article is supported by WikiProject Computer science (marked as Top-importance).
 
WikiProject Systems (Rated B-class, High-importance)
WikiProject icon This article is within the scope of WikiProject Systems, which collaborates on articles related to systems and systems science.
B-Class article B  This article has been rated as B-Class on the project's quality scale.
 High  This article has been rated as High-importance on the project's importance scale.
Taskforce icon
This article is within the field of Cybernetics.
 
edit·history·watch·refresh Stock post message.svg To-do list for Information theory:

Here are some tasks awaiting attention:
  • Wikify: 21:06, 11 April 2013 (UTC)

Untitled[edit]

This article is also assessed within the mathematics field Probability and statistics.

Problem in Section on Entropy[edit]

Where it says that entropy is maximized when an r.v. is uniformly distributed, there appears to be a minus sign missing from the H(X)=log(n). Also from the definition it appears that in this case it should read H(X)=-log(1/n), but perhaps I'm mistaken. — Preceding unsigned comment added by 129.67.121.230 (talk) 12:45, 18 March 2013 (UTC)

Black holes vs conservation of information[edit]

In this astrophysics series on the Discovery Channel they said something about black holes "...breaking the fundamental law of conservation of information" and some quarrelling between theoretical physicists over black holes. I had never before heard of such a fundamental law and I thought it sounded very strange. So I went looking for it and what I found on Wikipedia had little to do with physics as far as I could see. Can someone point me to an article better describing what the TV series means with conservation of information, or is the series just misinformed? Thanks! Eddi (Talk) 00:27, 9 May 2012 (UTC)

I think I've found the answer here and here. The TV show either overstates a "common assumption" as a "fundamental law" to make it more sensational, or it uses "information" as a synonym for "entropy" to make it easier to understand. This may not be completely unreasonable, but I found it a bit confusing... Eddi (Talk) 09:47, 14 May 2012 (UTC)
See also quite a lot on the talk page of the first of those articles, for discussion on this. Jheald (talk) 21:29, 14 May 2012 (UTC)

Conformity Measurements[edit]

Information Theory (at least in the modern research arena) also includes the concept of Conformity, a measure that describes whether any unit bit of information conforms to the wider sea of information within which the bit of information rests.

What constitutes conformity is rather like the old PBS television shows and other educational efforts designed for children which show a set of objects which all conform to some set or sets of grouping, one object of which is mildly or wildly different than the rest or, applied to Information Theory, one of which is wildly out of Conformity with the rest of the information within the arena of bits of information.

Information Theory shows vectors of Conformity that can be graphed. When there are 2 information units being considered, Conformity is zero, and when there are an infinite number of information bits, conformity is also zero. For information bits > 3 yet less than infinity, the measure of Conformity behaves much the way that photons behave when they interact with each other.

Any way, the article here does not touch upon Conformity as a measurement within the wider realm of Information theory -- which is a shame since it's currently being used as a way of detecting corrupted data within streams of data. If I was better informed about the concepts involved I would add a section. Maybe someone who has formal training could be sought to add some. Damotclese (talk) 20:19, 15 May 2012 (UTC)

clarification of text[edit]

the following text which is quoted from the article "specifying the outcome from a roll of a die (six equally likely outcomes)" assumes using a six sided die. Dice are made in sides other then six. Please edit this to make such an assumption explicit. Thanks in advance. — Preceding unsigned comment added by 76.196.10.224 (talk) 18:23, 23 March 2013 (UTC)

Older foundations than Shannon[edit]

I have read "The Book of the Sign (Sefer HaOTh)", by the Spanish Abraham Abulafia, written around 1279. One of the founding texts of medieval Kabbalah, and I realised it is a work on Mathematical Information Theory. First every hebrew character gets an analog number (see hebrew numerals) and then he operates basic operations such as permutations or addings and then the numbers resulting get translated, get back a meaning. His work on permutations of the tetragrammaton, which gives 144 posible words, is clearly a work of Information Theory. Purposes of this theory were decrypting some message on sacred Jewish texts and getting a more direct spiritual communication through the finding of the lost sign. Abulafia wrote a letter had been lost in the process of creation of writing, so that explained the imperfections of World and Words. I am finding the exact references on the book.