Information and Entropy
Any information that results from an observation, a measurement, or an experiment, and that tells us only what we already know, produces no change in the number of possible responses; it does not diminish our uncertainty. The lower the probability that a message or an event will occur, the greater is the information carried by that message. The information obtained by drawing the correct response the first time (I = 32/1) is the inverse of the probability of obtaining this response before the drawing is made, or before the message is received (P = 1/32). Probability and entropy are related by statistical theory (see p. 102). By bringing together the different mathematical expressions, we can see that information is the inverse of the entropy of the physicists- it is the equivalent of an antientropy. The term neguentropy, negative entropy, has been proposed to identify this important property. Information and neguentropy are therefore the equivalents of potential energy.
The alliance goes further. By choosing suitable constants and values one can express information in thermodynamic units and relate it directly to entropy. We can then calculate the smallest expense of energy needed to generate one bit of information. To obtain an amount of information equal to one bit, we must degrade in entropy a very low but finite and therefore significant quantity of the energy of the universe.
This important finding has led physicists like Leon Brillouin to generalize Carnot's principle in such a way as to express the indissoluble relationship that exists between information acquired by the brain and the variation of entropy in the universe: Every acquisition of knowledge based on an observation or a physical measurement obtained with the help of an instrument uses energy in the laboratory--and therefore some of the energy of the universe.[2]
Consider an example. The reading of this page involves several elements: the text (printed in black on the paper) ( see notes ), a source of light (natural or artificial), the eye, and the brain. The lamp is the source of neguentropy. It emits a flow of light that is refracted on the succession of black and white segments of the printed words and modulates the light beam that strikes the eye. The eye receives the message and the brain decodes and interprets it. Thus the reader's brain has acquired information. But this must be paid for in energy: the watts of the lamp in exchange for the 24,000 bits of information on the printed page.
http://pespmc1.vub.ac.be/macroscope/chap4.html