Chapt6

6. Knowledge, Information, and Entropy

The book John von Neumann and the Foundations of Quantum
Physics contains a fascinating and informative article written by
Eckehart Kohler entitled "Why von Neumann Rejected Carnap's
Dualism of Information Concept." The topic is precisely the core issue before us: How is knowledge connected to physics? Kohler
illuminates von Neumann's views on this subject by contrasting them to those of Carnap.

Rudolph Carnap was a distinguished philosopher, and member of the
Vienna Circle. He was in some sense a dualist. He had studied one
of the central problems of philosophy, namely the distinction between analytic statements and synthetic statements. (The former are true or false by virtue of a specified set of rules held in our minds, whereas the latter are true or false by virtue their concordance with physical or empirical facts.) His conclusions had led him to the idea that there are two different domains of truth, one pertaining to logic and
mathematics and the other to physics and the natural sciences. This led to the claim that there are "Two Concepts of Probability," one
logical the other physical. That conclusion was in line with the fact that philosophers were then divided between two main schools as to
whether probability should be understood in terms of abstract
idealizations or physical sequences of outcomes of measurements.
Carnap's bifurcations implied a similar division between two different concepts of information, and of entropy.

In 1952 Carnap was working at the Institute for Advanced Study in
Princeton and about to publish a work on his dualistic theory of
information, according to which epistemological concepts like
information should be treated separately from physics. Von
Neumann, in private discussion, raised objections, and Pauli later
wrote a forceful letter, asserting that "I am quite strongly opposed to the position you take." Later he adds "I am indeed concerned that the confusion in the area of the foundations of statistical mechanics not grow further (and I fear very much that a publication of your work in its present form would have this effect)."

Carnap's view was in line with the Cartesian separation between a
domain of real objective physical facts and a domain of ideas and
concepts. But von Neumann's view, and also Pauli's, linked the
probability that occurred in physics, in connection with entropy, to knowledge, in direct opposition to Carnap's view that epistemology
(considerations pertaining to knowledge) should be separated from
physics. The opposition of von Neumann and Pauli significantly
influenced the publication of Carnap's book.

This issue of the relationship of knowledge to physics is the central question before us, and is in fact the core problem of all philosophy and science. In the earlier chapters I relied upon the basic insight of the founders of quantum theory, and upon the character of quantum
theory as it is used in actual practice, to justify the key postulate that Process I is associated with knowing, or feeling. But there is also an entirely different line of justification of that connection developed in von Neumann's book, Mathematical Foundations of Quantum
Mechanics. This consideration, which strongly influenced his thinking for the remainder of his life, pertains to the second law of
thermodynamics, which is the assertion that entropy (disorder,
defined in a precise way) never decreases.

There are huge differences in the quantum and classical workings of the second law. Von Neumann's book discusses in detail the
quantum case, and some of those differences. In one sense there is
no nontrivial objective second law in classical physics: a classical state is supposed to be objectively well defined, and hence it always has probability one. Consequently, the entropy is zero at the outset and remains so forevermore. Normally, however, one adopts some
rule of "coarse graining" that destroys information and hence allows probabilities to be different from unity, and then embarks upon an
endeavor to deduce the laws of thermodynamics from statistical
considerations. Of course, it can be objected that the subjective act of choosing some particular coarse graining renders the treatment not
completely objective, but that limited subjective input seems
insufficient to warrant the claim that physical probability is closely tied to knowledge.

The question of the connection of entropy to the knowledge and
actions of an intelligent being was, however, raised in a more incisive form by Maxwell, who imagined a tiny "demon" to be stationed at a
small doorway between two large rooms filled with gas. If this agent could distinguish different species of gas molecules, or their energies and locations, and slide a frictionless door open or closed according to which type of molecule was about to pass, he could easily cause a decrease in entropy that could be used to do work, and hence to
power a perpetual motion machine, in violation of the second law.

This paradox was examined Leo Szilard, who replaced Maxwell's
intelligent "demon" by a simple idealized (classical) physical
mechanism that consumed no energy beyond the apparent minimum
needed to 'recognize and responded differently to' a two-valued
property of the gas molecule. He found that this rudimentary process of merely 'coming to know and respond to' the two-valued property
transferred entropy from heat baths to the gaseous system in just the amount needed to preserve the second law. Evidently nature is
arranged so that what we conceive to be the purely intellectual
process of coming to know something, and acting on the basis of that knowledge, is closely linked to the probabilities that enter into the constraints upon physical processes associated with entropy.

Von Neumann describes a version of this idealized experiment.
Suppose a single molecule is contained in a volume V. Suppose an
agent comes to know whether the molecule lies to the left or to the right of the center line. He is then in the state of being able to order the placement of a partition/piston at that line and to switch a lever either to the right or to the left, which restricts the direction in which the piston can move. This causes the molecule to drive the piston
slowly to the right or to the left, and transfer some of its thermal energy to it. If the system is in a heat bath then this process extracts from the heat bath an amount 'log 2' of entropy (in natural units).
Thus the knowledge of which half of the volume the molecule was in
is converted into a decrement of "log 2' units of entropy. In von
Neumann's words, "we have exchanged our knowledge for the
entropy decrease k log 2." (k is the natural unit of entropy.)

What this means is this: When we conceive of an increase in the
"knowledge possessed by some agent" we must not imagine that this
knowledge exists in some ethereal kingdom, apart from its physical
representation in the body of the agent. Von Neumann's analysis
shows that the change in knowledge represented by Process I is
quantitatively tied to the probabilities associated with entropy.

Among the many things shown by von Neumann are these two:

(1) The entropy of a system is unaltered when the state of that
system is evolving solely under the governance of Process II.
(2) The entropy of a system is never decreased by any Process I
event.

The first result is analogous to the classical result that if an objective
"probability" were to be assigned to each if a countable set of
possible classical states, and the system were allowed to evolve in accordance with the classical laws of motion then the entropy of that system would remain fixed.

The second result is a nontrivial quantum second law of
thermodynamics. Instead of coarse graining one has Process I, which in the simple 'Yes-No' case converts the prior system into one where the question associated with the projection operator P has a definite answer, but only the probability associated with each possible answer is specified, not an answer itself.

One sees, therefore, why von Neumann rejected Carnap's attempt to
divorce knowledge from physics: large tracts in his book were
devoted to establishing their marriage. That work demonstrates the
quantitative link between the increment of knowledge or information associated with a Process I event and the probabilities connected to entropy. This focus on Process I allowed him to formulate and prove a quantum version of the second law. In the quantum universe the
rate of increase of entropy would be determined not by some
imaginary and arbitrary coarse graining rule, but by the number and nature of objectively real Process I events.

Kohler discusses another outstanding problem: the nature of
mathematics. At one time mathematics was imagined to be an
abstract resident of some immaterial Platonic realm, independent in principle from the brains and activities of those who do it. But many mathematicians and philosophers now believe that the process of
doing mathematics rests in the end on mathematical intuitions, which are essentially aesthetic evaluations.

Kohler argues that von Neumann held this view. But what is the origin or source of such aesthetic judgments?

Roger Penrose based his theory of consciousness on the idea that
mathematical insight comes from a Platonic realm. But according to
the present account each such illumination, like any other experience, is represented in the quantum description of nature as a picking out of an organized state in which diverse brain processes act together in an harmonious state of mutual support. A mathematical illumination
is a grasping of an aesthetic quality of order in the quantum state of the agent's brain/body. But apparently every experience of any kind is fundamentally like this: it is a Process I grasping of a state of order.

This notion that each Process I event is a felt grasping of a state in which various sub-processes act in concert provides a foundation for answering in a uniform way many outstanding philosophical and
scientific problems. For example, it provides a foundation for a
solution to a basic issue of neuroscience, the so-called "binding
problem". It is known that diverse features of a visual scene, such as color, location, size, shape, etc. are processed by separate modules located in different regions of the brain. This understanding of the Process I event makes the felt experience a grasping of a non-
discordant quasi-stable mutually supportive combination of these
diverse elements as a unified whole. To achieve maximal
organizational impact this event should provide the conditions for a rapid sequence of re-enactments of itself. Then this conception of the operation of von Neumann's process I provides also an
understanding of the capacity of an agent's thoughts to control its bodily behavior. The same conception of Process I provides also a
basis for understanding both artistic and mathematical creativity, and the evolution of consciousness in step with the biological evolution of our species. These issues all come down to the problem of the
connection of knowings to physics, which von Neumann's treatment
of entropy ties to Process I.