Information and intelligence (was Re: Objects and Relations)

Doug Morse wrote:

> Hi,
>
> I'd like to offer a few thoughts regarding Bob's post that might be
> helpful.
>
> First, when talking about "data" and "information" (esp. the latter),
> there's some value in taking Shannon's (1948) view of information into
> account. For Shannon, information (I) is inversely proportional to
> uncertainty (U). So, for example, there were 16 Democratic
> presidental nomination candidates in November 1975. The amount of
> uncertainty can be represented with 4 bits (assuming equal
> probabilities, which wasn't the case).

Lots of folks have come up with quantitative measures for things they call information. If one works for a telco, knowing Shannon's communication theory quantification is important. However, it has the odd outcome that random white noise has the highest information content.

Fisher is likewise important in other contexts.

Other quantifications have import in still other contexts.

Because our field solves so many problems in so many domains, knowing any of those definitions or quantifications might be important to one or another of us or even to all of us at various times. At the same time, our field has its own idea of information as I described earlier. When we use the word here without qualification or additional context, the word conveys the meaning I gave earlier, and it is a definition with profound importance to our field.

Without knowing the distinction, how does one comprehend or communicate the difference between the conceptual, the logical (more and more I prefer the term 'formal' to 'logical') and the physical?

> My main point is that increasing information is fundamentally about
> reducing uncertainty.

Not all definitions have that meaning, and I don't see the relevance of uncertainty to our field. Plenty of things about which we have certainty later turn out to be wrong.

> Second, Goedel's theorem indicated that no formalism can prove it's
> own completeness, except by referencing a large, more encompassing
> formalism. This does NOT necessarily mean that any formalism will
> "always require additional external information".

To-may-to/to-mah-to

   This is only needed
> when trying to prove the completeness of the formalism. We use
> formalisms all the time, with great success and utility, w/o the need
> of more encompassing formalisms.

Goedel's first: All of our general formalisms are either consistent or complete.

Goedel's second: Inconsistent formalisms lack utility because they 'prove' everything and anything including their own consistency.

Thus, we prefer consistent general formalisms, which are necessarily incomplete. The consistency of the formalism is information one cannot encode for machine processing within the formalism itself. This is true for all general formalisms.

Just as we accept incomplete formalisms we sometimes choose to accept undecidable formalisms; although, I often suspect we do when we shouldn't. In any case, all of our formalisms have limitations or pitfalls.

   Formalisms are a kind of
> representational systems / model, so they do of course make reference
> to "the real world" and/or our knowledge concerning it -- perhaps this
> was Bob's point (but it doesn't arise from Goedel).

Please, let Bob speak for himself. Bob said what me meant and attributed its source with accuracy and precision (if perhaps with insufficient clarity.)

> So, what emerges from both of these points is that "intelligence" (and
> hence AI) is much, much more than just acquiring vast amounts of data
> and information.

Of course. I don't recall anyone equating intelligence with aquiring fast amounts of data. To have strong AI, I observe the AI will have to interpret information as readily as humans do which suggests all encodings will then become suitable for machine processing.

Earlier this morning, I looked out my window and saw little clouds of white racing across the potato field next to my house. The appearance and speed gave me useful information. Later this afternoon, I looked out my window and saw a bright serene day absent any cloudlets over the potato field. That also gave me useful information. I acted on the information by shovelling snow drifts from my driveway while the sunlight and calm made the task more pleasant.

None of that information was encoded suitably for machine processing until I typed it in this post a few moments ago. However, when we achieve strong AI, we will have created machines capable of interpreting the information encoded in racing cloudlets and sunshine.

At that point, I suspect the machines managing our data for us will come up with their own definitions that may suit their psyches better. Until then, I find the definitions we have profoundly useful.

   Indeed, IMHO, data and information is just the
> "stuff" or "fodder" upon which an intelligent system operates. What
> makes a machine or human "intelligent" is, amoung many things,
> powerful external and internal pattern recognition that supports the
> dynamic, flexible, and adaptive construction of representations and
> processes that can solve difficult and complex problems.

I don't see how equating data and information with stuff or fodder helps us manage data, though.

> That's it! Just my $0.02 about Bob's first few points, hopefully
> providing just a little more detail / clarity.

Thank you. I hope things are much clearer now too.

>
> Cheers!
> Doug
>
>
> P.S. -- Just for completeness, for those interested, Shannon defined
> uncertainity as the number of alternatives (his information theory is
> where the term "bit" came from). So, for equally likely (probably)
> alternatives,
>
> U = log2 k
>
> where k is the number of alternatives (bits). When the probability of
> alternatives is not equal, then
>
> U = - Summation { P[i] log2 P[i] }
>
> where P[i] is the likelihood of each alternative.

[snip] Received on Mon Feb 12 2007 - 00:41:13 CET