Is Watson Elementary?: Pt. 2

I used to be on top of ¬†Artificial Intelligence — I wrote a column for and ultimately went on to be the Editor-in-Chief of AI Expert, the leading trade magazine in the AI field at the time. I’ve tried to stay, not professionally competent, but familiar with the field. That has been rather difficult because the AI field has largely put aside grand theories and adopted two pragmatic themes: statistical techniques and mixed-approaches.

Statistical techniques rely on large bodies of data that allow you to guess, for instance, that “push comes to”->”shove” not from any understanding of metaphor or causation but because the word “push” followed by “comes” followed by “to” is followed 87.3% of the time by the word “shove”. Statistics excel at extracting patterns from large input sets.

Mixed approaches are ones which use different strategies to try to tackle different aspects or stages of a problem. Imagine a blackboard around which people raise their hands, come forward, add or erase a small bit of information, and step back into the crowd. For instance, one (relatively) simple tool might know that “X comes to Y” implies temporal ordering. Another might say that temporal ordering implies escalation. And another might say “A ‘Shove’ is an escalation of a ‘Push'”.

The more I read about Watson, the more it seems that while Watson used mixed approaches, what it’s mixing are almost all statistical techniques. So while it would undoubtedly be able to answer that “shove” is what “push often comes to…” I think it would do so without any reasoning, or schema, about temporal ordering or escalation.

The problem with statistical techniques is they are not general.

If a child is shown how to win tic-tac-toe by always starting with a ‘X’ in the upper-left box, and then we asked them if they could always win by starting in another corner, we would be disappointed if they couldn’t figure it out. Maybe not at first, but if tic-tac-toe was something they enjoyed, they would eventually recognize the pattern. If they never achieved the recognition, it would be troubling.

Pattern recognition, not pattern extraction, seems to be “how” we work. If pattern extraction were at the core, we wouldn’t be troubled by sharks when entering the ocean and we wouldn’t spend money on lottery tickets.

So it seems that Watson uses a fundamentally different “how” in its achievement. Yet the capability of rapidly and accurately answering questions (ones that have been intentionally obfuscated!) is clearly epochal. Clearly Watson has a role in medicine (diagnostics), law and regulatory compliance (is there precedent? is this a restricted behavior?), and intelligence (where’s the next revolution likely?). The problems of “Big Data” are very much in the mind of the software development community and Watson is a stunning leap forward in combining big data, processing power, and specialized algorithms.

Posted in AI

Top 10 Watson Answers

  1. What is Lady Gaga?
  2. What is Batman would totally defeat Godzilla?
  3. What is boxers, because the boys need to breathe?
  4. What is the Butlerian Jihad?
  5. What is Team Edward?
  6. What is orange you glad I didn’t say banana?
  7. What is antikythera mechanism? (“Sorry, we were looking for ‘Who is Adam?'”)
  8. What is a ring of pretty flowers that smells bad?
  9. What is 42?
  10. What is, there is now?

Is Watson Elementary?

We have to be very quick to ask if the astonishment we feel at Watson’s performance on Jeopardy is a projection. Eliza, the computerized Rogerian psychoanalyst does little more than slightly disguise the question “why do you feel that?” — a question that people are so delighted to answer that they overlook the mechanical blankness generating it.

Is Watson just a 21st century Eliza?

At a certain level we know the answer is “yes.” We know that Watson is not the product of a grand theory of consciousness. We know that Watson is essentially a (very impressive) parser backed by a blackboard system that uses various known techniques working against a (very capacious) database of facts and frames.

We also know that the capacious database of facts that Watson refers to could not have been hand-crafted. There has simply not been time for people to type in the names of every Olympic athlete, their results, their unusual physical characteristics, etc.¬†We know that Watson’s answers in that category, whether correct or incorrect, were based on facts that it read and processed on its own (or at least largely on its own). That’s astonishing.

We can also be certain that Watson is capable of inference. Given the facts “All men are mortal” and “Socrates is a man” we can be certain that Watson would be able to conclude correctly that “Socrates is mortal” (or, in Jeopardy form, “What is Socrates is mortal?”).

Although we’re sure that “Socrates is a man” is the type of fact that Watson can extract from unstructured input, what about “All men are mortal”? Can Watson infer that rule on its own or does that have to be hand-crafted by a programmer and spoon-fed to Watson?

Everything I know about the field of AI makes me think that just as there has not been enough time to hand-craft every fact fed into Watson, so too has there not been enough time to hand-craft enough higher-order associative logic to allow Watson to perform as well as it has demonstrated. (The Cyc Project has essentially been hand-crafting such logic for 20 years, with little sign of progress — but perhaps the Cyc database plays a role in Watson?)

Or can Watson induce “All men are mortal” from the facts that it has absorbed? Can it draw (tentative, statistically hedged) conclusions? If that is the case, then it seems logical that Watson’s learning can continue in an autonomous or semi-autonomous way. Such an inflection point in learning has long been seen as the critical moment in the generation of intelligence, whether in humans or, presumably, machines.

Computer Tackles Jeopardy This Week

The TV quiz show Jeopardy this week will feature an IBM Computer (“Watson”) competing against the two winningest Jeopardy champions. My prediction is that the computer is going to win. I base that on the name of the machine — not Blue J but “Watson.” That’s the name of IBM’s founder and I don’t think they’d put their corporate identity on the line if they didn’t have the breakthrough technology — technologies — to pull it off.

I’ve been reading about the technology for awhile. Apparently, it’s powered by (or at least was originally powered by) the UIMA framework, which is now an Apache project.