I watched part 3 of IBM's Watson computer on Jeopardy. It was pretty impressive, that they could make a computer that frequently gave correct answers.
The computer had some unfair advantages. The most obvious advantage was near-perfect buzzer timing. Whenever both the computer and a human knew the answer, the computer usually got to the buzzer first.
To make it a more valid comparison, they should have sequestered each player and given them the opportunity to answer each question without buzzer competition. The computer may not have won under those conditions. In Jeopardy, the "buzzer advantage" is huge.
There's a light that tells players when they're allowed to buzz in. The computer was given the buzzer signal electronically. To be fair, the computer should have been forced to point a camera at the light and parse the signal.
The computer crashed while the show was taping. They stopped the show while the computer was repaired. That had to do that, because that was the whole point of the show. Stopping when the computer crashes is a type of cheating. A human player would not be allowed to take a 1 hour break during the show. The breaks were edited out of the show that aired.
The computer was given the question electronically, at the same time players saw the question. The computer should have been forced to point a camera at the question screen and parse it out.
They didn't include audio/video Daily Doubles, because the computer hadn't been programmed to parse them.
The show was one big advertisement for IBM. It's not clear that Watson would be as useful in other contexts. For example, a computer lawyer probably would not handle the logical contradictions of insane State law as well as a human. Could a computer judge match the insane pro-State trolling of a human judge?
The Watson computer had unfair advantages, especially the buzzer advantage and the "stop taping when it crashes" advantage. Still, it was impressive to write a computer that could answer more than half the questions.
The computer made two obvious mistakes a human wouldn't make. It repeated an answer given by a previous player, because it hadn't been programmed to parse that. It thought "Toronto" was a US city, because it was programmed to downplay the value of categories. (Also, on "Final Jeopardy", the computer is forced to answer. It can't pass if it has no clue. The computer was programmed to give its best "Final Jeopardy" guess even if it had low confidence.) The computer would not have passed a "Turing Test", because it made mistakes no human would make.
The computer had some unfair advantages. Still, it was impressive to make a computer that could answer more than half the questions correctly.
Saturday, March 5, 2011
IBM's Watson On Jeopardy
Subscribe to:
Post Comments (Atom)
8 comments:
Agree and disagree.
Yes! The whole thing was nothing but the ad for IBM. Fake, ridiculous and self-evident shameless promotion. If placed in fair conditions, this computer not only would have never had any chance to win, it would have not even appeared on the show, because it "lost it's consciousness".
ha-ha-ha
I argued with all my coworkers because they didn't see how this was and ad for IBM and how unfair the setting was.
----
Nope! There was nothing impressive at all about computer's answers. I could have programmed it better and I am not even considered at IBM. There was nothing, I repeat, nothing cool about the code.
All they did was:
-Translate the question into series of requests.
-Search available databases (even wikipedia saved on it's HD would suffice) for answers that satisfy all the requests in milliseconds.
-Take results and form human-sounding answer.
This is NOT how human being thinks and is not comparable or indicative of any comparative performance.
This is a joke. To me, it shows the dismal, pre-collapse state of domestic science, that poses itself as breaking new barriers, while in truthful judgment it is standing still considering the financing it gets.
Some folks are impressed, also, because the Jeopardy is a game that looks impressive to an American from an American school. But, as compared to the intellectual challenges being played in other countries this jeopardy really looks like it is for retarded people. This even further decreases the supposed "achievements" of IBM.
What you are living through right now is "Zastoi", where they kept declaring one "achievement" after another. So, keep your compass straight. The IBM computer was a shame in every respect.
From Wikipedia:
"Watson had access to 200 million pages of structured and unstructured content consuming four terabytes of disk storage, including the full text of Wikipedia. Watson was not connected to the Internet during the game."
Even though it wasn't connected to the Internet, it still had "personal knowledge" of many more "facts" than a human could ever have, so it was completely unfair. Is it an impressive software engineering feat to be able to sift through that mountain of data and come up with a specific answer to a specific question? Yeah, I suppose so. But so what? Like you said, it wouldn't even be able to pass a Turing Test, so how impressive is it really?
Your analysis is identical in everyway to the one I came up with after watching the show.
1. It's an ad for IBM, nothing more.
2. The timing advantage is not realistic at all since we already know that computers can react to electrical signals faster than humans, so what is the point, the computer will press the buzzer faster 100% of the time if both know the answer before the timing light/signal goes off. That has nothing to do with natural language understanding, which the ad is trying to promote. This point was not mentioned at all during the show.
3. The computer did not have to parse the shapes into words like humans do. Can computers scan ascii faster than a person? Yes, but that has nothing to do with understanding and pre-processing the computers data in advance by converting it to an instantly transmitted ascii block is another advantage that was not discussed.
If the show was run honestly the computer would not have won.
A more important question is can humans control their own bodies?
If humans were able to control molecular interactions inside their bodies then computer artificial intelligence would be almost irrelevant.-DC
Check this article out:
Is the Mind a Computer?
http://creationsafaris.com/crev201103.htm#20110305a
"After a computer named Watson beat two contestants on Jeopardy last month, people are asking if the human mind is becoming obsolete. What are the similarities and differences between gray matter and deep blue?"
"Is the Mind a Computer?"
There's a shitload of articles just like that that IBM's publicist agency has placed throughout the media. No different than all the FaceBook coverage Goldman Sachs has been ordering up, including the Academy Awards win, Time's Man of the Year, and the upcoming Nobel Peace Prize (not yet announced).
You can tell it's propaganda because it all sticks to a consistent talking points memo and doesn't fgo off that topic.
Every intelligent man must at some point in his life realize that he was born with a program implanted in his head (it might be similar for females, but I suspect less so because females are more highly programed).
I wonder if that is how it might be for the first "aware" computer?
Humans are "aware", but they are still programmed. Some of them know that they are programmed but even then they can not escape the program. This is a very sticky point.
Being aware of the programing changes nothing?
-DC
I think if a human contestant went unconscious they would stop taping, so that argument is really dumb. You have enough good points to leave this ridiculous one out.
Post a Comment