ARIE, YE WRETCHED OF THE EARTH.
The new neighbor had been put in there the night before. Rubashov had woken, but had only heard muffled sounds and the locking of cell No. 406. In the morning after the first bugle-blast, No. 406 had at once started to tap: ARIE, YE WRETCHED OF THE EARTH. He tapped quickly and deftly, with the technique of a virtuoso, so that his spelling mistake and the senselessness of his other messages must have had not technical but mental causes. Probably the new neighbor’s mind was deranged.
–Arthur Koestler, Darkness at Noon
Back in 2006-2007 there was a lot of buzz around a startup called Powerset, whose product was a natural language enabled search engine. Where Google made you distill your query into keywords, Powerset would allow you to ask questions of a computer in everyday English, just like on Star Trek. I was doing graduate work in computational linguistics at the time and was thrilled by the prospect. I learned from Powerset founders that their product did not merely utilize clever word count statistics to give the appearance of linguistic comprehension, but actually made use of deep theoretical ideas about the structure of human language. We all agreed that this was going to be great, and soon I had plotted out a future that had me finishing my degree, joining Powerset while they were still small, holding on for the meteoritic rise, and becoming absurdly wealthy sometime around late 2011 or so.
In preparation for our impending world domination, I thought I would compile a list of queries that could not be answered with a simple Google keyword search. They would be no-punches-pulled products of my own intellect, full sentences with complex conditions and qualifications, as tricky as I could make them. I thought I’d be able to rack up ten or so stumpers in an afternoon, but this proved to be infuriatingly difficult. It didn’t matter how ornate I made the question, as long as the answer was something other people cared about, a query consisting of the content words arranged in random order turned it up in the first page of Google’s search results. Powerset lumbered along in beta for a while before being bought by Microsoft and disappearing into Bing. Keyword-based search remained unchallenged, and linguists were proved useless again. In part this is just a cautionary tale about positioning yourself as a competitor to Google at this moment in history, but it also points to broader truths about software that have only become apparent in the last twenty years or so.
Early on it seemed clear that computers were thinking machines. They managed payrolls, planned moon shots, and played passable games of chess. Lacking anything that resembled eyes, ears, or limbs, the process of elimination led us to conclude they were replacements for our brains. What else could a locked room full of wires possibly be? Consider the computer villains of movies like Colossus: The Forbin Project, Demon Seed, and 2001. They were basically just really smart, really disagreeable people: disembodied Hannibal Lecters. Meanwhile back in the real world computers were expected to be less sociopathic but crucially still bright. Imagine giving a tour of Eniac’s workings to a group of your Army backers.
–What does this thing do?
–Well, it had better be pretty goddamn smart. Don’t you know there’s a war on, buddy?
With the rise of personal computing in the 1980s, the general public got wise to a dirty little secret: computers were not smart. In fact, they were astonishingly dumb. The task which they had co-opted from human brains–essentially adding columns of numbers—was not something for which the human brain was particularly well-suited, but rather something for which it had previously been the only tool at hand. The computer was not a startling new automation that drove our John Henry intellects to their deaths. It was more like someone handed John Henry a sledgehammer and said, “Here, it’s easier if you use this.”
Personal computers bluffed their way through about a decade of uselessness on Talking Moose, the Oregon Trail, and sheer novelty until the 1990s when they started to come into their own as communication devices. Even then this utility took a while to grasp because we had very mature and effective communication systems–phones, radio, television, postal systems–already in place. It’s only the rise of social media and smart phones that have made it apparent that the Von Neuman machine’s true métier is chit-chat.
At about 0:46 in this clip David Letterman starts razzing Bill Gates about what computers are good for and, though neither of them realizes it, gets right to the heart of the matter.
Once you see things this way, the unnatural effectiveness of dumb keyword searches is less surprising. At this point in history, computers are a medium of conversation, not a participant. Intelligence is irrelevant to that function. We don’t expect semaphore flags to be smart. We don’t require telephone handsets to understand English. Why build a brain in the middle when you’ve already got one on either end? Human brains may be only so-so at arithmetic, but their aptitude for forging social connections knows no bounds. Arthur Koestler’s Darkness at Noon is mostly concerned with the Soviet purge trials, but has a longish aside that describes the tapping code prisoners use to talk to each other through their cell walls. Though mostly a chilling portrait of the way a revolution eats its young, Koestler’s novel does have a bright spot in that it shows how even Stalin cannot shut people up. It’s great that our phones these days are smart, but hardly necessary. In a pinch a brick wall will do.
Computers may yet think. And if they do, they will undoubtedly make a better show of understanding human language, but it is not clear that for them linguistic ability will have the same front-and-center significance it has for us people. Technology is prosthetics, but only to a point. A bird is an animal that flies. A 747 is a machine that flies. But a 747 is not a giant mechanical bird. Likewise, a computer is a machine that processes information, and homo sapiens is an animal that does the same. But a computer is not an individuated consciousness that employs a nebulous web of syntax, semantics, and connotation to connect with what it perceives to be a separate autonomous mind, and likely will never be. The eerie part of artificial intelligence lies not in the contemplation of the intelligence, but in trying to imagine what would make it unmistakably artificial.