In The Music Man, con man Harold Hill runs a scam on the people of River City, Iowa, in which he promises them that he’ll form the children of the town into a fine marching band if their parents will spring for instruments and uniforms. He claims to use the revolutionary new Think System of instruction which works by simply having the child envision the music they would like to play. When forced to actually teach music, Harold Hill looks a child in the eye and says in a voice resonant with confidence and sincerity, “Think…Think!”
In software development there’s this notion of a “code review” that sounds like a good idea. Before a developer checks in a big change, they describe it to their colleagues and ask for feedback. The aim is peer review, but in practice it is usually just an empty ritual. The developer who wrote the code is the only one who understands it at anything but the most superficial level, so they say, “Here’s what I did” and all the reviewers nod and say, “That sounds like a reasonable thing to do.” You’ll all sit there trying to imagine the bugs that might arise, but you will fail because a bug is ipso facto a behavior that cannot be anticipated. This isn’t the scientific method. It’s the Think System.
If you study theoretical syntax in a graduate linguistics program you learn how to map sentences to tree structures that represent their grammatical structure. The subject goes here, the predicate goes here, large noun phrases are built recursively out of smaller building blocks like so. Students are encouraged to come up with sentences that resist a given tree-mapping procedure. As you move on to more advanced syntax classes, the game of Stump the Grammar becomes ever more recondite, and you develop a sophisticated ability to distinguish sentences that only provide an apparent difficulty for a theory and ones that truly do force a change of the rules. In the most advanced classes you focus not on sentences per se, but rather on a sort of meta-archaeology in which you study the history of the field in order to gain an appreciation for the subtle and illuminating ways contemporary tree-mapping techniques are superior to the tree-mapping techniques that were cutting edge back in the 1970s. If you a have a knack for it, trust me, this can be loads of fun, but given that no human being can hold more than a few dozen counterexample sentences in their head at any given time, how do we know we’re actually discovering something instead of just talking in circles? Well, what you do is you get a whole bunch of syntacticians to focus on the same problem, and foster an environment of vigorous debate. These are a really smart bunch of people, so if something is going awry one of them is bound to spot it. Think…Think!
It’s not spoiling much to say that The Music Man has a happy ending. Harold Hill turns out not to be such a bad guy after all, and the parents of River City get the marching band they hoped for, though this is due more to confirmation bias on their part than any effort on his. But the part about the Think System is a cautionary tale nonetheless. Because no matter how smart you are, you’re never smart enough to see the things you’re not smart enough to see. This tautology grows teeth when you realize that there will always be something you’re not smart enough to see. And credentialed, salaried, professionally smart people–that goes double for us.