Invert, always invert

Charlie Munger, the less famous Billionaire partner of Berkshire Hathaway, is famous for his critiques of human psychology.  He constantly warns of common mistakes that people make, and suggests simple ways you can improve your thinking capability.

One of his basic maxims is: "Invert, always invert".  The idea is that by always looking at something in the same way, you get stuck in basic assumptions.  By inverting your perspective, you can often find radically new approaches to problem solving.

Let's take a look at how this might be applied to the world of computer security.

Many years ago, while reading the otherwise excellent DAK Catalog, I found a description of one of the most poorly designed security products I can imagine.  It was a password protection system designed for a dial-up phone system (yes, at risk of dating myself, this was pre-Internet).  Concerned about the possibility that an intruder would attempt a brute force attack, the designers of this product came up with what they thought was a very clever idea.  The first incorrect character that was entered would immediately disconnect the call.  No longer would it be possible for an intruder to try  hundreds or thousands of passwords at a go.  One strike and you're out, dial back and try again.

Let's think about what this means.

Suppose this product used a ten character password.  Further assume it can use upper and lower case letters, and numbers.  That gives you a total of 62 options per character, and 62 to the tenth power possible passwords, or roughly 839 quadrillion possible passwords.  On average, a brute force attack will guess it correctly after 419 quadrillion tries.  Assuming you could try ten times a second (which is a pretty aggressive assumption for a slow speed dial-up system), hacking this system using a brute force attack would probably require 1.3 billion years.

However, this system accidentally leaked critical information to the intruder.  As soon as you guessed the first character correctly, IT TOLD YOU THAT YOU WERE CORRECT.  That is, it failed to disconnect you immediately.  This means that on average, you can guess each character in 31 attempts.  Guessing the entire ten character password would likely take you 310 attempts.  Assuming you could try one combination per minute (much slower than before, as it forces you to redial each time), hacking this system would require a bit over 5 hours.

Ouch.

This is why it's difficult to build secure systems just right.  If you're not really careful, it's very easy to leak all sorts of unintentional information, sometimes using the very mechanisms that you think are making your system more secure.

This is a great example of a security flaw, because it provides an interesting clue about how we might invert our entire approach to security.

Let's go back to the original idea of a ten character password.  Let's assume that we don't leak any information to the would-be intruder to let them know how much progress they're making on cracking that password.  We still suffer from the same basic weakness: we tell the intruder when they have successfully guessed the password.

This sounds ridiculously obvious.  How can a legitimate user of the service function if they don't get access to their data when they type the correct password?  As soon as the data comes up, you know the password is correct.

Unless.. you invert the assumption.

What if every user id and password combination provided access to the system?  Note I didn't say "legitimate access", just access.  An incorrect id and password combination would take the intruder straight into a bogus screen, a concept known as a honeypot.  If this was a banking system, then it would provide access to an account with an imaginary name, holding imaginary sums of cash, and an imaginary history of transactions.

Could an intruder tell that all this was fake?  Probably.  Eventually.  They could research that name, and see if it was a legitimate person.  Look up that name at the account's address.  There's all sorts of things they could do.

It would take minutes at the best.  It might take hours.  Or days.  What a waste of time.  How is an intruder ever going to do a brute force attack if they have to spend minutes or hours or days on each guess, using human intelligence rather than a machine algorithm to detect if they've found a legitimate combination?  And all the while, alarm bells should be going off in the bank, warning administrators and police that somebody is attempting to snoop where they shouldn't be.  Choosing one wrong id and password is understandable.  Ten in a row is clearly a criminal.  And finding a legitimate account will take trillions (or more) of tries.

Clearly, this idea is more applicable in some contexts than in others.  It takes time and effort to create effective algorithms to generate realistic looking data (and especially to make sure that real data doesn't leak through).  For a bank system, which has highly structured, very valuable, and easily generated data, this makes sense.  For an online newspaper, not so much.  So it's not a silver bullet.  Still though, it's an interesting concept that can be combined with existing security technologies to drastically raise the difficulty for intruders, simply by inverting a basic concept.

Always remember to invert!

Contemplating the Personal Data Warehouse

A few weeks ago, my friend Sivakumar suggested that we could improve human life by creating a Personal Data Warehouse, that tracked the development and abilities of people.  Imagine if you could compare the age at which your children developed certain language or other cognitive skills relative to their parents or grand-parents.  We might be able to detect and correct developmental problems early.

He then makes an interesting observation about a trait in his own family, as the mental math skills of each generation are slightly less well developed than the previous one.  He speculated that perhaps this was the result of an increasing reliance upon technology.

Is this true?

If it's true, is it a problem?

The idea that people will degenerate due to increasing reliance on technology (or something that functions like it, such as slavery) is an old one, and has been put forth as one of the possible causes of the fall of the Roman Empire.  More recently, it was brilliantly illustrated in Pixar's film Wall*E, where the human race is depicted as having depended on their robotic creations for so long that they have descended into a race of barely animate blobs, largely unaware of their surroundings, and motivated by nothing but an increasingly desperate search for entertainment.

It's also a very old idea that civilization is on the verge of collapsing.  Some of the earliest examples of writing we have, stemming from ancient Egypt thousands of years before the birth of Christ, complain about how much better things were in the good old days, and how everything since then has been going to the dogs.

How is it that everything has been sliding out of control for so many thousands of years, and yet there are still people around to complain about it?

I would argue that the crux of the issue is the constancy of change.  Sometimes things change rapidly, sometimes things change slowly, but it never comes to a complete stop.  Political revolutions challenge the old power structures, waged by youth who wear outlandish clothing, and completely ignore the accepted methods of political engagement and boldly protest and rebel in brand new ways.  And lest you think I'm talking about the 1960s, or perhaps the Occupy Wall Street movement, I actually had in mind Julius Caesar's rise to power in the late Roman Republic.  Maybe some things never change.

Change inevitably seems destructive to the previous generation.  The old values are thrown out the window.  It seems like the world is coming to an end.  And it is - their world.  What they (understandably) fail to appreciate is that when one world comes to an end, another one rises up to take it's place.  And so the cycle continues.

As much as we may be annoyed by the ghastly and garish fashions that the next generation inevitably adopts, we can presumably agree that fashion is in the eye of the beholder, and that one generation's tastes cannot be objectively demonstrated to be inferior to another's.  (At least if we ignore the 1970s.)  But some things can be measured.  Doesn't a decline in mathematical ability across the generations indicate a genuine loss?  This can be objectively measured.  So can knowledge of critical facts about the world.  Or the ability to react effectively to a crisis.  Aren't there objective yardsticks by which we can measure gain and loss?

The Fall of the Roman Empire once again provides an interesting case study.  The population of the city of Rome had, fairly objectively, lost their physical and psychological ability to wage effective war.  They were pushovers for the rising powers of the Goths and the Huns.  Surely this is an objective example of civilization going to the dogs?

Maybe not.  Sometimes lost in the discussion is the fact that the Roman Empire never exactly fell, not in the sense of having the entire continent of Europe sitting happily as a thriving metropolis on one day, to be replaced by a smoldering crater the next. The city of Rome was sacked, to be sure.  Some of the large scale economic trade routes and complex industries disappeared, true.  But for many of the inhabitants of the Empire, the fall of the Empire was barely noticeable.  One day they were citizens of Rome.  The next day, somebody came along and told them that for the last five years, they'd been citizens of the Ostrogothic Kingdom.  Taxed continued to be paid, official corruption continued, and everybody complained about how much better things used to be, when everybody wore togas and spoke proper Latin, instead of this degenerate Italian which seems to be spreading everywhere.

And why was it that nobody seemed interested in learning to speak proper Latin anymore?  It's not that people had become stupid, or otherwise incapable of learning the language of their forebears.  It simply wasn't useful anymore.  What was the point of learning a language that you couldn't do trade in, woo a girl with, or use to gripe with your friends?  It might be useful if you wanted to read a bunch of really old books, but that didn't really describe most of the population.

So returning to our previous question, is the loss of mathematical ability the sign of decline?  Or is it simply a reflection that those skills no longer make any sense?  What's the point, when your computer, your phone, your tablet, and maybe even your watch can calculate any mathematical product you can type, faster than you can type it?

Critics of this perspective will point out that it represents a loss of independence.  If all our computers, phones, tablets, and digital watches go simultaneously on the fritz, then we'll all be sorry we never learned to do math properly.

Or not.  More likely, we'll wish we had spent more time learning how to fight off mutant zombies using home made spears and improvised explosives, because the zombie apocalypse (or something like it) is the most likely scenario which will result in a complete breakdown of technology.

This brings us back to our original question: is there value in creating a Personal Data Warehouse, which can be used to measure our skills and relative progress across our lives and across the generations?  In theory, I'm a keen supporter of any way that  technology can be used to improve our individual lives.  I like the concept.

But a Data Warehouse is intrinsically a structured repository of data.  It allows you to organize vast amounts of data to spot complex patterns.  It's a great way to see sales trends, or weather patterns, or traffic flow, among reams of data that would otherwise be unintelligible.  The challenging part of designing a Data Warehouse is understanding what types of questions you may want to ask, which influences the structure of your data.

So what are your values?  What things are you going to measure?  And what makes you think those values will continue to be valued by the next generation?  My forebears might have created their own version of a data warehouse to measure skill in archery and driving horse-drawn chariots, skills at which I would fail miserably.  But I'm pretty good at navigating a Subaru through snow covered roads, a skill I find highly valuable.  My ability to re-materialize following an inter-dimensional trans-warp is nonexistent, a fact that bothers me not at all, but which might make me a laughingstock if I'm still around in two or three generations when that's the only way to travel.  I might snarl at these kids who don't know the first thing about a stick shift, but I suspect they won't care.

So if we're going to try to measure ourselves through time, it's going to be a trick to do so in a way that holds up over time.  Mind you, I'm not saying it can't be done, but I'll have to be convinced that whatever measures we choose are truly useful over time, and not simply a reflection of our current tastes.  The closest thing we have right now for this type of human record is the memoir.  It's a useful document, highly flexible in it's ability to track lots of variable data, but it lacks something in terms of analytical and comparative analysis.  At least it lasts.

Now, should I publish mine on paper or ebook...?

When will the future finally get here?

It's interesting how insanely wrong predictions of the future tend to be.  Even people who dream about the future all the time have terrible track records.  It's not that they predict too aggressively, or not aggressively enough.  They just focus on the wrong things.

One of my favorite examples of this is Robert Heinlein, widely considered to be one of the Godfathers of science fiction, along with Isaac Asimov and Arthur C. Clarke.  Heinlein dreamed about the future all the time, but consistently got two things wrong:

·         He vastly overestimated progress in physical science and engineering, especially regarding space travel.

·         He vastly underestimated progress in computers.

The implications of this can be comical.  In Starman Jones, he envisions a world in which we have  technology that can beat the speed of light by manipulating the fabric of space, but computers which are incapable of performing simple mathematical calculations, so astrogators spend their time on flights looking up figures in printed tables and keying them into the navigation systems by hand.

Even when Heinlein speculates specifically about computers, he gets it all wrong.  In The Moon Is a Harsh Mistress, he envisions a supercomputer so powerful that it wakes up and becomes artificially intelligent.  It had so much computational ability that it can calculate the odds of the Lunar colonists launching a successful revolution.  And yet this massive supercomputer, capable of self-programming and orders of magnitude more powerful than anything we have today, is just about maxed about by doing a simple video rendering of a human torso.

Part of the challenge in predicting the future is that technology doesn't improve at regular intervals.  In a given area, it might barely move for years or centuries, then explode forward in the blink of an eye.  Five years ago, electronic books were a joke.  Companies had been noodling with the concept for decades, but nothing they came up with was any good, and it seemed like it might be decades before anything might catch on.

Then the Kindle appeared, and the iPad, and the Nook, and numerous smaller competitors.  Now the question isn't how fast ebooks will grow, it's how long and in what form paper books will survive.  (I think they will survive for the long term, but at a fraction of their previous prominence).

So we've got ebooks.  Are we in the future yet?

Overall, I'd say no.  We're almost at 2015, and the world still seems a far cry from the vision presented in the movie Back to the Future 2.  It got a few parts right, like the prominence of large screen TVs and the tendency for kids (and adults!) to excessively multitask, but we seem as far away as ever from the flying cars, weather control, and hover boards.  Curiously enough, the movie made no mention of areas where we have made enormous progress, such as computers that can beat chess and jeopardy champions, and consumer devices like iPhones and iPads.  Did Heinlein consult on this movie?

On the other hand, we've definitely arrived at the future in some aspects.  I realized we had crossed a line into the future when I saw my first web URL on a movie poster back in 1995 (it was for  Mortal Kombat).  If you grew up with the Internet, you have no idea how startling it was when this obscure bit of networking technology finally broke into the mainstream as people had been speculating it might for years.

Once it did, all bets were off.  All of us back then who were on the cutting edge (using advanced services like Compuserve and Prodigy) could have predicted email and Wikipedia.  Nobody could have predicted twitter as a force that could organize revolutions, blogging that would take on mainstream media, or youtube that would turn funny cat videos into global sensations.

So for purposes of figuring out when the future has arrived, I'm going to arbitrarily divide it into three basic stages:

1.    Computers and networks go mainstream

2.    Household robots take over household chores

3.    Flying cars

As noted, we're already well ensconced in Phase 1, and have a generation of kids and young adults who can't imagine the world any other way.  (And I must confess I still scratch my head when I try to remember how people used to find things out before Google, or even its predecessor Alta Vista.)

But if you think Phase 1 was a game changer, just wait until Phase 2.  This is going to revolutionize the very concept of what it means to be human, and will rock our society and economy to its core.  After all, if we have robots that can make the bed, do the dishes and take out the cat,

Do we still need housekeepers?

Do we still need cashiers?

Do we still need auto mechanics?

Do we still need airline pilots?

You might get a bit queasy thinking about that last one.  After all, do you really want to trust your life to a machine that might suffer a blue screen of death at any moment?  And you'd be justified in your concerns, as long as you ignore Bruce's Law of Technology.  Because shortly after we graduate from today's autopilot to something that can approximate a takeoff and landing in good conditions, the technology is going to  improve so quickly that insurance premiums for using human pilots instead of automated ones will go through the roof.  Expect automated pilots to catch on as fast as ebooks.

How far are we away from this transition to a robotic world?  It could be a long way off.  Decades.  In fact, from where we stand, it looks as far off as ebooks looked in 1986.  And 1996.  And 2006.

At Google's 2011 I/O Conference, Google announced an initiative to develop an open source operating system for robots (ROS).  As you might expect from a Google initiative, this OS will be easy to connect to the Cloud.  This means that robots can leverage Google's massive server farms for complex tasks such as object recognition, and not have to lug around the huge volume of CPUs that would be required to do this in real time, nor their associate batteries.

It also potentially means that researchers can share ideas, technology and databases much easier.  Why reinvent the complex algorithms to recognize a household object, when somebody else has already invested hundreds or thousands of hours on it?  Leverage their code, and spend your time thinking about what cool things your robot can do with that object, once it has recognized it.

Does this mean the robot revolution is just around the corner?  That's the problem with predictions - we just can't tell.  Google's latest initiative certainly looks promising.  But I could find other initiatives back in the nineties and eighties that looked just as promising.  The technology needed wasn't ready back then.  Maybe its not now, either.  Or maybe it is.  When it finally is, expect robots to turn our entire world upside down.  It will revolutionize life, work, employment, and leisure, although for good or ill is impossible to say.

All I can say is that humans better still be allowed to drive by the time I can finally buy my first flying car.  I'm sticking firm on that one.

Intentions versus Capabilities

In the last few weeks, one of the most prominent technology stories has been about one of the most divisive legislative proposals in recent memory, called Sopa (Stop Online Piracy Act).  This proposed bill makes for strange bedfellows, bringing together organizations that have probably never agreed on anything in the past.  For example, the AFL-CIO and the Chamber of Commerce are supporting the legislation, while the Tea Party and The Huffington Post find themselves united in opposition.

Part of the reason for the opposition to the bill is the sweeping powers it provides.  It is possible, for example, to interpret the legislation as allowing a judge to shut down Google if any of Google's search results end up aiding pirates (which is virtually inevitable, given the sweeping nature of Google’s services).  Proponents of the bill dismiss this concern, saying that's not the intention of the legislation.  This dismissal soothes the critics not at all.

It's worth noting that critics of the bill include a large number of programmers and other people with technical backgrounds.  Programmers have deep experience with a problem that was perhaps best illustrated in the story “The Sorcerer's Apprentice” (of which Disney's 1940 Fantasia cartoon is one of the most accessible and popular versions).  In this story, we see the young apprentice, eager to use his new-found skills to lighten his load, enchanting a broom to carry the water.  Everything goes fine until the apprentice realizes he doesn't know how to make it stop.  His attempts to do so makes things infinitely worse, and he nearly drowns before the Sorcerer returns to the scene to save the day.

The moral of the story is that having access to power (or technology) does not convey understanding of or control over it, which can lead to unexpected and very unpleasant results.  Any programmer who has done even basic debugging has come to realize that it doesn't matter in the slightest what they intended their code to do.  What matters is what it actually does.  Discovering and correcting the difference between the two can be fiendishly difficult.

This is why programmers are not soothed by reassurances of what the legislation is intended to do.  It doesn't matter that somebody says "give me these sweeping powers, and I promise not to abuse them".  The world doesn't work that way.  Once the genie has been let out of the bottle, it can't be stuffed back in.  Power that can be abused, will be.  Even if you trust the current person in charge, what about the next one?

I don't support piracy.  But even less do I support sweeping reform that is poorly understood, with concerns swept under the rug because the architects say "That's not what we meant."

Spend some time learning to debug C, then we can talk.

The End of PCs?

You are most likely reading this blog article on a machine that would have been considered inconceivably powerful for most of the scope of human history.  You can easily communicate with people around the world in the blink of an eye.  You can effortlessly solve mathematical problems that would have confounded Euclid and Archimedes.  You can access an information repository greater than the Library of Alexandria.  If you transported this machine back a thousand years or so (and let’s pretend it would have had access to the necessary battery life and internet knowledge bases), wars would have been fought to possess it.

You hold in your hands the power of the Gods.

"With great power comes great responsibility." (Spider-man comics)

Spider-man learned this lesson to his cost when his inaction lead to the death of his beloved Uncle.  And Spider-man's powers were insignificant next to the powers of a modern personal computer.  This power is literally in your grasp.  Are you ready to accept the mantle of responsibility?

You probably think I'm stretching a point here.  If you're like most people, you just want to use your computer to read the news, gossip with friends on Facebook, and maybe watch some videos on Youtube.  You're just minding your own business.  You have no intention whatsoever of, say, joining a ring of criminals in Eastern Europe and participating in a scheme to extort money from e-commerce sites.  Except that, unless you’ve been extraordinarily careful with your super-powers, you probably already have.

"All that is necessary for the triumph of evil is that good men do nothing." (Edmund Burke)

The most common form of cyber-crime involves collecting a group of PCs to form a botnet.  This involves infecting each of these PCs with malware, which quietly turns that PC into a slave of the botnet owner, rather than the PC owner.  If the malware is at all clever at its task (which most of them are), it leaves the PC owner oblivious to the fact that anything has changed.  You still think that you're just minding your own business.  You don't realize that you've started engaging in criminal activity.

If it's any consolation, you're not the only inadvertent criminal out there.  You're in the company of millions of others.  Tens of millions.  Possibly hundreds of millions.  When you're talking about this magnitude of numbers, it’s hard to suggest a lack of personal ethics or failure of responsibility from any particular individual.  What we have is a systemic problem.  Systemic problems require systemic solutions.

"We have met the enemy and he is us." (Pogo cartoon strip)

It is possible to keep a PC free of malware.  You need to keep up to date with your patches.  Not just your operating system patches, however.  Also your browser patches.  And your Adobe patches.  And Java.  And any of the tens or hundreds of other programs you have installed on your computer.  And you need to make sure that you have a detailed understanding of any peer to peer software you run, in order to ensure that it's configured correctly.  And know how to configure your NAT router or firewall correctly.  And understand how to create good passwords.  And understand how to spot false links in emails.

The list goes on and on.  It can be done.  But it’s a full time job just to keep up with it.

There is an alternative.  It is to realize that not everybody has what it takes to be Spider-man, and not even to try.  This means something that makes most people cringe.  It means the end of PCs.

As revolutionary as this sounds, it's not actually a new idea.  We've already started using iPads, which are not PCs.  Not in the traditional sense.  They are extraordinarily limited.  There’s only one way to get additional software on them.  They don't have an exposed file system.  You can't connect them to all the cool USB devices that make your PC so flexible.  They are limited.  They are restricted.  They are, in a word, much safer than PCs.

"PCs are going to be like trucks.  They're still going to be around.  They're still going to have value.  But they're going to be used by one out of x people." (Steve Jobs)

Steve Jobs had an interesting vision: most people don't need PCs.  Most people don't need the level of power and flexibility that a full-blown computer provides.  You don't need a PC to browse the internet, check your email, and watch Youtube.  And Apple isn't simply filling this need with iPads.  They're moving in that direction with Macs as well.  In March of 2012, Apple will be implementing sandboxing for all applications sold through the Mac store.  This means that every application must request the specific permissions it will require before it is sold by Apple.  And Apple will have to approve it.  This will just be the online store.  At first.  But if Apple has its way, I suspect that it won't be long before the online store becomes the only way to purchase applications for a Mac.

This is going to slow down innovation.  People won't be able to write and release new and interesting applications nearly as fast as they could in the past.  If this had been the model back in the 80s, personal computers might never have gotten off the ground.  But we're no longer in the 80s.  Maybe it's ok for us to finally slow down a tad.

Of course, this doesn't impact Microsoft in the least.  Not yet.  But it seems that even Microsoft is realizing that unlimited power and flexibility in the operating system is not always such a good thing.  In 2001, feeling a surge of Unix envy, Microsoft released a feature called "raw sockets" into Windows XP.  Raw Sockets are cool.  They're powerful.  You can do all sorts of interesting things with them.  Maybe a little too interesting.  Some hackers leveraged them to perform some sophisticated attacks, some against Microsoft itself.  Raw sockets were quietly removed a few service packs later.

We may not yet be at the end of the PC era.  But maybe we should be.  Because most people simply don't need them.  Most people are unable or unwilling to spend the time and energy to use them safely.  And that's OK.  Not everybody needs to be Spider-man.

Bruce's Law of Technology

Once upon a time, Information Technology consisted primarily of Mainframes, sold by IBM and one or two small time competitors.  Oh sure, there were a few Unix and VAX minicomputers around if you looked hard enough, but these were being used mostly by scientists and engineers, and nobody paid them much mind.  Mainframes were running the corporations.  Mainframes were where the action was.

Then Apple (and quickly followed by several others, including IBM) released computers that would fit onto a single desktop.  They were small.  They didn't have much power.  They lacked the basic functionality needed to do any kind of real computing, such as job scheduling.  The IT professionals gave them a quick look, realized they were toys, and then promptly ignored them, to get back to the real business of doing IT.

Once upon a time, music was played on physical media.  Changes in technology involved migrating from one media to the next.  LPs gave way to tape cassettes (with a short detour through 8 track tapes), and then to Compact Disks.  Record labels looked forward to these changes, because it meant they could resell all their old titles in the new media.

Then, somewhere in the nineties, people started talking about music files called MP3s, which could play directly on computers, or even on dedicated devices.  These files took up a large percentage of the hard drive space that was available at the time.  Their sound quality was lousy.  The record labels took a look at MP3s, realized they couldn't compete with CD quality music, and then promptly ignored them, to get back to the real business of selling music.

Once upon a time, books consisted of sheets of paper bound together between two covers. People liked their paper books a lot.  So did authors and publishers.  They were cheap and portable.  The format had survived relatively unchanged for roughly a thousand years.  They seemed likely to be the dominant format for the next thousand years.

Then it became possible to store books on digital media.  It was lousy.  Nobody liked reading a CD Rom on their computer desktop.  Portable book readers had poor screens, terrible battery life, and were all incompatible with each other.  Publishers took a quick look at these eBooks, realized they'd never catch on, and then promptly ignored them, to get back to the real business of publishing paper books.

Clearly, we have a pattern here.  A pattern that was consistently missed by the people who most desperately needed to spot it.  They failed to understand what was happening, because they ignored Bruce's Law of Technology (which is perhaps understandable, as it is being published here for the first time).  Bruce's Law of Technology is as follows:

"New technology sucks.  Until, suddenly and unexpectedly, it doesn't."

It seems obvious.  But based on the anecdotes above (and I could add many more), people consistently ignore it's implications.  They know that technology improves.  But they don’t want to think about the possibility that it might overturn the world they know and understand and love.  They see its limitations, and refuse to see its possibilities, until it's far too late to do anything about it.

Somewhere out there, there’s technology that has the potential to turn your world upside down.  When you discover that technology, don’t discount it because it sucks.  Plan ahead for what the new world will look like once it stops sucking.  Think about how you will need to reinvent yourself, no matter how unpleasant that prospect may be.  Remember that not reinventing yourself will be much worse.  Remember Bruce’s Law of Technology.

The Limits of Reason

In the late eighteenth and early nineteenth centuries, a device known as the Mechanical Turk toured Europe and the United States, astounding it’s audiences by playing chess with an impressive degree of skill.  Many leading thinkers and engineers of the day were utterly convinced that it was a true chess playing machine, although there were persistent suspicions that perhaps the device (which included a large cabinet, apparently filled with gears and cogs) hid a real human chess player, possibly a dwarf, in its interior.

One of the most convincing exposes on the subject was written by Edgar Allen Poe in 1836.  His article was printed in the Southern Literary Messenger, and elicited responses from numerous newspapers and magazines including the New Yorker and the Baltimore Gazette.  It was well written, and completely correct in it’s conclusions – the Mechanical Turk was eventually revealed as a fraud.  However, it’s interesting to note that some of the most critical arguments employed by Poe were utterly incorrect.

His primary point was that as a mechanical device, it must necessarily be 100% fixed and determinate.  From this foundation, he makes several observations about the nature of the machine: “The moves of the Turk are not made at regular intervals of time, but accommodate themselves to the moves of the antagonist – although this point (of regularity), so important in all kinds of mechanical contrivance, might have been readily brought about by limiting the time allowed for the moves of the antagonist.”

He goes on to make a very interesting argument about the nature of the machine's purported algorithm: “The Automation does not invariably win the game.  Were the machine a pure machine, this would not be the case – it would always win.  The principle being discovered by which a machine can be made to play a game of chess, an extension of the same principle would enable it to win a game; a further extension would enable it to win all games – that is, to beat any possible game of an antagonist.  A little consideration will convince any one that the difficulty of making a machine beat all games is not in the least degree greater, as regards the principle of the operations necessary, than that of making it beat a single game.”

I’m not aware that any of Poe’s contemporaries had any disagreements with these points.  However, from our modern perspective, it’s easy to see that he was utterly incorrect.  We have chess playing machines today.  And what we know about the principles of making them play chess did not inevitably lead to an understanding of how to make them play perfect chess – it took decades of improvement (mostly on the hardware side) to evolve from the first primitive chess program to Deep Blue, which finally defeated world chess champion Gary Kasparov.  Even now that we've left human players far behind, nobody would claim that any computer in the foreseeable future could play a “perfect” game of chess (a feat which would require calculating every conceivable move on the chess board – a trivial exercise with a game like tic-tac-toe, but infeasible on a 64 square chess board).  Nor has our success at developing computer chess programs led to similar success teaching computers to master the much more fluid game of Go.

What would it have taken to convince Poe that he was wrong, especially when his ultimate conclusion was correct?  We could try to introduce him to the concept of heuristics, or allowing variable response times through the use of triggers, or even introducing the concept of indeterminacy to a system by seeding a complex algorithm with an external (and completely random) value.  Maybe this would have worked.  But none of this would be nearly as effective as simply having him grow up in a world where computers play chess all the time, and nobody thinks very much about it.

We like to think of ourselves as living in the age of reason and critical thinking.  We like to point at our technological progress and say “Here is where logic and reason have taken us.  They will solve any problem.”  We forget how much of what we know is built not on reason, but on trial and error, iterative improvement, and pure luck.