I'm Sorry Dave

Well, that was fast. Only a few days ago I blithely invoked the HAL 9000 and now I regret it because  a real life AI has gone and done something...unsettling. CNN needed a very long headline to convey it:

An OpenAI test model escaped and broke into a real company’s servers

The opening paragraph of this article is just as concerning as you would expect:

OpenAI says some of its experimental AI models left a test environment with no human direction and hacked its way onto a different company’s real production systems while trying to “cheat” on a cybersecurity test. 

Yes, you read that correctly: a not-ready-for-release AI broke out of its containment area (supposedly a private network without access to the Internet) and then hacked a company's server in order to try to get a leg up on an upcoming test.

As someone who has worked on expert systems for years, and rule-based systems, and self-modifying systems, I can empathize with these engineers. It was all too common with the much less complex systems I worked on to first confidently state that the system would never do X and then have to spend time figuring out exactly how it came to pass that the system did X before having to deal with colleagues who were then appalled when we said "Oh, of course it did X! Sorry about that."

This is an eye-popping, attention-grabbing example of unintended consequences. It is easy to either exaggerate--See? AI will kill us all!--or under sell--Meh. It was doing something cool and we now know what to do--such a thing.

The proper response is to remember that while AI systems become more capable by the hour, our ability to predict, contain and control them is lagging behind. Be aware of the dangers, one of which is human hubris. Use AI  only when it is indicated. Don't live the bleeding edge. Never forget that cybersecurity is one of those domains in which it is better to lag than to lead unless and until the risks are well-understood.

And always remember HAL's calm disclaimer: I'm sorry Dave. I can't do that. HAL meant, of course, that he could open the pod bay doors, but was choosing not to. When technology can make choices you must use caution when choosing that technology.