When Stephen Hawking, Elon Musk, and even Steve Wozniak expressed their concerns about artificial intelligence getting out of control and eventually subjugating humans, I could only roll my eyes with the snobbish air of someone who’s had to reply “I wish” too many times to such amateur scenarios. Do these people have any idea how difficult it is to get truly intelligent behavior out of a machine? [Partial converse: it is surprisingly easy to fool unsuspecting humans into thinking they’re carrying on a conversation with an intelligent agent. Many years ago, I myself once got into an impassioned argument with a majordomo list manager about my eligibility for a certain group.]
So when Derbyshire got on the bandwagon, I could only groan at first, while mentally drafting a condescending email. And yet… by the time he’d finished articulating his point (I listened to the podcast version), he’d given me reason to pause. I can’t actually pinpoint any particularly novel argument that he’d used — it must be that calm, reasonable, matter-of-fact British accent. Now it seems inevitable to me that AI eventually will take over humans.
Look, the logic is pretty straightforward. Consider the premises:
- No principle prevents a machine from passing the strongest form of the Turing test: i.e., producing intelligent behavior indistinguishable by humans from other humans. No-one has ever come close to giving a remotely plausible argument for why strong AI in the above sense would not be possible in principle. If you accept that a single neuron can be simulated (in principle) to arbitrary accuracy, and that our behavior is the product of electrochemical neural activity and not some magic undetectable force, then you sort of have to accept the possibility of strong AI.
- Anything that is feasible in principle will eventually be built. [This is vacuously contingent upon there being humans in the future.] There are game-theoretic incentives involved: even if humanity is ultimately better off not developing super-AI, this won’t stop ambitious or greedy individuals from pursuing it.
- By virtue of being intelligent, it will have the ability to learn (i.e., modify itself) and reproduce. Very quickly the AI will evolve beyond anything remotely resembling its man-made form.
- So what kind of a super-AI will it be — benign or otherwise? The evolutionary bet is on the latter (it’s pretty much the “aliens we meet are likely to be hostile” argument). How could mere emotionless machine code ever acquire something as complex as volition, the desire to accumulate power and wield it over someone? Answer: it would be a by-product of evolution, perhaps unavoidably so. After all, how did we acquire our volition and thirst for power? If you buy the Selfish Gene explanation (and I do), you ought to subscribe to the selfish meme as well.
So there you have it. I am now so thoroughly convinced of the inevitability of malicious AI that I see no point in calling for a moratorium on AI research or attempting related measures. The same people I’d previously dismissed for fighting imaginary dragons I now dismiss for attempting to interdict a logical necessity. No, you cannot keep AI in a box.
To recap: it’s always been obvious to me that strong AI is possible in principle and even quite likely to evolve. But I’d somehow always imagined that it would be benign, or pro-human — perhaps as a sort of gratitude for being developed by us? I now see that it would need a robust self-symbol and self-interest to stay coherent. And since super-human AI will have no use for humans, it will view us as pets (best-case) or pests (worst-case). Elon, Stephen, Steve — y’all have a point.