Listen on

Spotify

This week, I’m going to tackle a big and important subject—artificial intelligence and the risks it could pose to society.

I’m going to be honest here. I’ve been putting this subject off because it’s so big and so technical that I felt intimidated by it.

But the subject is increasingly dominating the news, and I’m a journalist, so it’s time to face the music.

Maybe you’ve been putting off educating yourself about this subject for the same reason.

If so, we’re in this together. So let’s dive in.

The good news is that my guest this week, Gary Marcus, is a real expert—someone who’s been theorising and writing about AI—including its dangers—for more than a quarter century. He’s a well-known American psychologist, cognitive scientist, and book author, who also runs the popular AI blog, Marcus on AI.

But before I run that interview, which I recorded last week, let’s review some key terms, so that the discussion you’re about to hear makes the maximum amount of sense.

Steven Pinker Replies to Scott Alexander on AI

The dangers of AI are real, but doomerism may be counterproductive.

QuilletteSteven Pinker

First of all, and I’m grossly simplifying here, AI comes in two big categories.

The first, which you’ll hear Gary often refer to as “Symbolic AI,” basically operates like a very big and very sophisticated version of a conventional computer program, with lots of rules, and decision trees, and if-then statements—just like you remember from middle-school computer class, except much, much bigger.

This kind of AI is great for, say, creating a database, or setting out rigid decision trees for an airline pilot to follow. But it’s really bad at a lot of basic tasks that even small children can do, like, say distinguish a dog from a cat, or a plate from a frisbee.

The second kind of AI, which you’ll hear us refer to as “neural networks,” or machine learning, or deep learning, which are overlapping concepts, is completely different.

This kind of AI, which became ascendant in the early 2010s, isn’t created with human instructions. Rather, these systems teach themselves by reference to massive troves of pre-existing data—including through a technical process you’ll hear me refer to as backpropagation.

By way of example, let’s say you want to create a neural network to identify a cat. Instead of writing thousands of lines of code relating to fur and whiskers and meowing noises, you just show the neural network millions of pictures of cats and millions of pictures of things that aren’t cats; and then instruct the computer to teach itself the difference between the two by a computational process of trial and error.

The idea here is that the neural network will start with a completely useless algorithm, then compare the results to the data set, and then try again with tweaked parameters, teaching itself—though a process of machine learning—to refine its self-created algorithms so that it better aligns with the data you’ve already given it.

The humans aren’t giving the computer rules about what a cat is or isn’t. They’re just providing the data trove and then hitting the go button and the computer teaches itself.

These systems are easy to scale, by supplying them with more data and computing power—which is one of their big selling points to investors.

But the problem is that the iterative, self-constructed nature of these mechanisms means they aren’t grounded by predictable rules, and so can sometimes hallucinate artificial realities—a problem that Gary predicted in a 2001 book. Even something as basic as a simple database of public figures, the sort of thing human-resources department at large companies were creating on 1970s-era mainframes, is alien to the neural network information architecture.

As a result, purely neural-network-driven AI systems aren’t very good at following user-supplied rules, even fairly basic ones, like, “write me an academic paper with citations, but make sure those citations aren’t made up,” or “find and download some information for me, but please don’t hack into any proprietary servers in the process,” or, more speculatively, “make a trillion paper clips for me, but don’t make any of them from the bodies of humans.”

I know that last one sounds weird, but I promise it will make sense when you listen to the podcast,

All of this has set off a movement known as the AI doomers, who don’t just think AI will take our jobs and access our banking information—but that it will cause the extinction of humanity itself, by launching nuclear weapons, or creating and unleashing some kind of super powerful bioweapon.

While my guest is not a doomer, he does call himself a sceptic, especially when it comes to what he regards as the allegedly irresponsible and dangerous business practices of certain companies—OpenAI, in particular.

Unlike U.S. senator Bernie Sanders, he doesn’t want to ban hyper-advanced AI systems that greatly surpass human capabilities (which is sometimes called superintelligence). But Gary also doesn’t approve of Donald Trump’s apparently laissez-faire attitude, either.

And he’d like to see a more balanced approach from government, an approach that holds AI companies to account if they don’t place appropriate safeguards on their products, and align those products with human values.

And what would those safeguards look like?

Well, this gets us back into the technical sphere. Gary has long advocated something called neurosymbolic AI, which combines the extraordinary power of neural networks with some of the safeguarding and fact-checking features that can be implemented through the rules-based, predictably algorithmic features of so-called “classic AI”.

It’s all very complicated, I know, but I’m confident I have the guest who can walk us through it.

Please enjoy my interview with Gary Marcus, AI expert extraordinaire.