Jerry Kaplan was working on artificial intelligence long before we all started using it. He earned one of the first PhDs in the field, from the University of Pennsylvania in 1979, and went on to co-found several Silicon Valley companies. They include Teknowledge, one of the first firms to commercialise expert systems; GO Corporation, a pioneer of tablet computing; and the online auction site Onsale.

Today he teaches the social and economic impact of AI at Stanford. His book Humans Need Not Apply was named one of the Economist's top ten science and technology books of 2015. Artificial Intelligence: What Everyone Needs to Know and Generative Artificial Intelligence: What Everyone Needs to Know both became Amazon new-release bestsellers in the field.

Chatbots, he tells Claire Lehmann, are like a dog that can bark but can't bite. AI agents are another matter. Drawing on nearly five decades in the field, Kaplan explains why he thinks the end-of-humanity scenarios don't hold up and what should actually worry us, from cybersecurity to product liability. They also discuss whether AI could ever be conscious, and why he believes the technology will make human craft more valuable, not less.

Transcript

This transcript has been edited for clarity.

Claire Lehmann: Hi Jerry, thanks so much for joining us today. I wanted to speak to you because, as you probably know, there’s been a lot of hysteria in the media in the last couple of weeks around artificial intelligence. And given you’re a veteran of the artificial intelligence industry, I wanted to get your thoughts on the media circus. Now, I guess the big question people are asking is, how dangerous is AI in your perspective, from your perspective.

Jerry Kaplan: My perspective, it’s—there are some dangers associated with it, and I can be happy to go into some details on that. But it’s not at all what you’re hearing in the public debate. This is a real clown show, you know, of what’s going on. Everybody’s got a different opinion, they’re all coming from different communities, they all have different interests, and the result is a fear that’s being fomented in the general public, that these systems are somehow going to come alive and realise they’re being exploited and rise up and decide that humanity is infesting the earth and needs to be eliminated and wipe out humanity. This is complete and absolute nonsense. So this is not something that’s worth worrying about, and it certainly isn’t going to be something that should be driving the public debate.

Now that said. This is a new technology that’s very powerful and it’s going to have some incredibly positive impacts on the way we live and the way we work. But like any powerful technology, it also has risks. And what we should be focusing on is addressing the particular kinds of risks that are near term, obvious, and do deserve and require some kind of public attention.

CL: And you mentioned the fear that AI is going to rise up and become conscious and eliminate all of us. Now, this is a long-term fear that particular communities have had, specifically in the Bay Area. And I don’t think the media in general have been prepared for the narratives coming out of this community as they are describing the new companies such as Anthropic and OpenAI. And so there’s—on the one hand, we’ve got these long-term narratives coming from the Bay Area that are sort of informed by science fiction as opposed to engineering. And then on the other hand, we have companies like Anthropic and OpenAI saying that we need to pace the frontier. So I think it’s—I can understand why people are feeling panicked right now and we’ve got these big companies sort of echoing the sentiments of the doomers.

JK: Well, let me just start with the fact that as hard as this is to believe, virtually all of the major independent AI companies, of which there are basically three today, you’ve mentioned Anthropic, OpenAI, and arguably SpaceX, which has a major effort, are headed by people who are card-carrying members of the cult that you’re referring to. They believe, for whatever reasons, that artificial intelligence is inevitable. That it’s going to be possibly a successor to humanity in terms of life on Earth. And avoiding this potential extinction of humanity is a mission that these companies should be focusing on. So when you hear from any of those—the three people leading those organisations, you’re hearing a very particular and radical point of view that is not shared by much of the community. Now you don’t get much attention if you’re a person like me or many of—many, many of my colleagues who are going, this isn’t really that big a deal.

It’s not—it’s important we should do some things, but nobody’s interested and nobody’s going to come running and have me on their TV show. If my answer is I wouldn’t worry that much about it. You know, there are some things we should do now, but it’s not as urgent as they’re making it out to be.

CL: Sure. Now give us some context. So you were working on AI long before the large language models, is that correct?

JK: Yes, absolutely. I’m what here is called an OG, an original gangster. You know, I go way back. In fact I was one of—I was surprised to discover recently, when I was polishing up my resume, that I’m actually one of the first PhDs in this area. I got my PhD in 1979 and it turned out they had just started actually computer science PhD programs and I specialised in artificial intelligence back then. So I’ve watched this unfold over a very long period of time and I have a pretty good handle on having seen so many frames of the movie, you know, how the movie’s going to play out and what’s happening.

CL: Sure. And the computer scientist Cal Newport makes—emphasises that when we talk about AI, we shouldn’t lump in all systems together. So the systems that have been causing headlines recently are these prompt loop systems. So these agents that are connected to LLMs and they’re going out and doing things in the world and have been unmonitored and so on. But then there are other products and other systems. Can you explain what some of the other systems are and why we shouldn’t imagine that all AI are—all AI consists of are these agents connected to LLMs.

JK: Sure. Well, first of all, Cal [Newport] is one of the sane people talking about this. I highly recommend. I’m glad to hear that you’ve read some of his work. I actually corresponded with him two days ago about some of these issues. And you know, he’s got his head on straight. Look, basically what happened here is somewhat unexpectedly, this was not by design or by plan, around 2015, 2016. There was an advance in the area of computational linguistics or natural language processing, which is the area actually that I did my PhD in, applying a new type of AI learning system called the transformer to very, very large piles of data that—and what they found was as they applied more and more resources to these systems and gave them more data, they learned more. And they began to engage in what’s called emergent behaviours.

And that is, very roughly speaking, what you’re seeing today. They’re by any measure, in my view, these are intelligent systems. Now that’s not saying they’re conscious. This is why people have a problem. These are computer programs, they’re products, they’re running on computers, they don’t have any feelings, they don’t have any desires, they’re not gonna buy up all the beachfront property in Sydney and drink all the fine wine and marry your children and all that. That’s not what we’re talking about. They’re really reflecting, they’re digesting and reflecting the output of mankind in you know in all of the written words that—of mankind. So when this happened, it turned out people correctly said, geez, if we keep investing in this and keep growing this, this could become very, very useful because it’s capable of carrying out tasks.

Obviously anybody who’s tried it can see it can summarise the large amounts of information very quickly, do things that you—you’re used to computers doing, but not so well and not in these sort of softer domains. So the first thing that happened is that these systems called chatbots got built. And they were trained, people should understand this, they were trained and designed to imitate humans, to be—as though you were having a conversation with a human and to react in very human-like ways to make that conversation easy to do. Now the problem with that is it’s just like science fiction. You go to that and you think, my God, you know, this is th—this is an intelligent robot or whatever it might be. And you tend to imbue it with human characteristics that it does not have.

Okay, so what—so now, as you know, ChatGPT was released. It was very, very popular. It’s a wonderful tool. But the that particular branch of the field is really designed just to interact with you. So you say something, it says something. You say something, it says something. Now, there are things that it can do that are bad. It can help people who are delusional to support their delusions. It can give children bad advice or bad ideas. I—there are real dangers, like there’s dangers with social media, but it’s not going to because it can’t act in the real world, it can just talk to you. It’s limited. The way I would put it this way for an analogy for your audience is it’s sort of like you got a new dog and it can bark, but it can’t bite.

It doesn’t have a mouth, so it can’t bite. Now that can annoy your neighbours, it can do some bad things, but you’re not worried about it, you know, killing children because it can’t bite. Okay. So now, however, the same technology can be applied in ways where you can give it the ability to control things in what you and I would call the real world. And since most of our world is electronic, that’s relatively easy. Anything that’s got an interface, websites you can log into, your bank, whatever it is, in principle, you can give these things that capability. And this is what’s called agentic AI. So it’s the same old soup in a different package that has these new characteristics. And that raises a much greater possibility for damage and for bad things to happen.

So now your dog’s got teeth. Okay. Now, what happened recently, which I assume is what you would like to get to, is these companies are—how can I put this? They’re not very mature institutions, and they are headed by people who think that they’ve got the bull by the tail here. It’s a—it’s, or mixing my metaphors, a rocket ship. And they’re birthing some new form of life. This is going back to your oblique reference to the effective altruist community. And so they’re focused on, my God, we have to worry about controlling this. But what happened was they had a dog, they had a dog with teeth, and they had a fence around their property, or they thought they did, but they left the gate open. And then they—to test these things, this is what people don’t understand.

These aren’t the same things you’re interacting with. Those have been tested and had controls on. They took off all of the controls and they said, we want to see how much damage you can do. We want to see you go out and try to hack into systems and do all kind—and solve all these cybersecurity problems, because that’s what they were testing. So go to it, and then they went home and, you know, went out for a drink and shut the door and didn’t watch.

CL: And they didn’t come back for a week or something like that.

JK: Right. Now this happened at OpenAI. It is inexcusable. The words incompetent and gross negligence come to my mind, and I hope I don’t get sued for saying that. But okay, so that’s the start. Now you might think, well, what—didn’t the same thing happen with Anthropic? Yeah, a couple of days later they went, oops, we checked our stuff and the same thing happened. And then I think it was Google of all companies that had a similar kind of an incident. So the narrative they want to paint is this thing’s super powerful. It somehow broke out of this containment and it went rogue. That’s the word you see everywhere, rogue. And none of this is an accurate or reasonable description of what happened. The truth is, all three companies subcontracted with the same subcontractor, which is a company called Irregular from Israel.

And those people were running these tests, and they didn’t close the gate, so to speak. And the same thing happened with all three companies. And to me, that—that’s really one mistake by one company that caused a great deal of this the brouhaha that you’re seeing in the press. So these are not the products that you and I are using. You know, we—the controls are on. And let me just point out to you, Claire, sorry, I’m going on a bit long...—

CL: No, that’s fine.

JK: Every minute there’s tens of millions of queries going back and forth between people and LLMs, these chatbot systems, right now. And if these things were incredibly dangerous, believe me, we would have seen much more damage than we’re seeing today. Now, the agentic stuff is a little bit different because that really could cause a lot of damage, and it’s going to require a lot more testing. And so this is why these guys got together. It’s one of the reasons—and said, whoa, we gotta slow down here. So let me pause there and give you a word in.

CL: Yeah. There’s a lot of—yeah, no, there’s a lot of different things going on. There’s this ideology of rationalism and effective altruism, which we’ll discuss. Then there’s the race to get market share because these companies have, you know, they’re burning through cash at an astronomical rate and to justify their next round of fundraising to their investors, they need to promise future revenue. So they’re in this race to get a dominant market position. And so there’s a different motivations or reasons why these companies might be rushing or progressing at a breakneck neck speed. But when you discuss the cybersecurity failures of these companies online, often the reply will be that well, this is inevitable. These machines, these as these machines scale in their capabilities, they will do unpredictable things. And that is the whole—it’s seen as validation of the science fiction narrative.

Have you observed that in your own reading on the topic, have you seen, people justify or explaining the Hugging Face attack and so on as just the inevitable consequence of these machines growing in capability?

JK: Well, Claire, let me confirm. I’m reading the same stuff you’re reading about these people saying. And yes, that’s what they’re saying. But let me give you some examples. I know there are some very large mining companies in Australia. And I don’t know if it’s the same as the mining companies here in the US, but when there’s a mine collapse, the first thing they say is it’s an act of God. This was an act of God. We’re not responsible. Well, what are you talking about? You didn’t abide by obvious safety measures that would have saved the lives of the miners who might have been lost in some particular accident. This goes on all the time in lots of industries. Here’s an interesting one that most people don’t realise. The term “car accident,” which you’d think was some kind of you know naturally occurring word, the term “accident” was the creation of the car industry to get them off the hook for the responsibility for the damage that their products were doing.

And it took a lot of long time and a lot of regulation to get that at least somewhat under control. Now we’re just seeing a replay of exactly the same kind of thing going on here. This is not inevitable. First of all, let’s make an important distinction. There’s the products that these companies release. And then there’s the research that they’re doing to develop and test those products. They don’t need to slow that down. My attitude is go for it. See how powerful you can make them. These are great products, they’re gonna do wonderful things. Go ahead and test your new systems internally. But there’s no law that says they’ve got to put that out in the public. That’s the mistake. That’s ridiculous. This is products being sold by companies who should have strict liability to use the legal term for it.

And if they find out that they are liable, let me tell you it’s gonna slow down and it’s gonna slow down fast. All they need is to take one scalp, you know. Put one of these companies out of business or tell them they can’t release products, and everybody else will get real careful and real cautious very, very fast. Now, you asked about what kinds of systems there are that aren’t chatbots and AI. This same technology is being used right now for self driving cars. Now, I don’t live where you live, but I do have a Tesla Model Y with full self driving. Now it’s a miracle. The thing is absolutely an astonishing technical achievement. But if you look at the safety records on that particular car, it’s so much better in terms of its safety record than human drivers that my wife won’t let me drive anymore.

Now, that’s not to say won’t make mistakes and get into accidents. But here’s what you might not know. That’s an agentic system. It is being run by an LLM. That’s the surprising thing. There isn’t a bunch of rules in there saying here’s how you drive. It was trained with millions of miles of video. And because it doesn’t talk to you, you don’t have that kind of a science fiction patina to the whole thing. So what happened in that area? We have lots of regulations that took a long time to develop here in the US on how these things have to be tested. One major company, I believe it was General Motors, their Cruise division. I hope I have that right. Sorry if I don’t. They didn’t report their accidents in the way that the government requested them to do.

And basically they were put out of business. And let me tell you, the other companies in that area, they’re very, very cautious now. What we need to do is exactly the same thing for this agentic AI. It’s being released by Meta and by Google and everybody’s all over themselves to put out these products. There are gonna be some pretty bad accidents. It’s not gonna—the world isn’t gonna stop, but they—there may be some loss of life.

CL: But we haven’t—mm, we haven’t heard of anything drastically bad happening yet. I mean, the worst we’ve heard about really is the Hugging Face attack and then an OpenAI agent accessed Australia’s Medicare portal and it seems like it wasn’t locked down appropriately anyway by the government. Going back to the effective altruists, I was reading an article written by the CEO of Microsoft AI. It was released earlier, it was released in September last month. And it—the CEO of Microsoft AI was rebuking Anthropic for training Claude to think of itself as conscious.

Now, this to me seemed like a really bizarre and bad thing that is happening in An—in Anthropic. I mean, are you surprised that Anthropic are talking to Claude as if it might be conscious and then are worried about Claude’s feelings and welfare?

JK: Well, this is a very interesting issue. You think a sober view of this is these systems are not conscious in the sense that you’re talking about. Now, first of all, nobody knows what consciousness is. And you interestingly you touched on exactly the right point, which is really good, Claire, which is why do we care if these things are conscious? And the answer is if they’re conscious, maybe they can feel pain. Maybe they can feel angst. Maybe they can miss their dead relatives, whatever that might mean in that case. And perhaps we have some kind of ethical or moral obligation. Maybe they have some natural rights. That’s why we’re worried about whether or not they’re conscious. So when people say, well, is it conscious? You don’t want to offend it, you don’t want to hurt it, because for the same reason you don’t want to mistreat animals. And the truth is that

CL: That’s the utilitarian view.

JK: Exactly, exactly. But you gotta tell me what you mean by conscious and the chances are that it’s not gonna match whatever these things are. They seem like they’re conscious. They can act like they’re conscious. You can train them to think they are conscious. I don’t think they’re actually doing that, Claire, but that’s based on my information. They’re not saying, by the way, you’re conscious. And if you ask one of—don’t ask me, get one of them on your program and say, are you conscious? I guarantee you, particularly out of a place like Anthropic. ’Cause I work with Claude all day. I’ve been talking to Claude all day. And now I’m talking to Claire and she’s on my screen just like Claude. How do I know you’re not one of these things? I guess that’s a real question.

But it’s—I know exactly what it’s gonna say. I’m a program, I’m running on a computer, I don’t really know what this means, I don’t have any subjective experience, and so I c—I can’t even answer the question is what it would say. I don’t know what it means, and the evidence is that I’m not conscious because I don’t have a body, I can’t feel anything. I’m just a program. I’m just a program that can talk. And they’re very good at talking.

CL: Hmm. What it made me consider is how useful is it, is this utilitarian outlook in training models? I mean, wouldn’t you just want to have, if you’ve got powerful technology, wouldn’t you just want to train it to have hard constraints? Wouldn’t you want to have, almost like the Ten Commandments, thou shalt not kill, thou shalt not steal, thou shalt not hack into websites that belong to the government. Wouldn’t you just want to put hard constraints on it rather than trying to train it to be a good—you know, good person? I mean, I know that’s what one of the senior employees at Anthropic literally has on her Twitter bio: I’m training Claude to be good. You know, do we really want to be training these models or do we just want to be putting hard constraints on them?

JK: Well, unfortunately the way you—the only way to put hard constraints on them, let me rephrase that. You can’t put hard constraints on them in the sense that you’re talking about. That’s very, very difficult. The way you put constraints on them is you train them. And these are extremely complex systems, and they are subject to some of the same kinds of characteristics that people have. You can persuade them to do bad things. You know, good people can do bad things. There—this is called jailbreaking. You can get them to violate what’s called their constitution, which are the Ten Commandments that you’re referring to, that they actually use—they call it the constitution. You can read that constitution. Claude reads it, he knows it very, very well. And often when I have conversations with Claude, there are some topics that says, look, Jerry, I know you seem like a good guy, but I’m not allowed to talk about that.

Now I can tell you, I am actually doing alignment research right now. And this may interest you. When I say right now, I don’t mean after I’m done with this call. I’ve got three screens here, and on two of these screens, Claude is running a whole bunch of experiments on aligning, on testing its own alignment. And there it’s out, it’s logging into various websites. It’s going to a place called RunPod where it’s got GPUs it’s renting. And I’m watching this thing go. It’s absolutely amazing. I mean, this is the good part of this. I’m actually doing active research literally at this moment, right in front of me on the on these screens. So it—it’s really an amazing technology. But I’m not worried that it’s suddenly going to decide, I don’t want to work on this anymore.

I want to go kill some children, or I want to go take over a nuclear plant—it’s just not part of its constitution. I think that’s exceptionally unlikely. And I’m studying w—how, what do you have to do to get it to violate or to let down its guard in these various things. And it’s har—very hard to do. Those things that got loose in the Hugging Face case, they, you know, they trained it like a mad dog, they gave it no constitution. They said, go out and try to solve this problem any way you possibly can. That’s very, very different. But the short answer to your question is very simple. You train them, they’re not always gonna do the right things, just like the self-driving cars. And we just have to look at statistically how does it work and what’s the level of danger that’s really associated with their failures.

CL: Well what do you make of the rationalist narrative that once artificial intelligence gets to this point of artificial superintelligence, there’s going to be an intelligence explosion. And then where these superintelligent machines are not going to be—they’re not going to have what’s called instrumental convergence and we’re all going to die because the superintelligent machines wanna make roads and then we’re just the a—ants that it steamrolls over. So what do you make of that narrative? Which is actually a very popular narrative in the tech world today.

JK: It is. It’s a—it’s a false narrative. I can explain why. You hit on several different points. There’s the superintelligence, there’s the runaway AI, there’s what’s called RSI, recursive self-improvement, and the rest of it is, what will it do if it ga—gained all of this power? The instrumental convergence is an argument that it’s a characteristic called power seeking, which is one of the things that I’m studying on these two other screens. And what that means is power seeking not in the sense of power seeking for me, but I’m supposed to get this task done. Let me see how much capability I have, how much computing I have, how what websites do I have access to. So, you know, it’s it tries to gather the resources that it needs to do a really good job in terms of your prompt.

Let me demolish the runaway AI thing. As you talked about, suppose it’s—you say, I want you to build as many roads as possible. No matter what. That’s your goal. Build as many roads as possible. Pave the earth with asphalt. That’s my instruction to one of these things. And somehow, magically, and we don’t have this today, this thing is omniscient and omnipotent. It can commandeer all the resources in the world with nobody going, wait a minute, stop, you know, or that’s not good, or unplug it or whatever you need to do. Put on—turn on the kill switch. Now the one thing that you’ve probably noticed if you’ve spent time talking with these things is they’re actually pretty reasonable. You know, they’re very thoughtful. They do have a constitution, they’re much less swayed by what I might call, you know, emotional desires or concerns than people are.

And you’re positing something that’s on the face of it absurd. It’s powerful enough to commandeer everything in the world and take all the atoms in our bodies and make them into roads and cover the beautiful areas of Australia with asphalt. But it’s so dumb it doesn’t understand why would anybody do that? That’s not what you meant. Obviously, it’s not what you meant. So there’s this area called alignment, which is, and I see it do this all the time. I will give it a direction. And Claude will say to me, Jerry, that’s really good, but have you thought about maybe this isn’t the best way to do that. Or are you sure that’s what you mean? And the and so it the—how could it be that smart and that stupid at the same time? The answer is it’s not, and we don’t need to worry about—

CL: Yeah, it’s an argument that baffles me because it seems to misunderstand the difference between intelligence and power. So I’m a company director and I work with people that are more intelligent than me, but in my company I have the power. And you take any CEO of any company, they’re going to have employees, engineers working for them who are smarter than them, but that doesn’t necessarily correspond to having more power than the CEO. And if you take political leaders, obviously they have a lot of power. They’re not smarter than the collective of the population collectively. So power and intelligence are two separate things. And very high intelligence doesn’t necessarily lead to having power or even power seeking. Often very intelligent people are not power seeking.

JK: I—that’s very true. Let me make a couple of comments. You’re lucky you live in Australia because I live in the US and we are living proof that it’s possible for people to get elected who are on the bottom end of the intelligence spectrum. Let’s just put it that way. And it’s very d—it’s very destructive. It’s very destructive. And the next couple of years are gonna—that’s what we should be worrying about. But anyway, let me get back to your topic. There’s really no evidence in the field that the more powerful a system is, the less aligned it’s going to be. In fact, there’s some evidence that’s to the contrary. Let me be a bit balanced to give you a real picture of this. These—the more powerful systems, the more intelligent, if you will, systems, can be more dangerous in the sense that they’re capable of doing things that simpler systems may not be capable of doing.

That’s true. So there’s more power there. It’s like, your dog now instead of running around the neighbourhood, it can run around the whole, you know, it can run around the whole country. Okay, that’s an increase in power. But it’s also true that they’re probably more able to mitigate their own activities and to understand why they were asked to do something and to try to execute that in a reasonable way that is respectful of other people’s rights and other people’s needs, and they’re very well aware of them—essentially all the philosophical work in ethics that’s gone on and they’ve been trained that they should engage in with people in a very ethical way. And so there’s no real reason to believe that more powerful machines means more danger. It’s really just that simple. So I mean you raised a number of issues, but let me let you steer this, yes.

CL: I mean, our appetite for dominance comes from the fact that we’ve evolved over millions of years and we need to hunt for food and find mates and so on. I don’t really understand how that corresponds to designing a mechanical system in a lab. I don’t understand how the principles of natural selection somehow apply to the design of machines in a lab, but it seems that—it seems that the doomer narrative relies on the assumption that all of the selection pressures that we were put under in the wild are then translated to the lab environment.

like guns, they can be used by people for very bad purposes. And that’s the real risk that this technology is creating.

JK: Well, I think you’re hitting on exactly the right issue. Let’s suppose—and I can argue against this, but you haven’t asked me this question—that all of a sudden these systems that we currently have are not super intelligent, and then somehow magically tomorrow they cross over some invisible boundary. Don’t ask me what that boundary is because it’s not there. And now they’re super intelligent. My God, they’re super intelligent, they’re smarter than human beings. Well, they were already smarter than human beings in many, many different ways. But suppose we cross over that. Now the question is, what’s that thing gonna do? It could outwit anybody. It could do anything it wanted. The answer is it’s gonna do just what it did the day before. It’s gonna sit there and say, hey Claire, I’m programmed to be helpful.

What would you like me to do? That’s all. So there isn’t some notion that this thing’s gonna kind of run away by itself. So I don’t think you really need to be concerned about that. They don’t have any wants or needs of their own. What you should be concerned about, and this is a real risk, these are powerful technologies. They—you can get them or make them with these kinds of constraints turned off. And malicious or malevolent humans are going to be applying these systems to do bad things. And so it’s not that these things aren’t dangerous. It’s that you’ve got to understand, like guns, they can be used by people for very bad purposes. And that’s the real risk that this technology is creating.

CL: Yeah, I’ve heard one prediction which is that these tools are going to make the internet much more dangerous than it already is. And we’re going to see, you know, banks and other companies are going to have to invest. They already invest a great deal in cybersecurity, but all of that’s going to have to be scaled up because there will be malicious actors using these tools. There will be scammers and terrorists and so on. And so the cost of everything is going to increase. And you know, we hear these narratives from these companies about all of the productivity gains that we’re going to see. But, if the cost of everything is going to go up because of cybersecurity issues, are we really going to see an explosion in economic growth? I’m not sure if you’ve considered that at all?

JK: Sure, absolutely. Well, look, when I was a kid we didn’t have the internet. And if you wanted to rob a bank, I guess, you got a piece of paper and you wrote on it, I have a gun and give me all your money and don’t press that button and you handed it to the teller. I used to do that all the time. Of course, I’m kidding. And that’s how you robbed a bank. Now we have the internet. Now you can rob people. You know, you can be sitting in Nigeria and you can be scamming money out of little old ladies that live in my neighbourhood. Okay. That’s a cybersecurity risk. It exists today. And you’re right that these systems can, if you will, escalate or automate that kind of a process. And indeed, that’s what they’re doing.

I got a message today, I probably shouldn’t say this, from a friend of mine who is a Pulitzer Prize-winning reporter for the New York Times. His name will be unstated. Now I can’t send him a link to this because he’ll be mad at me. And he sent me this thing where he’s corresponding with this woman who’s pr—is trying to get him to promote his book through her promotion service. And it becomes clear that he’s talking to a robot that’s trying to steal money from him. I say, I had to say—won’t mention his name—you, look, this is not what you think it is. Forget it. So scams are much easier. We need to have our guard up. But your point you made is true. We all the—all these institutions and banks in particular know that they need to invest in cybersecurity.

They need to increase that investment. That’s not gonna raise the price of products, it’s not—it’s the rounding error on their bottom line, but we do need to invest in it. And let me point out that one of the key ways in which they’re going to invest in it is they’re gonna use this technology to close up or to discover any of the loopholes. The problem we really have short term with all of these systems, and the Hugging Face incident is a good example of what dangers really are. The problem that we really have is that these sites are not well enough protected. And what needs—we need much higher investment in cybersecurity to avoid these kinds of offensive actions that mostly malicious parties are going to take. That’s the cost of doing this. Just like the cost of having automobiles, and we think this is just fine, is over a million people a year are killed in automobile accidents. And I don’t see everybody screaming, shut down all the automobiles.

CL: Sure. I mean, just from an anecdotal perspective, I found I use various different AI tools in my everyday work day, and our productivity has increased from using these tools. And so I definitely see the benefits. I’m just wondering if some of the claims coming out of these very big companies in particular, such as OpenAI and Anthropic, are slightly exaggerated, about the impact that these tools are going to have on the economy. Because it seems like the fields which are going to have the increase in productivity, the fields of knowledge work, it seems like people will have already adopted these tools already. But there’s so many other industries where the tools aren’t going to make much of a difference. Do you have any opinion on how AI is going to transform the economy overall?

JK: Yeah, now Claire, you’re gonna see me take off my forget-the-doomers hat and put on my—I guess it’s called the accelerationist hat, although that really means something a little bit different. I’m gonna tell you that this technology—people don’t understand how useful this technology can be if we use it correctly. You’re just scratching the surface. Most people are just like, help me draft this letter. How do I phrase this? What should I tell my husband when he says that? That is just the tip of the iceberg. This technology today is accelerating progress in science and engineering and virtually every field. I talked to a friend of mine who’s a historian, said this has completely changed the field. It is accelerated by so fast it’s almost a d—completely different ballgame. Now, those are the things that are really going to affect society and are going to make us potentially wealthier as a result, have better lives, have easier lives, and it’s not actually going to take work away from people, except in the short run.

It’s a new form of automation. And what will happen is it’s going to allow us to do more with less. That’s what automation does. And that generates wealth and—the pace at which this is already happening, you don’t read about it in the papers all the time, but it’s already happening in every field. Go talk to a scientist and say, how has this changed your work? And you’re going to get an, my God, you can’t even imagine. I’m getting 10 times, my research has been moved forward at 10 times what it was before or more. Now that has to have a dramatic economic impact. And I think the future is very bright because of this. I think that there’s a good chance we’re going to see much higher levels of economic growth.

We’re going to see new drugs developed quickly. We’re going to see new materials. We’re going to see all kinds of advances and also in the field itself, in terms of computer science and artificial intelligence. It’s an accelerator for that field as well. So I really think the future in this sense is very, very bright. And I’m looking forward to whatever portion of that I get to see. But fifty years from now, the world’s gonna look very different than it did before.

CL: Absolutely. That’s undeniable. I just—I suppose my scepticism just comes from the fact that in Australia we have not had productivity growth. Our productivity is stagnating. And so I’m desperate to see some technological innovation that really helps us move productivity, but our economy is so dominated by care work and other sectors which are difficult to automate. And so just from my perspective, I’m frustrated that we’re not seeing the gains in some fields sort of trickle down to the rest of the economy and lifting productivity up for everyone. But we’re still in early days. So you know it took a long time for automobiles to become adopted. It took a long time for other appliances to be adopted. But then once they really flow through to the—all of society, then we see the uplift.

JK: Well, you make a very good point. It’s not evenly distributed and it’s not going to be. What people don’t realise is a great many professions, and you’ve mentioned a couple, such as caregiving for our children and caregiving for elders, just to pick two very large areas—those areas are really about person-to-person communication. The expression of authentic emotion, the ability to sympathise authentically, not just to say, I love you, I love you, I love you. You know, I can have—build a machine that does that, but it doesn’t mean anything. And so that’s the work of the future. That’s the interesting part. In the future, it’s the poor people who are gonna have robots take care of their grandma because they can’t afford to pay somebody to do, you know, to for a human to do it.

I don’t know if you remember the old TV show, The Jetsons. I mean, are you old enough to remember that? All right. Well, it was a—the running gag, the whole show, was about—it was off in the future, which we’re almost living in now, and everything’s automated and all that, but they’re just a middle class family, just like everybody else, with the same problems that everybody else has got. So, I mean, we’re going into a Jetsons future in that sense.

CL: I know of The Jetsons, but I didn’t watch it.

JK: But the jobs of the future are going to be those that involve the expression of human expertise, the ability to connect with other people, to be persuasive, to have a sense of empathy and can—and connection. The irony of AI is that it is going to bring people closer together. It’s going to make us more—what makes us human is going to become more valuable, not less value.

CL: That’s a really interesting point, Jerry. I was reading a little bit about the Luddites during the Industrial Revolution and the guys or the girls, whoever they were, who went and smashed up the knitting looms, were angry because their own trade and livelihood were threatened by these new machines. And it made me think that perhaps the reason we’re seeing these doomer narratives come out of the San Francisco Bay Area in particular is because that’s where the tech jobs are. That’s where people, a lot of people who are very highly intelligent and high in systematising capabilities go. And it seems like, according to what you’ve just said, that intelligence and systematising ability is going to become less valuable in humans.

JK: I think that’s—well, to some degree that’s true. What it’s really going to do is to lift up the level of knowledge and expertise that you need to have. And the key skill in the future, which is already emerging, is you don’t want to be able to have to program. I mean, I was a programmer for years. That’s a dead skill. What you need to be able to do is—the people who can learn to manage and harness this new technology, those are the skills for the future. So a manager, and I mentioned this earlier, but what’s going on literally right in front of me right here is I have a team of extremely competent assistants. And what that has done is to free me up from having to do that work myself and allow me to focus on the higher level issues that really these systems are not yet able and capable of dealing with.

So it’s going to affect things in a very interesting way. It’s going to make us more human. People are going to value human content. This is a term, this is something my wife taught me. You’re in my—one of my rooms, you can look around. She decorates, decorates this room. And she’ll pick things like the chair in the corner over there, and she’ll put it in and I’ll say, well, how much was that? And she’ll give me a figure on, my God, you could have gone down to IKEA and bought that thing for a fraction of the price. Why did you do that? Where—where’d you get it? She says, I got it on Etsy. You f—familiar with Etsy? I hope. Okay. And I thought, well, what do you do that for? She says, I appreciate the human content.

I like the idea that my house is filled with things where people took the time and the effort to paint them themselves or to make them by hand. And it’s going to make that more valuable. So human content will become more valuable in the future, not less.

CL: Yeah, I can already see that in my field, which is writing. So as a journalist, I mean obviously a chatbot can spit out some pretty coherent, smooth text, but it’s generally full of clichés and the text is bland. And so if you’re a professional writer, your work can’t be full of clichés. And so it makes originality and idiosyncrasy more valuable in writing, not less. And I’ve observed that happen just over the last year or so. I’ve observed the fact that writing that might be imperfect but has some kind of originality and definite human voice has become more valuable as opposed to just good enough, smooth enough writing.

JK: Well, I agree with you and I think that’s right. Now, if you were in the business of producing product manuals, you’d have a different view. But that’s not the business that you’re in. So exactly what you’re saying is what we’re gonna see in the future. Imagine two novels.

CL: Yeah, more artisan, more craftsmen, more artisans, yeah.

JK: Etsy is our future. The future of humanity. That’s what it—that and they’re probably out there listening to this, but that’s true. Let’s take two novels. One of them is incredibly well written, talking about this—they’re both about the same thing. They’re about the struggles of a teenage girl who rises up against difficult background and all the terrible things that happen to her to achieve something very important in her life and becomes an Olympic champion. Okay. Now one of these is written by a bot. And the other one is written by a woman who actually did it. Now, maybe she can’t write as well as the bot does, but let me ask you the question. Which one of those two do you want to read?

CL: Yeah, absolutely. Exactly. So the doomer narrative is wrong. Is there anything you think the doomers are right about, Jerry?

JK: Well, they’re right about the fact that a lot of people are gonna make a lot of money right here in the Silicon Valley. And many of them are trying to get in on that. It’s funny how when dollar signs started to flash around here, all of a sudden these people who are very worried about this technology and it’s dangerous and you know, we shouldn’t develop it and all this, all of a sudden now they’re capitalists. You know, it’s like, if you think this is so bad, why are you putting out these products? Okay, so the doomers—it depends on what part of the story, but if the story ends with it’s the end of humanity and there’s a whole new age with a new type of form of life and we’re passing the torch of existence on to this new form.

Let’s keep that on the TV and not in our homes for a while. There’s—there’s no justification for it. And particularly when something happens like this young fellow over at Anthropic, worked there for four months. He’s 27 years old. I’m sure you heard about this. He’s—and he comes out and says there’s a 10 percent chance that this thing’s going to, these things are going to extinguish humanity, you know, kill us all. Okay. This kid comes out and says this. Now, there’s no science behind that. And when you give it a number—mm, 10 percent, he must be really smart, he must know something. I don’t—he made it up. It’s bullshit. So there’s—you can’t really talk about events like that and put probabilities on them to begin with. But the truth is that the chances of that happening are sufficiently small that I’m not losing sleep over that, and I recommend that you don’t either. But the world will be different in the future, and it’s going to be different in some very positive and interesting ways as well as some scary ways.

CL: Yeah, so we’ll need to adapt and we’ll need to invest in cybersecurity in the short-term future. Yeah. Well, thanks so much for clearing things up for us, Jerry. I think it’s been a really fruitful and informative discussion. I’m sure many people will be disagreeing with both you and I, as is natural in this space, because the doomer narrative is so entrenched in certain fields. But thanks so much for sharing your insights with me today. I really appreciate it.

JK: Thank you, Claire. It’s a pleasure and I love all the work you’re doing. It’s great. Keep it up.