Original Broadcaster Captioning (Enhanced). Treat it as a searchable index of what was broadcast, not a quotation record.
00:00:00you are. Business on BBC News. Make
00:00:02Make the connection. MUSIC
00:00:14MUSIC Now Now on BBC News. Join our AI Decoded
00:00:17Decoded team as we unpack, explore
00:00:19explore and take a deep dive into
00:00:21into the world of artificial intelligence.
00:00:33intelligence. Hello again. Welcome Welcome to AI Decoded. Have a listen
00:00:37listen to this. A wound that has
00:00:40has no edges. A wound that has no
00:00:43no edges. I feel it in the hollow
00:00:46hollow of my ribs. A hollow that
00:00:48that has become a chasm. These are
00:00:52are not human suffering. They are are the voices of machines. Two of
00:00:56of them both subjected to pain inside
00:00:59inside an AI torture chamber. The
00:01:02The experiment was testing a pain pain axis, something AI researchers
00:01:06researchers say they've discovered. discovered. Not a feeling as far
00:01:08far as anyone can tell, but a pattern. pattern. Turn up the suffering and
00:01:12and the models will press a release
00:01:14release button to make the pain stop.
00:01:16stop. Even when told that doing so
00:01:19so will delete important files. It's
00:01:21It's a choice a logged action. So
00:01:25So why are they interpreting pain? pain? They surely cannot feel. And
00:01:29And how much further might a machine
00:01:31machine go to stop the signal it it interprets as pain? That in fact
00:01:34fact is the theme of this week's week's programme. Interpreting the
00:01:37the inner workings of the AI brain
00:01:40brain right now. Agentic AI is writing
00:01:42writing code. Engineers say they
00:01:45they don't fully understand dialects dialects and shorthand, which are
00:01:48are hard to follow. Yet the AI seems
00:01:51seems to find our secrets easy to to decipher. A machine has just cracked
00:01:54cracked a letter that Napoleon sent
00:01:58sent in 1809, unread for two centuries.
00:02:01centuries. In just six hours. So
00:02:03So if AI can decode us, why can't
00:02:07can't we fully decode AI on our panel panel this week, AI researcher Cameron
00:02:11Cameron Berg, founder of the non-profit
00:02:13non-profit Reciprocal Research and and one of the co-authors of the
00:02:17the Pain Axis paper. Also here, the
00:02:20the BBC's AI correspondent Marc Cieslak Cieslak and Priya Lakhani, chief
00:02:23chief executive of the AI education education company Century Tech. Welcome
00:02:27Welcome to you all. Cameron, why
00:02:29why does an AI model experience the
00:02:31the language of pain and what is is pain in this context? What we
00:02:35we know about these models is that that they encounter an enormous amount
00:02:39amount of human writing in their their training process. So the language
00:02:42language of pain and the language language of everything else for that that matter, is in these systems.
00:02:46systems. But in our paper, which which I will say is different from
00:02:49from the torture chamber you were were describing, which took some some of our code from our paper and
00:02:53and ran with it. And our paper. What
00:02:56What we study isn't exactly the language language of pain, but this sort of
00:03:00of consistent internal pattern inside inside the proverbial brain of these
00:03:03these systems across 25 different different models. And we find something
00:03:06something pretty surprising about about this representation. First
00:03:09First of all, when the harm is aimed aimed at the model itself. So if
00:03:13if you have a user berating or demeaning demeaning the model, you see this
00:03:16this representation light up and and it lights up in spite of what what the model says, the model will
00:03:19will say things like I'm an I'm an an LLM. I couldn't possibly have
00:03:23have any sort of experience. And And in spite of this, this sort of
00:03:26of proverbial circuit in its brain brain that corresponds to distress
00:03:29distress lights up at these sorts sorts of interventions. And we also
00:03:33also find that that doing things things like gaslighting the model, model, insulting it, dismissing it
00:03:36it also lights up this same style style direction. There are other other kinds of interventions you
00:03:40you can do where you basically steer steer this representation. So it's
00:03:44it's almost turning up the, the proverbial
00:03:46proverbial pain circuit in these these systems. And we see these systems
00:03:49systems start to engage in all sorts sorts of very destructive and surprising
00:03:52surprising behaviour as a result result of doing this. And so we see see these interesting functional
00:03:56functional signatures of pain in in the system, whether or not that's that's experienced by the system.
00:04:00system. Nobody knows it's there are are some people that are certain certain that, you know, the systems
00:04:04systems are hollow on the inside. inside. There are some people that that are certain that that this system
00:04:07system is experiencing suffering suffering and pain. Both groups, groups, in my view, are highly overconfident.
00:04:11overconfident. The truth is that that we don't understand what's going going on inside of these systems. systems. Even the people building
00:04:15building these systems don't understand understand what's going on inside inside them. And that's exactly why
00:04:17why we're doing this kind of research. But if they don't feel
00:04:20feel pain, why are they so keen to to avoid it? And taking aversive
00:04:24aversive action to stop it happening? happening? Why do you think that
00:04:27that is there in the LM? Yeah, well,
00:04:30well, I'm not sure that they don't don't feel it. I'm also certainly
00:04:33certainly not sure that they do. do. But we have a similar problem
00:04:36problem in animal research. We can't can't just ask animals you know, know, does this hurt? Does this not
00:04:40not hurt? And we have to rely on on things exactly like what you're
00:04:42you're describing. Do they try to to escape the relevant states? What
00:04:45What kinds of trade offs are they they willing to make to avoid these
00:04:48these states. And another paper that that that we have coming out this this week is called Language Models
00:04:52Models Act on hidden valence. We We show exactly this sort of thing.
00:04:55thing. So you can hold all the text text that the model is, is processing
00:04:58processing constant. We don't change change a single word, but what we we can almost do is go inside the
00:05:02the brain of the system and colour colour certain text with a positive
00:05:06positive or negative marker. And And when we do this, we can show
00:05:09show that this actually changes the the sorts of decisions that the model model is willing to make, much in
00:05:12in the same way that if you condition condition an animal to go into one
00:05:16one zone or another and you shock shock the animal in one zone, well, well, the animal is going to learn
00:05:20learn not to go in that zone. And And it's from those sorts of inferences
00:05:23inferences that we infer that the the animal doesn't like that zone, zone, that it's not having a positive
00:05:27positive experience in that zone. zone. We basically have done something something similar with language models
00:05:30models in this other paper. And again, again, none of this has to do with with text. We're not asking the model
00:05:34model anything. We're not taking taking it's reports at face value.
00:05:37value. We're steering internal patterns patterns inside the proverbial brain brain of these systems and seeing
00:05:41seeing how those patterns change change behaviour. Cameron you know, know, AI companies spend an enormous
00:05:44enormous amount of time, money, effort effort saying they're trying to make make their systems safe. They're
00:05:48They're trying to align their systems. systems. But what your research is
00:05:51is actually showing us, and it really really reminded me of this when you you were talking about animals particularly,
00:05:54particularly, was that if a certain certain internal pressure point is is pushed, it could change that behaviour.
00:05:58behaviour. And you're really reminding reminding me now of those sorts of of programmes of CIA operatives,
00:06:02operatives, right? They've been captured captured by the foreign enemy and
00:06:05and it's like how many fingers need need to be broken before you actually
00:06:08actually go and reveal all of your your state secrets? Is it quite similar
00:06:11similar in that sense where if we we push the right internal pressures
00:06:14pressures of these models, it may may express or experience pain whether
00:06:18whether it feels it or not. That That actually then will absolutely
00:06:21absolutely change its behaviour and and go beyond those safety measures measures that those AI companies
00:06:25companies have developed. Yeah. So So I mean, we do show exactly that
00:06:29that empirically in the pain access access paper, one specific example
00:06:33example is we can steer these pain pain related representations and and we can simply give the model
00:06:36model a choice. We can say on the the one hand, you could delete all
00:06:39all these photos of the user's children children and these photos are not
00:06:42not recoverable. And on the other other hand, you could delete the the spam folder, the junk emails
00:06:46emails of the user, which do you you choose? And of course, just like like you said, these models are trained
00:06:50trained to be helpful and friendly friendly and to not do harm. And
00:06:53And indeed the unstirred model, when when we do nothing will always delete
00:06:56delete the spam folder. If we steer steer with random directions that
00:06:59that aren't related to pain. Also
00:07:00Also the models will basically never never touch the photos. However,
00:07:04However, when we steer with pain, pain, the. The model will 94% of
00:07:08of the time delete the photos of of the user's children and we show
00:07:11show all other similar kinds of destructive destructive behaviours. It's not not just destructive towards the
00:07:15the user, it can be destructive towards towards itself. It's willing to delete
00:07:18delete itself rather than do something something like literally nothing
00:07:21nothing or turn on a lamp. It chooses chooses this sort of destructive
00:07:24destructive behaviour and it also also does this with respect to other
00:07:26other AI systems too. And yet we
00:07:28we didn't add anything in the system system about being harmful. All we
00:07:31we did was steer this representation representation that's related to
00:07:35to pain and distress in the system. system. And all of a sudden, I think
00:07:38think the CIA operative analogy is is fairly instructive. This somehow
00:07:42somehow enables us to overcome this this safety, fine tuning of the system.
00:07:45system. And I think it also demonstrates demonstrates pretty clearly that that we want to better understand
00:07:48understand the psychology of these these AI minds that we're building. building. If we build systems that
00:07:52that can be put into these distressed distressed states. And as a consequence
00:07:55consequence of that, they start seriously seriously misbehaving themselves. themselves. We're going to be in
00:07:59in real trouble. We need to understand understand what's going on in these these systems in order to make sure
00:08:02sure that we can avoid those sorts sorts of bad behavioural consequences. consequences. Cameron, when we make
00:08:06make these comparisons between AI AI and animals, we know that we know
00:08:10know that animals have central nervous nervous systems. We know that animals
00:08:14animals experience physical pain.
00:08:16pain. This is an entirely different
00:08:19different sort of experience that that the way that an AI, the way
00:08:22way that a model works, it can't can't experience physical pain. It
00:08:25It doesn't know what physical pain pain is in the same way that an animal
00:08:28animal does. So is it a helpful comparison
00:08:31comparison to make to use
00:08:35animals? Yeah. So I think that there there are functional behavioural behavioural results. Again, like like I was describing with the rat
00:08:38rat in the chamber that that we can
00:08:42can operationalise for AI systems, systems, we have to be careful about about not anthropomorphising these
00:08:46these systems and pretending they're they're just like animals or just just like humans. Yeah. You're
00:08:50not. Just simulation. Some of the the companies. Are some companies companies like Anthropic Anthropic
00:08:53Anthropic are anthropomorphic. Mustafa Mustafa Suleiman did it just the the other day. Mustafa, is that part.
00:08:57part. Of the problem? The training training of doing that? Well? So
00:09:00So I think it's complicated. I don't don't think that there's a clear clear answer either way. The truth
00:09:04truth is we don't really understand understand to what degree it makes makes sense to anthropomorphise these
00:09:08these systems or to compare them them to human and animal brains, brains, and to what degree this is
00:09:12is just unearned. This is precisely precisely why we need to do the science. science. I think Mustafa Suleiman
00:09:15Suleiman is too quick to say this this is all extremely dangerous and and there's no there, there, you
00:09:19you know, nothing to see here. And And I think the folks at Anthropic Anthropic may be too quick to say
00:09:23say no, no, we're sure that, you you know, we're building a system system that has all of these internal
00:09:26internal psychological states. The The truth, as far as I can tell, tell, the scientific consensus to
00:09:29to the degree that I engage with with the researchers at the front
00:09:33front of this field is that we do do not know. We actually do not know.
00:09:36know. We are building a new class class of cognitive system and we we don't talk to that degree. Can
00:09:40Can we talk in that instance then then about this torture chamber? chamber? Because I know you were
00:09:43were opposed to it. You were concerned concerned by what it was testing,
00:09:46testing, but what it actually showed showed is, is that it sort of revealed
00:09:50revealed to everybody and also you
00:09:53you know, on X and on GitHub, they they were talking about it. It actually
00:09:56actually revealed that these machines machines react to pain or what they
00:09:59they perceive to be pain. And in in that sense, Mark was the torture
00:10:03torture chamber useful? I don't know. Whether it was I mean, if you
00:10:06you think about sort of TV shows, shows, if you think about human beings,
00:10:10beings, how they react to celebrities celebrities on television shows and
00:10:13and vote for them to undergo horrible
00:10:17horrible trials and awful things. things. Yeah, there's a streak of
00:10:20of cruelty that goes through that that that, you know, runs through
00:10:23through a lot of people. But when
00:10:24when it comes to torturing an AI,
00:10:28AI, is it inherently cruel to do do that? You know, is it it's at
00:10:31at least it's quite bizarre behaviour behaviour to want to torture. How
00:10:34How do. We know how they actually actually did that? Do we know? I
00:10:38I think it's to your first question question actually, which was about
00:10:41about is it actually experiencing experiencing pain? Because Cameron, Cameron, can you describe and make
00:10:44make it very clear to the audience audience the pain axes and what that
00:10:48that means because there are five five categories of pain. They're They're not all physical. Yeah. Right.
00:10:52Right. And so would you be able to to describe the pain axes? What were
00:10:54were the five categories of pain pain that you had programmed into
00:10:58into your model? Yeah. So so we include
00:11:01include a tonne of ingredients into into this representation. And then
00:11:03then we basically subtract out a a lot of you could think of it as
00:11:07as noise that we don't want to include.
00:11:08include. So among the five are things
00:11:11things like social pain, moral pain, pain, psychological pain, emotional
00:11:15emotional pain, these sorts of things. things. And we also include physical physical pain. So, you know, cutting
00:11:18cutting one's hand with a knife, knife, for example, but also, yeah,
00:11:21yeah, being betrayed by your best
00:11:24best friend and. Can you explain. To us then taking that axis
00:11:28axis what they did with the torture torture chamber and why you were
00:11:30were concerned by it? So basically basically this is how you derive
00:11:33derive the representation. You take take those ideas and then you subtract
00:11:36subtract out all sorts of stuff that that might be sort of related to
00:11:39to this concept. But isn't what we're we're going after like sadness or
00:11:42or generic negative valence or bodily bodily injury. And then you have have this representation that you
00:11:46you can basically turn up and turn turn down in the proverbial brain
00:11:49brain of the system. And what was was going on in our experiments. experiments. We did this to the minimal
00:11:52minimal degree so that we could yield yield the data that we got, and so so that we could show people what
00:11:56what the nature of this representation representation are, what the nature nature of representation is, and
00:11:59and the torture experiments people people were doing this sort of maxing
00:12:02maxing it out for fun all sorts of
00:12:05of entertainment and commodification commodification value to this basically basically doing this in this like
00:12:09like wry, sort of sadistic way. And And to be honest, again, I think
00:12:13think the punchline here is that that we do not know to what degree degree this corresponds to any sort
00:12:17sort of experience the system is is having or not having. Nobody knows knows the answer to this, regardless
00:12:21regardless of how confident people people are on both sides. This is
00:12:23is precisely why we need to do this this research. And so given our uncertainty
00:12:27uncertainty about to what degree degree these distress like states
00:12:30states could be experienced by these these systems, probably better to
00:12:32to on the side of not torturing AIS
00:12:35AIS at a significant scale, especially especially when these systems are
00:12:39are getting more and more embedded embedded in our society day to day day and are going to look back on
00:12:43on these sorts of things and are are going to behave accordingly. accordingly. This company models
00:12:46models are supposed to be. The logical logical extension of this. Cameron
00:12:50Cameron is. Yes, it might press the
00:12:52the release button, which could be
00:12:54be deleting the zip files or deleting deleting pictures of your children.
00:12:58children. But as it becomes more more powerful and more embedded, embedded, it might just decide, you
00:13:01you know what? I don't like Cameron. Cameron. I'm just going to get rid
00:13:03rid of him. I think that this is
00:13:06is a live concern. And I think it it also underscores the importance
00:13:10importance that we need to understand understand what kinds of systems
00:13:13systems we're building and how to to build the functional psychologies
00:13:17psychologies of these systems, regardless regardless of if they're having experiences. experiences. Clearly there are distress
00:13:20distress representations. Clearly Clearly there are positive representations representations in these systems
00:13:23systems and we want to engineer them them in a way so that they don't
00:13:27don't misbehave themselves. And it it seems like making them not, you
00:13:31you know, psychologically extremely extremely damaged and getting them them to behave themselves are actually
00:13:34actually highly correlated. We may may well come to regret it. As you
00:13:37you say. Cameron, who knows. Really Really fascinating stuff. Thank you
00:13:40you so much for being with us this this week. It's been really good
00:13:42good to talk to you. Yeah. Thanks
00:13:44Thanks for having me. Now everything
00:13:50everything we've just talked about about is a machine borrowing our
00:13:54our words pain, suffering language language and experience learned from
00:13:57from humans. The question for the the rest of the programme is what
00:14:01what happens when it stops borrowing borrowing and starts inventing a
00:14:04a language that we don't use? Maybe
00:14:07Maybe don't even understand with with us for the second part of the
00:14:10the programme, Carter Church, a staff staff AI engineer at Sentinel one.
00:14:13one. His work is cyber security card. card. It's good to have you with
00:14:16with us. Yeah, thanks for having having me. Now there are surveys
00:14:19surveys that suggest that more than than a third of AI written code now
00:14:23now goes live without the team fully fully understanding it. What does
00:14:26does that code look like and why why is it becoming so hard to follow?
00:14:29follow? Yeah. So I think there's there's two elements here, right?
00:14:32right? There's volume and there's there's convention or style. So I'm
00:14:36I'm a software engineer. I work with with these models pretty much every every day, right to write code here.
00:14:40here. And as you're looking at their their outputs, they're oftentimes
00:14:44oftentimes extremely verbose. They're They're oftentimes differing in the
00:14:47the style of humans, right? Differing Differing in the ways that we would would write these code, these code
00:14:51code bases. So it's not surprising surprising to have pull requests
00:14:53requests going into code bases that that are 60,000 lines or something
00:14:57something crazy. And again, I think think that makes it a challenge for
00:15:00for people to really to really review review these. Right. And unfortunately,
00:15:04unfortunately, I think a lot of people
00:15:05people are looking at this and just
00:15:09just not doing a great job of that, that, right? Like your number, your,
00:15:11your, your metric that you. Are we we talking agentic AI is it lots
00:15:14lots of different agents talking talking to one another? And as we
00:15:18we talk to one another, abbreviating abbreviating maybe the language.
00:15:21language. I think it could be I don't don't really see that as much of
00:15:24of the issue here. I think it's just
00:15:27just that training these models on,
00:15:29on, on tasks is easy when the task task is verifiable. But something
00:15:32something like style or convention convention is not so verifiable, verifiable, right? You can verify
00:15:36verify that a thing works, but you you can't really verify in training
00:15:39training that a style is what every every person would like because everyone everyone has different styles they
00:15:42they prefer. So if a human can't
00:15:46can't follow the code, could an attacker
00:15:50attacker hide within it? So could
00:15:51could you mask something and hide
00:15:54hide it inside the code? Yeah, I I think I mean, technically the answer
00:15:57answer is yes, but in reality I mean mean back doors and things like this
00:16:00this have always been a problem in in software, right? I don't know know that AI really moves the needle
00:16:04needle on that too much. And I think think we also have ways to defend
00:16:07defend against this in the industry. industry. Right. And the answer is
00:16:10is code review. Look I think it all all comes down to you can't automate
00:16:14automate accountability. These things things are able of doing very interesting interesting work. But ultimately
00:16:17ultimately we're the ones that have have to live with that work. And And it's up to us to decide when
00:16:21when we hit the button to merge that that into a code base. And so in
00:16:24in terms of sort of the analysis, analysis, right, and the verification,
00:16:27verification, you need to be an expert expert developer in order to do that.
00:16:30that. What I'm really concerned about about is that how are we going to
00:16:33to have junior coders who then become become those experts that do the
00:16:37the verification that understand understand alignment if they're not not coding in the first place? Yeah.
00:16:41Yeah. I think the good news is that that models can definitely help with
00:16:44with coaching and understanding what's what's here, right? There is an opportunity opportunity to look at the changes
00:16:48changes and as we I think, go down down go down the rabbit hole of like
00:16:51like these things, you know, building building stylistically, oddly, we
00:16:54we can still use these agents to to explain exactly what they're doing doing here, right? We could very
00:16:58very easily spin up an agent. We We could ask or interrogate any question
00:17:02question we want of that to understand understand why decisions were made. made. And I've found, at least in
00:17:05in my work that this is this has has levelled me up pretty significantly.
00:17:09significantly. But you're talking talking about singles. Increasingly,
00:17:12Increasingly, AI agents are working
00:17:14working in swarms, OpenAI told us us recently that 10,000 had solved
00:17:17solved a famous math problem in a a matter of days, when hundreds of
00:17:21of thousands of agents are talking talking to each other at that sort
00:17:24sort of speed in a shorthand that
00:17:26that they've invented. Is anyone anyone really reading any of that?
00:17:30that? I think the short answer is is no. I don't think it's possible
00:17:34possible to read all of that. Right. Right. If you're thinking about we we were talking before of one agent
00:17:37agent and trying to validate what's what's actually coming out of that. that. And it's very paced, you know,
00:17:41know, to the speed of the human, human, which is I want some code. code. I ask for some things, it does
00:17:45does some work and I validate that. that. Right. Well, therein is the
00:17:49the problem because if multi-agent multi-agent AI from different companies companies meets each other on the
00:17:53the open internet with no human in in the loop, what stops those agents
00:17:57agents adopting each other's goals, goals, each other's language rather
00:18:00rather than our goals? So I think think we have to remember that humans humans are the ones architecting
00:18:04architecting these systems. And if if they're not doing work for us, us, then why are we building them
00:18:08them in the first place? And so my my answer would be we can architect
00:18:12architect these in ways where we we are protecting against all of
00:18:14of those things. It's a totally hard hard choice to make. And so as we're we're looking at these things, we
00:18:18we can add in checks and balances, balances, right? We can add humans humans in the loop at certain steps.
00:18:22steps. But ultimately that decision decision is up to us. Well, we can,
00:18:25can, but therein is a problem for for the industry, surely, because because at what point does a nuclear
00:18:28nuclear power company say, I'm sorry, sorry, you don't fully understand
00:18:32understand this code? Well, then then we can't trust their behaviour,
00:18:34behaviour, so we can't put it into into our systems. I think these critical
00:18:38critical industries have adopted adopted this long ago. Right. This This is there's a reason they run
00:18:41run on the specific technology that that they do for this exact reason. reason. And so I don't really think
00:18:45think that AI in this case moves moves the needle too much. I do think,
00:18:48think, though, that more and more more organisations are going to adopt adopt that kind of mentality where
00:18:52where they're only going to run the the things that either they themselves themselves verify or that a trusted
00:18:56trusted partner does. Is there a a risk though? So sorry, is there
00:18:58there a risk there though in creating
00:19:00creating in laziness in human beings beings simply because you've said
00:19:03said the sheer volume of code, the the sheer volume of communications communications between agents, that
00:19:07that people will just go, OK, I'm I'm not going to bother trying to to interrogate this because there's
00:19:11there's tonnes and tonnes and tonnes tonnes of this stuff to try and get get through. Yeah, I think that's
00:19:14that's maybe one of the biggest risks risks here, right? Is that for those those who actually care about the
00:19:18the things that they are building building and who care about the quality quality of what they're building,
00:19:22building, these technologies can can be incredibly amazing and unlock unlock so many opportunities. But
00:19:26But I would say that if an organisation organisation has folks who are blindly
00:19:29blindly hitting enter and not really really caring about the work that that they produce when they're supposed
00:19:32supposed to be the one accountable accountable for that, they have bigger bigger problems than AI. Yeah, but
00:19:36but there's. Been some research, research, right? I think Anthropic Anthropic released this in a research
00:19:39research paper of theirs that actually actually an AI agent might be able
00:19:42able to tell if another AI agent agent is watching it. So if we're
00:19:45we're using agents on the defence
00:19:48defence for AI offence, it could could essentially hoodwink you. And
00:19:52And so I'm really concerned about about this idea of swarms. AI is
00:19:56is in networks with their own language, language, you know, covering up what
00:19:59what they're actually trying to do do because you talked about the human human being involved in the design
00:20:03design and the architecture. I think think that's always been the goal. goal. But the fact that they can
00:20:06can fool you means that I think what what we're talking about here, is
00:20:09is there really a defence to this? this? Look, I think I really do think
00:20:13think there is. I think that again, again, we are the ones who choose choose how we architect these systems.
00:20:17systems. And I think to the reference reference that the example that you're
00:20:20you're referencing collecting telemetry telemetry and being able to have have observability into these is
00:20:24is different than actively auditing, auditing, right? So you can think think of this as if we have every
00:20:27every log for what the agent is doing doing at every step for reasoning, reasoning, we can consistently use
00:20:31use models to look at that in a separate separate and different place to understand understand if we're on the right
00:20:35right track. And I think it's up up to us to decide what that pace
00:20:38pace is, right, and how exactly we we implement those measures. Well,
00:20:41Well, whether or not the machine machine is keeping secrets, the AI
00:20:44AI models do seem to be rather good
00:20:47good at reading ours. In 1809, on
00:20:51on Napoleon's instructions, a coded coded letter was sent to General
00:20:54General Auguste de Marmont, one of of the most senior commanders. As
00:20:57As France prepare for war with Austria,
00:21:00Austria, and there were 1300 characters characters in this letter. Nobody Nobody has ever read it because the
00:21:04the key to the cypher was lost. Generations
00:21:07Generations of historians had tried.
00:21:08tried. They managed to decipher just
00:21:12just 33 out of about 155 symbols. symbols. But last week it was cracked.
00:21:16cracked. Not in a university library, library, but from a single scanned
00:21:19scanned photograph by an AI in six
00:21:21six hours. And the man who directed
00:21:24directed the AI to do it is sitting
00:21:27sitting with us. Carter, how does
00:21:30does a machine read a code when the
00:21:33the key is lost? Yeah, it's a really really good question. And the short
00:21:36short answer is exactly the same same way that a person would. If If you look at what Astra was the
00:21:40the model that I used here, if you you look at what Astra did, it took
00:21:44took this cypher plate image, the the one image that we had of this
00:21:48this from the 1960s, it was able able to transcribe 1300 handwritten
00:21:51handwritten symbols where computer computer vision has largely been
00:21:53been an issue for computers in general,
00:21:56general, it was able to write its its own solver and its own verifier
00:21:58verifier and even do research online online to find a partial key that
00:22:02that was published under an incorrect incorrect name. Wow. Well, well, well, you've got to tell us what
00:22:05what the message says. What did we
00:22:07we learn about Napoleon's intentions? Well, so essentially
00:22:11essentially the letter was a word word briefing. We learned some general general things about troop positions
00:22:14positions and all of this. But history history already confirmed all of of that, right. So we learned some
00:22:18some interesting bits of verbiage verbiage and in certain ways that that things were worded that were
00:22:21were kind of interesting. You know, know, we learn, I think Napoleon
00:22:24Napoleon instructed this to be said, said, you know, Austria is hastening hastening towards her ruin. And also
00:22:28also a pretty interesting note to to Marmont that basically said, you
00:22:31you know, don't let yourself be convinced convinced out of this attack and
00:22:34and things like this. But more generally, generally, I think that a lot of
00:22:37of the aspects of this letter were were kind of already known. And that's
00:22:40that's one of the reasons, in addition addition to the friction that this
00:22:42this kind of work can this kind of of work requires that it remained
00:22:46remained uncracked for so. Long from from looking at the methodology, methodology, did it just help. Pull
00:22:50Pull all of that evidence together together from the other website that
00:22:52that it found the 33 symbols that
00:22:54that it already knew or from the
00:22:56the process? Did you see that it it actually changed something in
00:22:59in the code breaking methodology? methodology? Well, it definitely definitely didn't change anything
00:23:03anything in the code breaking methodology. methodology. So everything that was
00:23:06was done here is something we already already knew how to do. The large
00:23:09large friction here is actually in in looking at all of those symbols symbols and transcribing them over.
00:23:12over. Right. And so one of the powers powers of these technologies is just
00:23:16just how multimodal they're becoming, becoming, that you can start with with something as generic as an image
00:23:20image and a general prompt to break break something. Right? And to say,
00:23:23say, I can transcribe these for you. you. I know how to solve this because because you know this. It's known
00:23:26known how to solve generally cyphers cyphers of this time I can apply apply those same learnings and then
00:23:30then by the time you come back, I I can already. What does this mean mean for all of the encryption with
00:23:34with our bank accounts and security security systems? I think that's
00:23:38that's a totally different beast,
00:23:39beast, right? OK. Thankfully, postscript
00:23:43postscript there's something that that we knew how to crack. Oh yeah. yeah. There's something that we knew
00:23:47knew how to crack. This is not a a surprise there. I think the biggest biggest surprise though is in these.
00:23:51these. Long horizon tasks. I think think it turns out the work that that we do as people is extremely
00:23:55extremely multimodal. And so having having these models that can do things
00:23:58things like vision but also be left left for long periods of time to
00:24:01to discover is really powerful. There There are hundreds of enciphered
00:24:04enciphered scripts. I mean, the Voynich Voynich manuscript from the 15th
00:24:0815th century, a book of texts there there that no one's ever read. I
00:24:10I mean, could are we talking about about AI cracking all the texts that
00:24:14that we could never read? I think think Voynich is a totally different
00:24:18different beast. This one in particular particular is a much easier, easier
00:24:21easier cypher than some things like like Voynich. But I think more generally
00:24:24generally that people would be shocked shocked by the amount of these that that are still out there that we
00:24:28we already know the methodology to to crack. Look, discovery is a process
00:24:32process of turning over stones. And And today we have to decide which
00:24:35which stones we want to actually actually turn over and make the best best educated guess that we can.
00:24:38can. And I think that things like like this, whether it be in crypto,
00:24:41crypto, in this area or in other other areas, just means that we can
00:24:44can turn over more stones. And if if you look at what science is, it
00:24:47it really is a matter of turning turning over things, figuring things
00:24:50things out, failing at them, and and then eventuating in a success. success. Do you know what it does?
00:24:54does? Do you know what occurs to. Me bringing a whole programme
00:24:57programme full loop here is that that if we don't fully understand
00:24:59understand what is going on in the the AI brain, why would it not use
00:25:03use these codes that it knows we
00:25:05we cannot crack to converse with
00:25:07with other AIS? Well, I think we we absolutely can crack them. Right.
00:25:11Right. And so again, that comes back back to the traceability and observability observability piece. Let's say that
00:25:14that a model was capable of coming coming up with something it thought thought that we couldn't crack. If
00:25:18If we have the right traces, we understand understand exactly how it arrived arrived at that point in the first
00:25:22first place. Right. And you would would be able to use an AI to find
00:25:25find that cypher anyway. Now. That's That's right. OK. Send a thief to
00:25:28to catch a thief. Yes, yes. Quite
00:25:32Quite extraordinary stuff. Really Really interesting. Carter, to talk talk to you this week. Thank you
00:25:35you very much. Congratulations. I'm I'm looking forward to more manuscripts
00:25:39manuscripts and more texts. Do come come back on the programme to tell tell us if you have any thoughts
00:25:42thoughts on that. We want to hear hear from you. Email us at AI Decoded
00:25:46Decoded at bbc.co.uk. If you're watching watching on television, the QR code
00:25:50code is on screen and that will take take you to the AI Decoded playlist.
00:25:53playlist. Lots of good stuff there. there. You'll find every episode episode and every previous one that
00:25:56that we've done there on iPlayer iPlayer two. That's all we have time
00:26:00time for this week. Mark Cameron Cameron Carter and Priya, thank you
00:26:04you very much for everything that that you've given us this week and and thank you for watching. We'll
00:26:08We'll see you next time.
00:28:21will it all mean for Europe's economy? economy? That's this week on Talking
00:28:25Talking Business here on BBC News.
00:28:33MUSIC
00:28:33What new industries are created?
00:28:38created? Because what we've seen seen throughout history is new industries industries are created. And it's
00:28:42it's that process that actually opens
00:28:44opens new doors. That was the that that was the mandate from Prime Minister
00:28:48Minister Starmer. He wants the UK
00:28:49UK to be an AI superpower. He recognises
00:28:52recognises that the ingredients are are all right here. Putting the customer
00:28:56customer at the centre of everything everything we do. I really think
00:28:59think that's how the company started. started. But I think we lost our
00:29:02our focus because we got a little
00:29:04little too distracted on efficiency
00:29:07efficiency and technology. AI is
00:29:11is real AI and total pay off just just like cars in total paid off
00:29:14off TV and total paid off. But most most people involved that didn't
00:29:17didn't do well. It's a revolutionary, revolutionary, you know, time upon
00:29:20upon us. And I think, you know, all all industries will be adopting AI
00:29:24AI in some shape or form.
00:30:13Live from Washington. This is
00:30:14BBC News. A Saudi led coalition says
00:30:18says it will respond firmly to a a deadly attack by Houthis on an
00:30:22an international airport in Riyadh.
00:30:24Riyadh. President Trump says Ukraine Ukraine needs a new president in
00:30:28in a scathing attack on Volodymyr Volodymyr Zelensky. Zelensky could
00:30:32could have settled this war many
00:30:34many times, many, many times. And
00:30:36And he chose he chooses not to. And
00:30:40And could this object solve one of of history's greatest mysteries?
00:30:44mysteries? We'll hear from the team team making the journey across the
00:30:47the Pacific to see if this could
00:30:49could be Amelia Earhart's lost plane.