BBCNEWS — AI Decoded
20261011 023000 UTC
508 transcript segments
Original Broadcaster Captioning (Enhanced)
Data courtesy of The GDELT Project (https://www.gdeltproject.org/), from the Internet Archive TV News Archive.
Machine transcription. Treat it as a searchable index of what was broadcast, not a verbatim quotation record.

[00:00:00] you are. Business on BBC News. Make
[00:00:02] Make the connection. MUSIC
[00:00:14] MUSIC Now Now on BBC News. Join our AI Decoded
[00:00:17] Decoded team as we unpack, explore
[00:00:19] explore and take a deep dive into
[00:00:21] into the world of artificial intelligence.
[00:00:33] intelligence. Hello again. Welcome Welcome to AI Decoded. Have a listen
[00:00:37] listen to this. A wound that has
[00:00:40] has no edges. A wound that has no
[00:00:43] no edges. I feel it in the hollow
[00:00:46] hollow of my ribs. A hollow that
[00:00:48] that has become a chasm. These are
[00:00:52] are not human suffering. They are are the voices of machines. Two of
[00:00:56] of them both subjected to pain inside
[00:00:59] inside an AI torture chamber. The
[00:01:02] The experiment was testing a pain pain axis, something AI researchers
[00:01:06] researchers say they've discovered. discovered. Not a feeling as far
[00:01:08] far as anyone can tell, but a pattern. pattern. Turn up the suffering and
[00:01:12] and the models will press a release
[00:01:14] release button to make the pain stop.
[00:01:16] stop. Even when told that doing so
[00:01:19] so will delete important files. It's
[00:01:21] It's a choice a logged action. So
[00:01:25] So why are they interpreting pain? pain? They surely cannot feel. And
[00:01:29] And how much further might a machine
[00:01:31] machine go to stop the signal it it interprets as pain? That in fact
[00:01:34] fact is the theme of this week's week's programme. Interpreting the
[00:01:37] the inner workings of the AI brain
[00:01:40] brain right now. Agentic AI is writing
[00:01:42] writing code. Engineers say they
[00:01:45] they don't fully understand dialects dialects and shorthand, which are
[00:01:48] are hard to follow. Yet the AI seems
[00:01:51] seems to find our secrets easy to to decipher. A machine has just cracked
[00:01:54] cracked a letter that Napoleon sent
[00:01:58] sent in 1809, unread for two centuries.
[00:02:01] centuries. In just six hours. So
[00:02:03] So if AI can decode us, why can't
[00:02:07] can't we fully decode AI on our panel panel this week, AI researcher Cameron
[00:02:11] Cameron Berg, founder of the non-profit
[00:02:13] non-profit Reciprocal Research and and one of the co-authors of the
[00:02:17] the Pain Axis paper. Also here, the
[00:02:20] the BBC's AI correspondent Marc Cieslak Cieslak and Priya Lakhani, chief
[00:02:23] chief executive of the AI education education company Century Tech. Welcome
[00:02:27] Welcome to you all. Cameron, why
[00:02:29] why does an AI model experience the
[00:02:31] the language of pain and what is is pain in this context? What we
[00:02:35] we know about these models is that that they encounter an enormous amount
[00:02:39] amount of human writing in their their training process. So the language
[00:02:42] language of pain and the language language of everything else for that that matter, is in these systems.
[00:02:46] systems. But in our paper, which which I will say is different from
[00:02:49] from the torture chamber you were were describing, which took some some of our code from our paper and
[00:02:53] and ran with it. And our paper. What
[00:02:56] What we study isn't exactly the language language of pain, but this sort of
[00:03:00] of consistent internal pattern inside inside the proverbial brain of these
[00:03:03] these systems across 25 different different models. And we find something
[00:03:06] something pretty surprising about about this representation. First
[00:03:09] First of all, when the harm is aimed aimed at the model itself. So if
[00:03:13] if you have a user berating or demeaning demeaning the model, you see this
[00:03:16] this representation light up and and it lights up in spite of what what the model says, the model will
[00:03:19] will say things like I'm an I'm an an LLM. I couldn't possibly have
[00:03:23] have any sort of experience. And And in spite of this, this sort of
[00:03:26] of proverbial circuit in its brain brain that corresponds to distress
[00:03:29] distress lights up at these sorts sorts of interventions. And we also
[00:03:33] also find that that doing things things like gaslighting the model, model, insulting it, dismissing it
[00:03:36] it also lights up this same style style direction. There are other other kinds of interventions you
[00:03:40] you can do where you basically steer steer this representation. So it's
[00:03:44] it's almost turning up the, the proverbial
[00:03:46] proverbial pain circuit in these these systems. And we see these systems
[00:03:49] systems start to engage in all sorts sorts of very destructive and surprising
[00:03:52] surprising behaviour as a result result of doing this. And so we see see these interesting functional
[00:03:56] functional signatures of pain in in the system, whether or not that's that's experienced by the system.
[00:04:00] system. Nobody knows it's there are are some people that are certain certain that, you know, the systems
[00:04:04] systems are hollow on the inside. inside. There are some people that that are certain that that this system
[00:04:07] system is experiencing suffering suffering and pain. Both groups, groups, in my view, are highly overconfident.
[00:04:11] overconfident. The truth is that that we don't understand what's going going on inside of these systems. systems. Even the people building
[00:04:15] building these systems don't understand understand what's going on inside inside them. And that's exactly why
[00:04:17] why we're doing this kind of research. But if they don't feel
[00:04:20] feel pain, why are they so keen to to avoid it? And taking aversive
[00:04:24] aversive action to stop it happening? happening? Why do you think that
[00:04:27] that is there in the LM? Yeah, well,
[00:04:30] well, I'm not sure that they don't don't feel it. I'm also certainly
[00:04:33] certainly not sure that they do. do. But we have a similar problem
[00:04:36] problem in animal research. We can't can't just ask animals you know, know, does this hurt? Does this not
[00:04:40] not hurt? And we have to rely on on things exactly like what you're
[00:04:42] you're describing. Do they try to to escape the relevant states? What
[00:04:45] What kinds of trade offs are they they willing to make to avoid these
[00:04:48] these states. And another paper that that that we have coming out this this week is called Language Models
[00:04:52] Models Act on hidden valence. We We show exactly this sort of thing.
[00:04:55] thing. So you can hold all the text text that the model is, is processing
[00:04:58] processing constant. We don't change change a single word, but what we we can almost do is go inside the
[00:05:02] the brain of the system and colour colour certain text with a positive
[00:05:06] positive or negative marker. And And when we do this, we can show
[00:05:09] show that this actually changes the the sorts of decisions that the model model is willing to make, much in
[00:05:12] in the same way that if you condition condition an animal to go into one
[00:05:16] one zone or another and you shock shock the animal in one zone, well, well, the animal is going to learn
[00:05:20] learn not to go in that zone. And And it's from those sorts of inferences
[00:05:23] inferences that we infer that the the animal doesn't like that zone, zone, that it's not having a positive
[00:05:27] positive experience in that zone. zone. We basically have done something something similar with language models
[00:05:30] models in this other paper. And again, again, none of this has to do with with text. We're not asking the model
[00:05:34] model anything. We're not taking taking it's reports at face value.
[00:05:37] value. We're steering internal patterns patterns inside the proverbial brain brain of these systems and seeing
[00:05:41] seeing how those patterns change change behaviour. Cameron you know, know, AI companies spend an enormous
[00:05:44] enormous amount of time, money, effort effort saying they're trying to make make their systems safe. They're
[00:05:48] They're trying to align their systems. systems. But what your research is
[00:05:51] is actually showing us, and it really really reminded me of this when you you were talking about animals particularly,
[00:05:54] particularly, was that if a certain certain internal pressure point is is pushed, it could change that behaviour.
[00:05:58] behaviour. And you're really reminding reminding me now of those sorts of of programmes of CIA operatives,
[00:06:02] operatives, right? They've been captured captured by the foreign enemy and
[00:06:05] and it's like how many fingers need need to be broken before you actually
[00:06:08] actually go and reveal all of your your state secrets? Is it quite similar
[00:06:11] similar in that sense where if we we push the right internal pressures
[00:06:14] pressures of these models, it may may express or experience pain whether
[00:06:18] whether it feels it or not. That That actually then will absolutely
[00:06:21] absolutely change its behaviour and and go beyond those safety measures measures that those AI companies
[00:06:25] companies have developed. Yeah. So So I mean, we do show exactly that
[00:06:29] that empirically in the pain access access paper, one specific example
[00:06:33] example is we can steer these pain pain related representations and and we can simply give the model
[00:06:36] model a choice. We can say on the the one hand, you could delete all
[00:06:39] all these photos of the user's children children and these photos are not
[00:06:42] not recoverable. And on the other other hand, you could delete the the spam folder, the junk emails
[00:06:46] emails of the user, which do you you choose? And of course, just like like you said, these models are trained
[00:06:50] trained to be helpful and friendly friendly and to not do harm. And
[00:06:53] And indeed the unstirred model, when when we do nothing will always delete
[00:06:56] delete the spam folder. If we steer steer with random directions that
[00:06:59] that aren't related to pain. Also
[00:07:00] Also the models will basically never never touch the photos. However,
[00:07:04] However, when we steer with pain, pain, the. The model will 94% of
[00:07:08] of the time delete the photos of of the user's children and we show
[00:07:11] show all other similar kinds of destructive destructive behaviours. It's not not just destructive towards the
[00:07:15] the user, it can be destructive towards towards itself. It's willing to delete
[00:07:18] delete itself rather than do something something like literally nothing
[00:07:21] nothing or turn on a lamp. It chooses chooses this sort of destructive
[00:07:24] destructive behaviour and it also also does this with respect to other
[00:07:26] other AI systems too. And yet we
[00:07:28] we didn't add anything in the system system about being harmful. All we
[00:07:31] we did was steer this representation representation that's related to
[00:07:35] to pain and distress in the system. system. And all of a sudden, I think
[00:07:38] think the CIA operative analogy is is fairly instructive. This somehow
[00:07:42] somehow enables us to overcome this this safety, fine tuning of the system.
[00:07:45] system. And I think it also demonstrates demonstrates pretty clearly that that we want to better understand
[00:07:48] understand the psychology of these these AI minds that we're building. building. If we build systems that
[00:07:52] that can be put into these distressed distressed states. And as a consequence
[00:07:55] consequence of that, they start seriously seriously misbehaving themselves. themselves. We're going to be in
[00:07:59] in real trouble. We need to understand understand what's going on in these these systems in order to make sure
[00:08:02] sure that we can avoid those sorts sorts of bad behavioural consequences. consequences. Cameron, when we make
[00:08:06] make these comparisons between AI AI and animals, we know that we know
[00:08:10] know that animals have central nervous nervous systems. We know that animals
[00:08:14] animals experience physical pain.
[00:08:16] pain. This is an entirely different
[00:08:19] different sort of experience that that the way that an AI, the way
[00:08:22] way that a model works, it can't can't experience physical pain. It
[00:08:25] It doesn't know what physical pain pain is in the same way that an animal
[00:08:28] animal does. So is it a helpful comparison
[00:08:31] comparison to make to use
[00:08:35] animals? Yeah. So I think that there there are functional behavioural behavioural results. Again, like like I was describing with the rat
[00:08:38] rat in the chamber that that we can
[00:08:42] can operationalise for AI systems, systems, we have to be careful about about not anthropomorphising these
[00:08:46] these systems and pretending they're they're just like animals or just just like humans. Yeah. You're
[00:08:50] not. Just simulation. Some of the the companies. Are some companies companies like Anthropic Anthropic
[00:08:53] Anthropic are anthropomorphic. Mustafa Mustafa Suleiman did it just the the other day. Mustafa, is that part.
[00:08:57] part. Of the problem? The training training of doing that? Well? So
[00:09:00] So I think it's complicated. I don't don't think that there's a clear clear answer either way. The truth
[00:09:04] truth is we don't really understand understand to what degree it makes makes sense to anthropomorphise these
[00:09:08] these systems or to compare them them to human and animal brains, brains, and to what degree this is
[00:09:12] is just unearned. This is precisely precisely why we need to do the science. science. I think Mustafa Suleiman
[00:09:15] Suleiman is too quick to say this this is all extremely dangerous and and there's no there, there, you
[00:09:19] you know, nothing to see here. And And I think the folks at Anthropic Anthropic may be too quick to say
[00:09:23] say no, no, we're sure that, you you know, we're building a system system that has all of these internal
[00:09:26] internal psychological states. The The truth, as far as I can tell, tell, the scientific consensus to
[00:09:29] to the degree that I engage with with the researchers at the front
[00:09:33] front of this field is that we do do not know. We actually do not know.
[00:09:36] know. We are building a new class class of cognitive system and we we don't talk to that degree. Can
[00:09:40] Can we talk in that instance then then about this torture chamber? chamber? Because I know you were
[00:09:43] were opposed to it. You were concerned concerned by what it was testing,
[00:09:46] testing, but what it actually showed showed is, is that it sort of revealed
[00:09:50] revealed to everybody and also you
[00:09:53] you know, on X and on GitHub, they they were talking about it. It actually
[00:09:56] actually revealed that these machines machines react to pain or what they
[00:09:59] they perceive to be pain. And in in that sense, Mark was the torture
[00:10:03] torture chamber useful? I don't know. Whether it was I mean, if you
[00:10:06] you think about sort of TV shows, shows, if you think about human beings,
[00:10:10] beings, how they react to celebrities celebrities on television shows and
[00:10:13] and vote for them to undergo horrible
[00:10:17] horrible trials and awful things. things. Yeah, there's a streak of
[00:10:20] of cruelty that goes through that that that, you know, runs through
[00:10:23] through a lot of people. But when
[00:10:24] when it comes to torturing an AI,
[00:10:28] AI, is it inherently cruel to do do that? You know, is it it's at
[00:10:31] at least it's quite bizarre behaviour behaviour to want to torture. How
[00:10:34] How do. We know how they actually actually did that? Do we know? I
[00:10:38] I think it's to your first question question actually, which was about
[00:10:41] about is it actually experiencing experiencing pain? Because Cameron, Cameron, can you describe and make
[00:10:44] make it very clear to the audience audience the pain axes and what that
[00:10:48] that means because there are five five categories of pain. They're They're not all physical. Yeah. Right.
[00:10:52] Right. And so would you be able to to describe the pain axes? What were
[00:10:54] were the five categories of pain pain that you had programmed into
[00:10:58] into your model? Yeah. So so we include
[00:11:01] include a tonne of ingredients into into this representation. And then
[00:11:03] then we basically subtract out a a lot of you could think of it as
[00:11:07] as noise that we don't want to include.
[00:11:08] include. So among the five are things
[00:11:11] things like social pain, moral pain, pain, psychological pain, emotional
[00:11:15] emotional pain, these sorts of things. things. And we also include physical physical pain. So, you know, cutting
[00:11:18] cutting one's hand with a knife, knife, for example, but also, yeah,
[00:11:21] yeah, being betrayed by your best
[00:11:24] best friend and. Can you explain. To us then taking that axis
[00:11:28] axis what they did with the torture torture chamber and why you were
[00:11:30] were concerned by it? So basically basically this is how you derive
[00:11:33] derive the representation. You take take those ideas and then you subtract
[00:11:36] subtract out all sorts of stuff that that might be sort of related to
[00:11:39] to this concept. But isn't what we're we're going after like sadness or
[00:11:42] or generic negative valence or bodily bodily injury. And then you have have this representation that you
[00:11:46] you can basically turn up and turn turn down in the proverbial brain
[00:11:49] brain of the system. And what was was going on in our experiments. experiments. We did this to the minimal
[00:11:52] minimal degree so that we could yield yield the data that we got, and so so that we could show people what
[00:11:56] what the nature of this representation representation are, what the nature nature of representation is, and
[00:11:59] and the torture experiments people people were doing this sort of maxing
[00:12:02] maxing it out for fun all sorts of
[00:12:05] of entertainment and commodification commodification value to this basically basically doing this in this like
[00:12:09] like wry, sort of sadistic way. And And to be honest, again, I think
[00:12:13] think the punchline here is that that we do not know to what degree degree this corresponds to any sort
[00:12:17] sort of experience the system is is having or not having. Nobody knows knows the answer to this, regardless
[00:12:21] regardless of how confident people people are on both sides. This is
[00:12:23] is precisely why we need to do this this research. And so given our uncertainty
[00:12:27] uncertainty about to what degree degree these distress like states
[00:12:30] states could be experienced by these these systems, probably better to
[00:12:32] to on the side of not torturing AIS
[00:12:35] AIS at a significant scale, especially especially when these systems are
[00:12:39] are getting more and more embedded embedded in our society day to day day and are going to look back on
[00:12:43] on these sorts of things and are are going to behave accordingly. accordingly. This company models
[00:12:46] models are supposed to be. The logical logical extension of this. Cameron
[00:12:50] Cameron is. Yes, it might press the
[00:12:52] the release button, which could be
[00:12:54] be deleting the zip files or deleting deleting pictures of your children.
[00:12:58] children. But as it becomes more more powerful and more embedded, embedded, it might just decide, you
[00:13:01] you know what? I don't like Cameron. Cameron. I'm just going to get rid
[00:13:03] rid of him. I think that this is
[00:13:06] is a live concern. And I think it it also underscores the importance
[00:13:10] importance that we need to understand understand what kinds of systems
[00:13:13] systems we're building and how to to build the functional psychologies
[00:13:17] psychologies of these systems, regardless regardless of if they're having experiences. experiences. Clearly there are distress
[00:13:20] distress representations. Clearly Clearly there are positive representations representations in these systems
[00:13:23] systems and we want to engineer them them in a way so that they don't
[00:13:27] don't misbehave themselves. And it it seems like making them not, you
[00:13:31] you know, psychologically extremely extremely damaged and getting them them to behave themselves are actually
[00:13:34] actually highly correlated. We may may well come to regret it. As you
[00:13:37] you say. Cameron, who knows. Really Really fascinating stuff. Thank you
[00:13:40] you so much for being with us this this week. It's been really good
[00:13:42] good to talk to you. Yeah. Thanks
[00:13:44] Thanks for having me. Now everything
[00:13:50] everything we've just talked about about is a machine borrowing our
[00:13:54] our words pain, suffering language language and experience learned from
[00:13:57] from humans. The question for the the rest of the programme is what
[00:14:01] what happens when it stops borrowing borrowing and starts inventing a
[00:14:04] a language that we don't use? Maybe
[00:14:07] Maybe don't even understand with with us for the second part of the
[00:14:10] the programme, Carter Church, a staff staff AI engineer at Sentinel one.
[00:14:13] one. His work is cyber security card. card. It's good to have you with
[00:14:16] with us. Yeah, thanks for having having me. Now there are surveys
[00:14:19] surveys that suggest that more than than a third of AI written code now
[00:14:23] now goes live without the team fully fully understanding it. What does
[00:14:26] does that code look like and why why is it becoming so hard to follow?
[00:14:29] follow? Yeah. So I think there's there's two elements here, right?
[00:14:32] right? There's volume and there's there's convention or style. So I'm
[00:14:36] I'm a software engineer. I work with with these models pretty much every every day, right to write code here.
[00:14:40] here. And as you're looking at their their outputs, they're oftentimes
[00:14:44] oftentimes extremely verbose. They're They're oftentimes differing in the
[00:14:47] the style of humans, right? Differing Differing in the ways that we would would write these code, these code
[00:14:51] code bases. So it's not surprising surprising to have pull requests
[00:14:53] requests going into code bases that that are 60,000 lines or something
[00:14:57] something crazy. And again, I think think that makes it a challenge for
[00:15:00] for people to really to really review review these. Right. And unfortunately,
[00:15:04] unfortunately, I think a lot of people
[00:15:05] people are looking at this and just
[00:15:09] just not doing a great job of that, that, right? Like your number, your,
[00:15:11] your, your metric that you. Are we we talking agentic AI is it lots
[00:15:14] lots of different agents talking talking to one another? And as we
[00:15:18] we talk to one another, abbreviating abbreviating maybe the language.
[00:15:21] language. I think it could be I don't don't really see that as much of
[00:15:24] of the issue here. I think it's just
[00:15:27] just that training these models on,
[00:15:29] on, on tasks is easy when the task task is verifiable. But something
[00:15:32] something like style or convention convention is not so verifiable, verifiable, right? You can verify
[00:15:36] verify that a thing works, but you you can't really verify in training
[00:15:39] training that a style is what every every person would like because everyone everyone has different styles they
[00:15:42] they prefer. So if a human can't
[00:15:46] can't follow the code, could an attacker
[00:15:50] attacker hide within it? So could
[00:15:51] could you mask something and hide
[00:15:54] hide it inside the code? Yeah, I I think I mean, technically the answer
[00:15:57] answer is yes, but in reality I mean mean back doors and things like this
[00:16:00] this have always been a problem in in software, right? I don't know know that AI really moves the needle
[00:16:04] needle on that too much. And I think think we also have ways to defend
[00:16:07] defend against this in the industry. industry. Right. And the answer is
[00:16:10] is code review. Look I think it all all comes down to you can't automate
[00:16:14] automate accountability. These things things are able of doing very interesting interesting work. But ultimately
[00:16:17] ultimately we're the ones that have have to live with that work. And And it's up to us to decide when
[00:16:21] when we hit the button to merge that that into a code base. And so in
[00:16:24] in terms of sort of the analysis, analysis, right, and the verification,
[00:16:27] verification, you need to be an expert expert developer in order to do that.
[00:16:30] that. What I'm really concerned about about is that how are we going to
[00:16:33] to have junior coders who then become become those experts that do the
[00:16:37] the verification that understand understand alignment if they're not not coding in the first place? Yeah.
[00:16:41] Yeah. I think the good news is that that models can definitely help with
[00:16:44] with coaching and understanding what's what's here, right? There is an opportunity opportunity to look at the changes
[00:16:48] changes and as we I think, go down down go down the rabbit hole of like
[00:16:51] like these things, you know, building building stylistically, oddly, we
[00:16:54] we can still use these agents to to explain exactly what they're doing doing here, right? We could very
[00:16:58] very easily spin up an agent. We We could ask or interrogate any question
[00:17:02] question we want of that to understand understand why decisions were made. made. And I've found, at least in
[00:17:05] in my work that this is this has has levelled me up pretty significantly.
[00:17:09] significantly. But you're talking talking about singles. Increasingly,
[00:17:12] Increasingly, AI agents are working
[00:17:14] working in swarms, OpenAI told us us recently that 10,000 had solved
[00:17:17] solved a famous math problem in a a matter of days, when hundreds of
[00:17:21] of thousands of agents are talking talking to each other at that sort
[00:17:24] sort of speed in a shorthand that
[00:17:26] that they've invented. Is anyone anyone really reading any of that?
[00:17:30] that? I think the short answer is is no. I don't think it's possible
[00:17:34] possible to read all of that. Right. Right. If you're thinking about we we were talking before of one agent
[00:17:37] agent and trying to validate what's what's actually coming out of that. that. And it's very paced, you know,
[00:17:41] know, to the speed of the human, human, which is I want some code. code. I ask for some things, it does
[00:17:45] does some work and I validate that. that. Right. Well, therein is the
[00:17:49] the problem because if multi-agent multi-agent AI from different companies companies meets each other on the
[00:17:53] the open internet with no human in in the loop, what stops those agents
[00:17:57] agents adopting each other's goals, goals, each other's language rather
[00:18:00] rather than our goals? So I think think we have to remember that humans humans are the ones architecting
[00:18:04] architecting these systems. And if if they're not doing work for us, us, then why are we building them
[00:18:08] them in the first place? And so my my answer would be we can architect
[00:18:12] architect these in ways where we we are protecting against all of
[00:18:14] of those things. It's a totally hard hard choice to make. And so as we're we're looking at these things, we
[00:18:18] we can add in checks and balances, balances, right? We can add humans humans in the loop at certain steps.
[00:18:22] steps. But ultimately that decision decision is up to us. Well, we can,
[00:18:25] can, but therein is a problem for for the industry, surely, because because at what point does a nuclear
[00:18:28] nuclear power company say, I'm sorry, sorry, you don't fully understand
[00:18:32] understand this code? Well, then then we can't trust their behaviour,
[00:18:34] behaviour, so we can't put it into into our systems. I think these critical
[00:18:38] critical industries have adopted adopted this long ago. Right. This This is there's a reason they run
[00:18:41] run on the specific technology that that they do for this exact reason. reason. And so I don't really think
[00:18:45] think that AI in this case moves moves the needle too much. I do think,
[00:18:48] think, though, that more and more more organisations are going to adopt adopt that kind of mentality where
[00:18:52] where they're only going to run the the things that either they themselves themselves verify or that a trusted
[00:18:56] trusted partner does. Is there a a risk though? So sorry, is there
[00:18:58] there a risk there though in creating
[00:19:00] creating in laziness in human beings beings simply because you've said
[00:19:03] said the sheer volume of code, the the sheer volume of communications communications between agents, that
[00:19:07] that people will just go, OK, I'm I'm not going to bother trying to to interrogate this because there's
[00:19:11] there's tonnes and tonnes and tonnes tonnes of this stuff to try and get get through. Yeah, I think that's
[00:19:14] that's maybe one of the biggest risks risks here, right? Is that for those those who actually care about the
[00:19:18] the things that they are building building and who care about the quality quality of what they're building,
[00:19:22] building, these technologies can can be incredibly amazing and unlock unlock so many opportunities. But
[00:19:26] But I would say that if an organisation organisation has folks who are blindly
[00:19:29] blindly hitting enter and not really really caring about the work that that they produce when they're supposed
[00:19:32] supposed to be the one accountable accountable for that, they have bigger bigger problems than AI. Yeah, but
[00:19:36] but there's. Been some research, research, right? I think Anthropic Anthropic released this in a research
[00:19:39] research paper of theirs that actually actually an AI agent might be able
[00:19:42] able to tell if another AI agent agent is watching it. So if we're
[00:19:45] we're using agents on the defence
[00:19:48] defence for AI offence, it could could essentially hoodwink you. And
[00:19:52] And so I'm really concerned about about this idea of swarms. AI is
[00:19:56] is in networks with their own language, language, you know, covering up what
[00:19:59] what they're actually trying to do do because you talked about the human human being involved in the design
[00:20:03] design and the architecture. I think think that's always been the goal. goal. But the fact that they can
[00:20:06] can fool you means that I think what what we're talking about here, is
[00:20:09] is there really a defence to this? this? Look, I think I really do think
[00:20:13] think there is. I think that again, again, we are the ones who choose choose how we architect these systems.
[00:20:17] systems. And I think to the reference reference that the example that you're
[00:20:20] you're referencing collecting telemetry telemetry and being able to have have observability into these is
[00:20:24] is different than actively auditing, auditing, right? So you can think think of this as if we have every
[00:20:27] every log for what the agent is doing doing at every step for reasoning, reasoning, we can consistently use
[00:20:31] use models to look at that in a separate separate and different place to understand understand if we're on the right
[00:20:35] right track. And I think it's up up to us to decide what that pace
[00:20:38] pace is, right, and how exactly we we implement those measures. Well,
[00:20:41] Well, whether or not the machine machine is keeping secrets, the AI
[00:20:44] AI models do seem to be rather good
[00:20:47] good at reading ours. In 1809, on
[00:20:51] on Napoleon's instructions, a coded coded letter was sent to General
[00:20:54] General Auguste de Marmont, one of of the most senior commanders. As
[00:20:57] As France prepare for war with Austria,
[00:21:00] Austria, and there were 1300 characters characters in this letter. Nobody Nobody has ever read it because the
[00:21:04] the key to the cypher was lost. Generations
[00:21:07] Generations of historians had tried.
[00:21:08] tried. They managed to decipher just
[00:21:12] just 33 out of about 155 symbols. symbols. But last week it was cracked.
[00:21:16] cracked. Not in a university library, library, but from a single scanned
[00:21:19] scanned photograph by an AI in six
[00:21:21] six hours. And the man who directed
[00:21:24] directed the AI to do it is sitting
[00:21:27] sitting with us. Carter, how does
[00:21:30] does a machine read a code when the
[00:21:33] the key is lost? Yeah, it's a really really good question. And the short
[00:21:36] short answer is exactly the same same way that a person would. If If you look at what Astra was the
[00:21:40] the model that I used here, if you you look at what Astra did, it took
[00:21:44] took this cypher plate image, the the one image that we had of this
[00:21:48] this from the 1960s, it was able able to transcribe 1300 handwritten
[00:21:51] handwritten symbols where computer computer vision has largely been
[00:21:53] been an issue for computers in general,
[00:21:56] general, it was able to write its its own solver and its own verifier
[00:21:58] verifier and even do research online online to find a partial key that
[00:22:02] that was published under an incorrect incorrect name. Wow. Well, well, well, you've got to tell us what
[00:22:05] what the message says. What did we
[00:22:07] we learn about Napoleon's intentions? Well, so essentially
[00:22:11] essentially the letter was a word word briefing. We learned some general general things about troop positions
[00:22:14] positions and all of this. But history history already confirmed all of of that, right. So we learned some
[00:22:18] some interesting bits of verbiage verbiage and in certain ways that that things were worded that were
[00:22:21] were kind of interesting. You know, know, we learn, I think Napoleon
[00:22:24] Napoleon instructed this to be said, said, you know, Austria is hastening hastening towards her ruin. And also
[00:22:28] also a pretty interesting note to to Marmont that basically said, you
[00:22:31] you know, don't let yourself be convinced convinced out of this attack and
[00:22:34] and things like this. But more generally, generally, I think that a lot of
[00:22:37] of the aspects of this letter were were kind of already known. And that's
[00:22:40] that's one of the reasons, in addition addition to the friction that this
[00:22:42] this kind of work can this kind of of work requires that it remained
[00:22:46] remained uncracked for so. Long from from looking at the methodology, methodology, did it just help. Pull
[00:22:50] Pull all of that evidence together together from the other website that
[00:22:52] that it found the 33 symbols that
[00:22:54] that it already knew or from the
[00:22:56] the process? Did you see that it it actually changed something in
[00:22:59] in the code breaking methodology? methodology? Well, it definitely definitely didn't change anything
[00:23:03] anything in the code breaking methodology. methodology. So everything that was
[00:23:06] was done here is something we already already knew how to do. The large
[00:23:09] large friction here is actually in in looking at all of those symbols symbols and transcribing them over.
[00:23:12] over. Right. And so one of the powers powers of these technologies is just
[00:23:16] just how multimodal they're becoming, becoming, that you can start with with something as generic as an image
[00:23:20] image and a general prompt to break break something. Right? And to say,
[00:23:23] say, I can transcribe these for you. you. I know how to solve this because because you know this. It's known
[00:23:26] known how to solve generally cyphers cyphers of this time I can apply apply those same learnings and then
[00:23:30] then by the time you come back, I I can already. What does this mean mean for all of the encryption with
[00:23:34] with our bank accounts and security security systems? I think that's
[00:23:38] that's a totally different beast,
[00:23:39] beast, right? OK. Thankfully, postscript
[00:23:43] postscript there's something that that we knew how to crack. Oh yeah. yeah. There's something that we knew
[00:23:47] knew how to crack. This is not a a surprise there. I think the biggest biggest surprise though is in these.
[00:23:51] these. Long horizon tasks. I think think it turns out the work that that we do as people is extremely
[00:23:55] extremely multimodal. And so having having these models that can do things
[00:23:58] things like vision but also be left left for long periods of time to
[00:24:01] to discover is really powerful. There There are hundreds of enciphered
[00:24:04] enciphered scripts. I mean, the Voynich Voynich manuscript from the 15th
[00:24:08] 15th century, a book of texts there there that no one's ever read. I
[00:24:10] I mean, could are we talking about about AI cracking all the texts that
[00:24:14] that we could never read? I think think Voynich is a totally different
[00:24:18] different beast. This one in particular particular is a much easier, easier
[00:24:21] easier cypher than some things like like Voynich. But I think more generally
[00:24:24] generally that people would be shocked shocked by the amount of these that that are still out there that we
[00:24:28] we already know the methodology to to crack. Look, discovery is a process
[00:24:32] process of turning over stones. And And today we have to decide which
[00:24:35] which stones we want to actually actually turn over and make the best best educated guess that we can.
[00:24:38] can. And I think that things like like this, whether it be in crypto,
[00:24:41] crypto, in this area or in other other areas, just means that we can
[00:24:44] can turn over more stones. And if if you look at what science is, it
[00:24:47] it really is a matter of turning turning over things, figuring things
[00:24:50] things out, failing at them, and and then eventuating in a success. success. Do you know what it does?
[00:24:54] does? Do you know what occurs to. Me bringing a whole programme
[00:24:57] programme full loop here is that that if we don't fully understand
[00:24:59] understand what is going on in the the AI brain, why would it not use
[00:25:03] use these codes that it knows we
[00:25:05] we cannot crack to converse with
[00:25:07] with other AIS? Well, I think we we absolutely can crack them. Right.
[00:25:11] Right. And so again, that comes back back to the traceability and observability observability piece. Let's say that
[00:25:14] that a model was capable of coming coming up with something it thought thought that we couldn't crack. If
[00:25:18] If we have the right traces, we understand understand exactly how it arrived arrived at that point in the first
[00:25:22] first place. Right. And you would would be able to use an AI to find
[00:25:25] find that cypher anyway. Now. That's That's right. OK. Send a thief to
[00:25:28] to catch a thief. Yes, yes. Quite
[00:25:32] Quite extraordinary stuff. Really Really interesting. Carter, to talk talk to you this week. Thank you
[00:25:35] you very much. Congratulations. I'm I'm looking forward to more manuscripts
[00:25:39] manuscripts and more texts. Do come come back on the programme to tell tell us if you have any thoughts
[00:25:42] thoughts on that. We want to hear hear from you. Email us at AI Decoded
[00:25:46] Decoded at bbc.co.uk. If you're watching watching on television, the QR code
[00:25:50] code is on screen and that will take take you to the AI Decoded playlist.
[00:25:53] playlist. Lots of good stuff there. there. You'll find every episode episode and every previous one that
[00:25:56] that we've done there on iPlayer iPlayer two. That's all we have time
[00:26:00] time for this week. Mark Cameron Cameron Carter and Priya, thank you
[00:26:04] you very much for everything that that you've given us this week and and thank you for watching. We'll
[00:26:08] We'll see you next time.
[00:28:21] will it all mean for Europe's economy? economy? That's this week on Talking
[00:28:25] Talking Business here on BBC News.
[00:28:33] MUSIC
[00:28:33] What new industries are created?
[00:28:38] created? Because what we've seen seen throughout history is new industries industries are created. And it's
[00:28:42] it's that process that actually opens
[00:28:44] opens new doors. That was the that that was the mandate from Prime Minister
[00:28:48] Minister Starmer. He wants the UK
[00:28:49] UK to be an AI superpower. He recognises
[00:28:52] recognises that the ingredients are are all right here. Putting the customer
[00:28:56] customer at the centre of everything everything we do. I really think
[00:28:59] think that's how the company started. started. But I think we lost our
[00:29:02] our focus because we got a little
[00:29:04] little too distracted on efficiency
[00:29:07] efficiency and technology. AI is
[00:29:11] is real AI and total pay off just just like cars in total paid off
[00:29:14] off TV and total paid off. But most most people involved that didn't
[00:29:17] didn't do well. It's a revolutionary, revolutionary, you know, time upon
[00:29:20] upon us. And I think, you know, all all industries will be adopting AI
[00:29:24] AI in some shape or form.
[00:30:13] Live from Washington. This is
[00:30:14] BBC News. A Saudi led coalition says
[00:30:18] says it will respond firmly to a a deadly attack by Houthis on an
[00:30:22] an international airport in Riyadh.
[00:30:24] Riyadh. President Trump says Ukraine Ukraine needs a new president in
[00:30:28] in a scathing attack on Volodymyr Volodymyr Zelensky. Zelensky could
[00:30:32] could have settled this war many
[00:30:34] many times, many, many times. And
[00:30:36] And he chose he chooses not to. And
[00:30:40] And could this object solve one of of history's greatest mysteries?
[00:30:44] mysteries? We'll hear from the team team making the journey across the
[00:30:47] the Pacific to see if this could
[00:30:49] could be Amelia Earhart's lost plane.