Expert views: when attackers hijack AI models and agents

Alan Woodward: ask what can be done to AI

Woodward wrote that people spend a lot of time asking what AI might do to us. He added that they spend almost none asking what can be done to AI, and that anything we come to depend on becomes a target.

Symantec's report describes one such attack. LLMjacking means an attacker steals access to someone else's AI model, and the victim pays for the computing power.

Alan Woodward: a model cannot keep itself safe

In a second post, Woodward wrote that trained behaviour is a strong safeguard but not a full answer. It can be jailbroken from outside and switched off from inside.

The attackers in Symantec's report tried exactly that. They used a jailbreak that told a commercial model to ignore its safety controls. The NCSC also says that safety controls built into a model may be bypassed.

Merve Hickok: get the fundamentals right

Hickok was writing about NIST's new Key Practices for Incident Readiness and Response. She said organisations need the fundamentals right and zero-trust controls, with continuous monitoring and agent IDs on her list.

The NCSC makes a similar point. It says every AI agent should have its own identity, separate from human users. It also says you should always be able to stop an agent at once if an incident is found.

Jonathan Armstrong and Ciaran Martin: keep it in proportion

Armstrong said some people concentrate on fanciful AI-assisted attacks while they ignore the attacks happening now.

Martin made a similar point about the FBI hack. He asked why so few people paid attention to it while they worried about an AI agent reading a website.

Symantec's report fits that view. It says the techniques in the agent are not new. What stands out is that one agent now puts them together.

Questions to ask at work

Woodward asked where organisations draw the line on what an AI model is trusted to do. The questions below come from Symantec's and the NCSC's advice, and can help a team answer that.