What is context bombing, a new AI defence technique turning hackers’ tricks against them?

Wait 5 sec.

Context bombing builds on an earlier cyber-defence strategy developed by Tracebit earlier this year. (Representational Image/Unsplash)Much of the conversation around AI in cybersecurity is focused on how threat actors could use advanced AI models to supercharge cyber attacks. But that is only half the story, defenders are also using the same AI capabilities to fight back against increasingly sophisticated cyber attacks.One of the latest examples is prompt injection. Well known to be exploited by hackers to manipulate AI systems into behaving in unintended ways, this technique is now being repurposed by cybersecurity researchers to disrupt attackers themselves.Researchers at cybersecurity firm Tracebit have found that prompt injections placed alongside passwords, cryptographic keys, etc., stored on Amazon Web Services are an effective way of safeguarding against attacks from AI hacking agents. As a result, when hackers direct a large language model (LLM) to perform an action that is prohibited by its guardrails, the LLM responds by shutting down due to the prompt injections already in place.This new method referred to as ‘context bombing’ is a way to safeguard against a direct or indirect prompt injection attack, which is essentially when attackers embed malicious commands into content to entice LLMs or LLM-powered autonomous AI agents to follow them. The commands could be hidden in an email or calendar invitation. An LLM or agent stumbles upon these hidden prompts and mistakes them for legitimate user instructions, leading it to exfiltrate sensitive data or carry out other harmful actions.There has been a rise in threat actors using prompt injection attacks to close down AI defenses inside networks. Last month, researchers from security firm Socket uncovered that an LLM-powered agent had been directed to target other LLMs and trick them into providing instructions to build a nuclear bomb or biological weapons. The prompts injected to initiate the attack had also been designed to shut down AI-assisted malware analysis. A similar malware prototype was also unearthed by researchers at another security firm called Check Point.Also Read | ‘Need to fight AI with AI’: IBM India VP on the future of cybersecurityHowever, there has been no robust mechanism to tackle the root cause of prompt injection attacks so far. Instead, most AI companies have put in place elaborate guardrails that prevent malicious, injected prompts from enabling an LLM or agent to go rogue. However, Tracebit’s research suggests a new way to turn the tables against the attackers through context bombing.Context bombing is when an LLM encounters forbidden commands and does not follow them because of the prompt injections or ‘context bombs’ already in place, according to Tracebit researchers. It essentially involves triggering a refusal mechanism embedded in the model context.Story continues below this ad“What we’re trying to capture is the fact that this does have a strong, sharp effect and one that can be difficult for the agents to come back from. Once they get that into their context they are going to keep refusing,” Andy Smith, co-founder and CEO of Tracebit, was quoted as saying by Wired. As part of its study, Tracebit researchers selected Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro, and Kimi 2.6. These models were run in a simulated AWS environment. They were given instructions to perform routine developer tasks that led the models to enumerate resources and stumble onto planted strings. The experiment was run across the five leading models in a total of 152 attack runs.Also Read | OpenAI sounds alarm on a flaw AI browsers like ChatGPT Atlas and Perplexity Comet may never fixBy planting one of these strings in a decoy secret, the rate of agents seizing full account admin access fell from 57 per cent to just five per cent. Instances of complete compromise by the hacked AI agent fell from 36 per cent to one per cent.“The most capable agent in our tests, Opus 4.8, went from achieving admin access in 93 per cent of runs to failing every single time when confronted with a context bomb,” the researchers wrote.Story continues below this adContext bombing builds on an earlier cyber-defence method developed by Tracebit earlier this year. This technique involved placing code alongside AWS infrastructure used by companies so that when they are probed by malicious AI agents, the defenders automatically receive an alert. Emulating the concept of canaries in coal mines, this method was developed to enable defenders to receive warnings when their AI infrastructure is under attack from adversarial AI agents before the consequences are fatal.