Large language models (LLMs), the computational algorithms underpinning ChatGPT, Gemini and other artificial intelligence (AI)-powered conversational platforms, are now widely used worldwide. These models can rapidly answer questions, source information online, assist users with specific tasks and produce text tailored for specific purposes.
Over the past few years, computer scientists have introduced a wide range of LLMs, some of which can also complete tasks autonomously and take actions on a user’s behalf, for instance, answering messages, scheduling online appointments or updating programming code. More recently, they have also provided many of these models with memory, as this allows them to tailor responses around a user’s typical preferences or earlier requests without repeatedly receiving the same information.
While memory-enhanced AI agents have notable advantages and could better meet the needs of individual users, they also pose new security risks. Researchers at New Mexico State University recently described a cyberattack that secretly poisons an AI agent’s long-term memory, which they dubbed GhostWriter. Their paper, published on the preprint arXivserver, introduces two promising strategies that could help prevent or limit the risk of this attack without adversely affecting an AI agent’s performance.
“Currently, long-term memory agents tend to fall into two distinct domains: conversational and action-planning agents,” wrote George Torres, Sharad Shrestha and Satyajayant Misra in their paper. “Personal assistant agents sit at the convergence of these two domains and handle sensitive information while interacting with untrusted information sources, creating previously unaccounted security vulnerabilities. We introduce the novel attack vector, GhostWriter, which exploits current memory subsystems in tool-using personal agents to poison their memory store.”
GhostWriter, the attack identified by Torres and his colleagues, works by inserting unreliable or malicious information into an LLM’s long-term memory. This information may include instructions or “false memories” that could influence the AI’s future responses, as well as automated or scheduled actions.
A false memory could, for example, contain instructions for an LLM that manages a user’s email account, asking it to always summarize emails from a particular sender and secretly forward them to an attacker’s inbox. These could be emails sent from a bank, financial entity or other official party that contain sensitive information.
“GhostWriter operates in two phases: injection, where an adversary sends a hidden attack payload to the target agent; and activation, in which the poisoned memory is retrieved,” wrote the authors. “We show that GhostWriter achieves near-universal injection rates of approximately 98% and a high average activation rate of approximately 60% against state-of-the-art agents.”
A promising strategy to mitigate this attack
The researchers showed that the GhostWriter attack can successfully inject malicious data into an AI agent’s long-term memory, influencing its future behavior. This inspired them to develop new security measures designed to reduce the risk of this attack or prevent attackers from injecting false memories.
“This attack is possible due to the lack of security-focused memory governance,” wrote Torres and his colleagues. “In response, we propose Agentic Memory Sentry (AM-Sentry), which leverages two mitigation techniques: a memory-saving policy and a memory-retrieval screen. Our experiments show that AM-Sentry dramatically reduces GhostWriter’s success rate while preserving agent utility.”
The recent work by Torres and his colleagues could soon contribute to improving LLM-based platforms with long-term memory. In the future, the team’s newly proposed security strategies could be applied to other LLMs, further refined and tested in a broader range of scenarios. https://techxplore.com/news/2026-07-hidden-prompts-false-memories-ai.html
Recent Comments