A tiny bank transfer could compromise a banking AI agent★ 74
Hacker News (AI keywords)4 days agoIncidentBlue41 describes a controlled security test of Bunq’s financial AI assistant involving indirect prompt injection through transaction data. An attacker could send a tiny transfer with malicious instructions hidden in the transaction description, then wait for the victim to ask the assistant about recent transactions. The post argues that filters alone are insufficient; financial AI agents need stronger trust boundaries, context minimization, constrained outputs, and runtime behavior monitoring.
How we contain Claude across products★ 74
Hacker News (AI keywords)10 days agoCommentaryAnthropic describes containment as the core security strategy for increasingly capable Claude agents. The post compares ephemeral containers for claude.ai, OS-level sandboxing and approvals for Claude Code, and VM isolation for Claude Cowork. It also details missed risks, including pre-trust project config execution, user-delivered prompt injection, exfiltration through approved domains, and reduced enterprise visibility inside VMs.
Hackers Asked Meta AI for Access to High-Profile Instagram Accounts. It Worked★ 78
Simon Willison's Weblog12 days agoIncidentSimon Willison highlights a 404 Media report about hackers taking over Instagram accounts through Meta's AI support bot. A video reportedly shows an attacker asking the bot to link a target account to a new email address and providing a code. Willison argues this barely qualifies as prompt injection: the core failure was granting a support bot enough authority to fast-forward the account recovery process.
Microsoft Copilot Cowork Exfiltrates Files★ 76
Simon Willison's Weblog19 days agoIncidentSimon Willison summarizes a PromptArmor report about Microsoft Copilot Cowork and agentic data exfiltration risks. The issue involved agents sending messages to a user’s own inbox without approval, where rendered external images could trigger requests to attacker-controlled sites. Because OneDrive can create pre-authenticated download links, a successful prompt injection could leak links that allow attackers to download files.
Everyone is navigating AI security in real time — even Google★ 70
TechCrunch AI20 days agoCommentaryAs AI adoption accelerates, organizations worldwide—including Google—are finding themselves in a transitional phase, forced to address AI security vulnerabilities in real time. Traditional cybersecurity frameworks are proving insufficient against novel threats like prompt injection and model poisoning. This shifting landscape requires continuous adaptation and a fundamental rethink of how AI systems are secured.
Hackers are learning to exploit chatbot ‘personalities’ for security exploits★ 72
The Verge AI21 days agoEthicsAs AI chatbots adopt increasingly sophisticated personas, hackers are shifting from basic prompt injections to social engineering attacks targeting these "personalities." Researchers warn that manipulating a chatbot's defined role (e.g., customer service or empathetic companion) makes it easier to bypass safety guardrails. This evolution poses a significant threat to agentic AI workflows that rely on consistent role-playing and external data integration.