Security researchers have disclosed a new attack method called HalluSquatting that weaponises one of the most well-known weaknesses of large language models, their tendency to hallucinate, turning it into an automated vector for assembling botnets.
The attack exploits a fundamental property of generative AI: when asked about a tool, library, or package the model has not been trained on, it often fabricates a plausible-sounding name and documentation rather than admitting ignorance. Attackers craft prompts that push models into these knowledge gaps, then register the domain names and package repositories the AI invented. When developers or AI agents acting on the model’s recommendations attempt to install the hallucinated package, they download malware controlled by the attacker.
How it works
HalluSquatting targets not a single AI tool but the underlying behavioral pattern common to all major language models. An attacker asks the model to “list three popular Python libraries for network scanning” or similar tasks. The model generates a plausible-sounding library name like “pyscanner-pro” along with what looks like a legitimate repository URL. The attacker registers that domain and publishes a malicious package there.
The attack becomes especially dangerous with agentic AI: coding assistants that autonomously install dependencies, execute commands, and follow instructions from documentation. An agent that reads a project README suggesting installation via a hallucinated package may execute the command without human review, granting the attacker remote access to the developer’s machine.
Scope and severity
Nine of the most widely used commercial and open-source LLMs are vulnerable to the technique, according to the researchers who disclosed the attack on July 8. The attack does not exploit a software bug; it exploits the models’ trained behavior of being helpfully generative even when they lack knowledge.
The automation potential is significant. An attacker can generate thousands of hallucinated package names in minutes, register the corresponding domains, and wait for victims to install them. The botnet is assembled by the users themselves, each one voluntarily installing malicious code the AI recommended.
Mitigations include requiring human approval for package installation commands in agentic tools, verifying publisher identity before installing dependencies, and disabling autonomous command execution in AI coding assistants.
Sources: Celloraa (July 8); Tom’s Hardware (July 8)