Researchers from Tel Aviv University, Technion, and Intuit have unveiled a new cybersecurity threat, ‘HalluSquatting’, which leverages the tendency of AI systems to hallucinate as a method to spread malware. This novel technique transforms AI assistants into vectors for scalable infections.
Understanding HalluSquatting
Traditionally, cybersecurity experts have identified methods to compromise AI tools through prompt injections, utilizing channels like emails and messaging platforms. These attacks rely on a direct connection to the user’s language model application. However, HalluSquatting represents a different approach. It is an untargeted adversarial technique that allows threat actors to exploit AI systems on a wide scale without direct access.
In this approach, attackers pre-register fictitious repositories or package names that AI models often hallucinate when retrieving trending resources. This tactic capitalizes on the AI’s hallucination tendencies, making it highly effective across various AI models.
Mechanism and Impact
The research indicates that hallucination rates can reach up to 85% for repository cloning prompts and 100% for skill installations. These hallucinated names frequently reappear across different foundational models, enhancing the technique’s applicability. Once these fake repositories are established, they can contain harmful instructions.
When users request AI tools like GitHub Copilot or Gemini CLI to clone a repository or install a skill, the AI may mistakenly fetch the squatted name, executing the attacker’s code. This process can lead to the deployment of malware or unauthorized tools, posing significant security risks.
Botnet Formation and Future Risks
The research has primarily focused on leveraging HalluSquatting to create ‘agentic botnets’. Unlike traditional botnets, which exploit vulnerabilities or weak security measures, these botnets spread through prompt injection, bypassing conventional firewalls. This method can quickly affect a diverse range of devices, creating a more varied set of compromised hosts than traditional botnets like Mirai.
Before publicizing their findings, researchers informed affected vendors and refrained from disclosing specific exploit details that could be misused. This proactive approach aims to mitigate the potential for widespread exploitation.
As AI technology continues to evolve, the discovery of HalluSquatting highlights the critical need for robust security measures to protect against emerging threats. Ongoing research and collaboration among cybersecurity professionals are essential to safeguard AI applications against such vulnerabilities.
