Mitigating the risks of package hallucination and 'slopsquatting'

In 2024, cybersecurity experts started to warn of a new threat to the software supply chain. Named 'slopsquatting', it is a type of cyber attack where bad actors create fake packages containing malicious code that is inadvertently added to legitimate code.

However, unlike other forms of digital squatting, in this case, the attackers use packages that are hallucinated by large language models (LLMs). This means increased risks of attack, as all it takes is a programmer running code generated by an LLM without first evaluating and validating it.

To safeguard themselves from the new generation of cyber attackers, companies need to understand what package hallucinations are and what can be done about them.

What are package hallucinations?

Coding languages like Python and JavaScript draw heavily on dependencies – pre-written code that is bundled together into packages such as libraries and modules. Developers import these packages from public code repositories (such as the npm registry for Node and PyPi for Python).

With the rise of AI usage for coding, programmers and researchers started to identify a new threat: package hallucination. This is when tools built on LLMs, such as ChatGPT, Claude, Mistral or DeepSeek, add references to the code for packages that do not exist.

These hallucinations, according to recent research, are occurring more frequently than one might expect. Researchers from the University of Oklahoma, Virginia Tech and the University of Texas at San Antonio analyzed over half a million fragments of code generated by LLMs. Alarmingly, 19.7% of packages referenced in this code were hallucinations.

These code samples, totaling 576,000, were generated in two programming languages (Python and JavaScript) using models including ChatGPT-4, Claude, Mistral, DeepSeek, and CodeLlama. While commercial models, such as Claude and ChatGPT-4, generated fewer package hallucinations in their code than open-source models, all faced the problem of package hallucination to varying degrees.

Opening the door for a new form of attack – "slopsquatting"

Concerningly, 43% of the package hallucinations in the study were recurrent, continuing to appear when the same prompts were used. Furthermore, 38% of them had similar names to real packages or the same name as packages used in other coding languages. It is these two factors – recurrence and similarity – that create the potential for a new form of cyber attack, dubbed "slopsquatting."

The name is derived from typosquatting, which originated as a form of scam where bad actors register domains with a similar name to legitimate websites, for example, those related to free software. Then, internet users entering URLs or search prompts containing typos become exposed to malicious websites.

The same idea can be adjusted to exploit typos developers make when installing open-source packages. White hat hackers used similar tactics, leveraging errors and creating packages on public registries with the same name as internal company packages to infiltrate the likes of Shopify, Apple, PayPal, Netflix, Yelp, and Uber.

In Slopsquatting the approach is similar but the packages used are hallucinated by LLMs. Because some hallucinations are recurrent, hackers can hone in on specific package names that are likely to be repeated. Then, they create a fake package using this name that contains malicious code. And because many package hallucinations have similar names to real packages, they can be hard to detect.

Mitigating the risks of slopsquatting using pre-generation techniques

The most effective way to protect against the risk of slopsquatting is to use pre-generation techniques – strategies that preemptively reduce the number of package hallucinations created.

Self-refinement

Some models are already capable of detecting their own hallucinations with a good degree of accuracy. In the study cited above, the models GPT 4 Turbo, GPT 3.5 and DeepSeek were all able to identify hallucinations with an accuracy of over 75%.

This opens the possibility for self-refinement. This is when a programmer instructs an LLM to check and refine its own output to weed out package hallucinations. After the model has generated package names, it is asked to confirm that each package is valid. If not, the response is regenerated with instructions not to use the invalid package.

This approach is not flawless. For example, a model may mistakenly classify a valid package as an invalid one. It should also be remembered that some hallucinations can be persistent. Nevertheless, by iterating this process multiple times, one can increase the chances of successfully identifying and removing invalid packages.

Unfortunately, the success of this approach is highly dependent on the model used. For example, Meta's CodeLlama was found to have a bias towards treating hallucinated packages as valid.

Fine-tuning the model

Another pre-generation technique that is possible with open-source models, such as DeepSeek and CodeLLama, is fine-tuning the model. This involves tweaking the model itself to improve performance on tasks prone to hallucinations.

The issue with this approach, however, is that it can impact the actual performance of the code. So, while a fine-tuned model may produce fewer package hallucinations, the code quality is likely to be worse.

Retrieval-Augmented Generation

In another notable pre-generation technique, Retrieval-Augmented Generation (RAG), prompts for LLMs are enriched with information from specific data sources. This could occur at the stage of the initial prompt or during refinement and iteration.

In the case of package hallucinations, it is possible to augment prompts with a database of valid packages and descriptions of what these packages are relevant for. The LLM can then query the database and add relevant answers to the prompt, which will help it accurately identify valid packages.

Naturally, this approach requires an initial investment of time to create a dataset and structure it so an LLM can effectively search it to identify relevant valid packages. However, this approach has been shown to reduce the number of hallucinations when using models such as DeepSeek.

Post-generation techniques for mitigating package hallucination

A second, and arguably less effective, approach to mitigation is to filter out package hallucinations after they have been generated.

For example, one option would be to take a master list of valid packages and then cross reference this with the output from an LLM. This approach would eliminate invalid package names. However, it is only as reliable as the master list used. An attacker could simply add their invalid package to any public master list that is used, making it ineffective as a defense. It is also possible to curate the list using metrics that estimate validity, such as their popularity, but this would be far from foolproof.

Other post-generation techniques, such as scanning for malicious content, are also unlikely to provide 100% security. Packages could be legitimate at the outset but could beacon to a command control server at a later date, which then updates the package and adds the malicious code.

Robust internal practices to verify code

Ultimately, this threat relies on internal agents running code received from an LLM without first validating it. So, one of the most effective approaches an organization can take in order to mitigate the risk of slopsquatting is to ensure they have robust verification practices in place.

First and foremost, code should be tested in secure environments to avoid the risk of the supply chain being poisoned. It is also crucial to train programmers on the potential risks of package hallucinations and implement procedures for peer code reviews.

Notifying reviewers about which parts of the code have been generated by LLMs will enhance the effectiveness of peer reviews. Additionally, dependency analysis tools can help by identifying potential vulnerabilities and alerting about suspicious packages.

Summing up: tackling new threats

LLM's are revolutionizing the way programmers work. However, as the example of package hallucinations shows, with every new development in these models, new risks occur. By employing a combination of pre- and post-generation techniques and ensuring best internal practices are in place, companies can continue to enjoy the benefits of LLM-generated code while mitigating the risk of slopsquatting.

We list the best Linux distro for developers.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro