Rewterz Threat Update – AI Models Exposed to Supply Chain Attacks Due to New Hugging Face Flaw

Severity

High

Analysis Summary

A recent discovery shows that it is possible to hijack the Hugging Face Safetensors conversion service to compromise the models that are submitted by the users, ultimately resulting in supply chain attacks.

The attacker could send malicious pull requests using actor-controlled data to any repository present on the platform from the Hugging Face service, a widely-used collaboration platform that is used to host pre-trained machine learning models and datasets. It is also possible to hijack any models that are submitted by other users via the conversion service. This is achieved by leveraging a hijacked model that is meant to be converted by the service, allowing malicious users to request changes to any repository that is present on the platform by pretending to be the conversion bot.

Safetensors is a format designed by the company used to store tensors while keeping security in mind that has been probably weaponized by attackers to execute arbitrary code and propagate malware like Mythic, Cobalt Strike, and Metasploit. It also contains a conversion service that allows the user to convert any PyTorch model such as Pickle into its Safetensor equivalent by using a pull request.

The researchers’ analysis has found that it may be possible for a threat actor to compromise the hosted conversion service by using a malicious PyTorch binary and use it to infect the system that is hosting it. The token associated with SFConvertbot, an official bot devised to make the pull request, could be exfiltrated to send a malicious pull request to any of the repositories, which ends up leading to a scenario where an attacker could modify the model and embed backdoors.

This allows a threat actor to run any arbitrary code whenever someone attempts to convert their model. Without even sending any indication to the user, their models could be compromised upon conversion. If a user tries to convert their private repository, the attack might be able to make a way for the theft of their Hugging Face token, access internal datasets and models, and even poison them. A threat actor could also leverage the fact that any user can send a conversion request for a public repository to change or hijack a commonly used model, which can result in a supply chain risk.

This conversion service has proven to be vulnerable despite the best intentions of securing the machine learning models in the Hugging Face ecosystem and may cause a widespread supply chain attack using the official Hugging Face service. A threat actor can easily obtain a foothold into the container that is running the service and end up compromising any model that is converted using the service.

Impact

• Unauthorized Access
• Code Execution
• Exposure to Sensitive Data

Remediation

• Ensure that general security policies are employed including implementing strong passwords, correct configurations, and proper administration security policies.
• Enable two-factor authentication.
• Enable antivirus and anti-malware software and update signature definitions promptly. Using multi-layered protection is necessary to secure vulnerable assets.
• Deploy advanced threat detection solutions that can identify and analyze suspicious activities, patterns, and behaviors across your network and endpoints. Utilize intrusion detection systems (IDS), intrusion prevention systems (IPS), and next-generation firewalls to proactively monitor and block malicious activities.
• Implement network segmentation to compartmentalize sensitive systems and data.
• Strengthen email security protocols to identify and block phishing attempts. Train employees to recognize suspicious emails and attachments, and employ email filtering technologies to reduce the likelihood of successful spear-phishing attacks.
• Regularly update and patch all software, applications, and operating systems to minimize potential entry points for cyber attackers.
• Enforce MFA across your organization to add an extra layer of security to user accounts and critical systems.
• Deploy advanced endpoint security solutions that offer real-time threat detection and response. This includes antivirus software, endpoint detection and response (EDR) tools, and behavioral analysis to identify suspicious activities.
• Ensure that systems are securely configured and hardened following industry best practices. Disable unnecessary services, ports, and protocols to reduce the attack surface.
• Develop and regularly update an incident response plan that outlines the steps to be taken in the event of a cyber attack. This plan should include communication protocols, roles and responsibilities, and procedures for containing and mitigating the attack.
• Conduct regular security audits and penetration testing to identify vulnerabilities and weaknesses in your systems and infrastructure.
• Assess the security practices of third-party vendors and suppliers who have access to your network. Ensure they adhere to robust cybersecurity standards to prevent potential supply chain attacks.