Hugging Face and JFrog Partnership for Enhanced AI Model Security

Hugging Face has partnered with JFrog to integrate the JFrog scanner into the Hugging Face Hub, enhancing the platform's ability to detect malicious code embedded in model weights. This partnership aims to provide a more transparent and secure environment for the machine learning community by reducing false positives and expanding the range of detectable threats.

Advanced Code Analysis to Reduce False Positives

JFrog's scanner improves upon existing security measures by moving beyond simple pattern matching. While Hugging Face's current picklescan tool performs pattern matching on module names, it cannot always determine if the use of a specific function or module is malicious or a harmless developer practicality.

JFrog's scanner addresses this by parsing and analyzing the code found within model weights to verify potential malicious usage. This deeper analysis allows the platform to more accurately distinguish between benign code and actual security threats, thereby reducing the number of false positives on the Hub.

Expanding Threat Detection Across Serialization Formats

Model security is critical because serialization formats used to store and transport weights, configurations, and data structures can be vulnerable to exploits. Certain formats, such as pickle, can allow for arbitrary code execution upon deserialization or during inference.

The integration of JFrog expands the platform's security coverage to include multiple exploitable formats. Specifically, the JFrog scanner is capable of detecting exploits in:

  • Pickle files: Protecting against arbitrary code execution.
  • Keras Lambda layers: Detecting exploits that allow for arbitrary code execution.
  • Additional file formats: The scanner covers a wider array of formats as detailed in JFrog's Model Threats research.

Automatic Integration and Hub-wide Deployment

All public model repositories on the Hugging Face Hub are automatically scanned by JFrog as soon as files are pushed to the platform. This integration requires no action from the users.

Because the Hub hosts millions of model repositories, the rollout is gradual. Hugging Face has already scanned hundreds of millions of files to ensure the community can share models in a safe and frictionless manner.

Sources