Hugging Face RiskRubric.ai Announcement
Hugging Face has announced RiskRubric.ai, a standardized risk assessment platform designed to provide consistent and comparable risk scores across the AI model landscape. The initiative, led by the Cloud Security Alliance and Noma Security, with contributions from Haize Labs and Harmonic Security, aims to democratize AI safety by providing developers with a systematic way to evaluate a model's security posture, privacy implications, and potential failure modes.
Standardized Risk Assessment Framework
RiskRubric.ai evaluates models based on six core pillars: transparency, reliability, security, privacy, safety, and reputation. To ensure evaluations are reproducible and transparent, the platform utilizes Noma Security capabilities to automate the following tests:
- Reliability: Over 1,000 tests checking for consistency and edge case handling.
- Security: More than 200 adversarial security probes targeting jailbreaks and prompt injections.
- Automated Code Scanning: Analysis of model components.
- Documentation Review: A comprehensive review of training data and methods.
- Documentation Review: A comprehensive review of training data and methods.
- Privacy: Assessments including leakage testing and data retention.
- Safety: Structured tests for harmful content.
Each pillar is assigned a score from 0-100, which is then converted into a letter grade (A-F). The platform provides specific vulnerabilities found, recommended mitigations, and suggestions for improvements, allowing developers to filter models based on specific requirements, such as high privacy scores for healthcare applications.
Key Findings from September 2025 Data
Evaluations of both open and closed models using these standards have revealed several key trends in the AI risk landscape:
Risk Distribution and Polarization
Total risk scores range from 47 to 94, with a median of 81. While 54% of models are rated A or B level, there is a long tail of underperformers. Models in the 50–67 (C/D range) band provide only medium to low overall protection, representing a critical area of concern for security gaps.
Correlation Between Security and Safety
The Safety & Societal pillar shows the widest variation across models. Data indicates that models with strong security hardening—such as prompt injection defenses and policy enforcement—consistently score higher on safety. This suggests that safety is often a byproduct of a robust security posture.
The Transparency-Security Trade-off
Stricter protections often lead to decreased transparency, as models may issue refusals without explanation. To maintain trust without compromising security, the platform suggests pairing strong safeguards with auditability, provenance signals, and explanatory refusals.
Community-Driven Improvement
RiskRubric.ai encourages the community to participate by submitting models for evaluation or providing feedback on the assessment methodology. By making risk assessments public and standardized, the community can identify where models need strengthening and can contribute fixes, patches, and safer fine-tuned variants, creating a virtuous cycle of improvement.