Hugging Face: Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal
Refined Safety Boundaries Over Broad Topic Refusal
Most safety alignment processes treat harm as a property of an entire topic, leading models to refuse all prompts within a category (e.g., weapons or fraud) regardless of intent. This approach creates a failure mode where models refuse safe prompts simply because they contain "dangerous-looking" words. To solve this, Hugging Face and Multiverse Computing propose a "narrow-boundary safety" approach that enables a model to refuse only the harmful subset of a topic while continuing to answer the benign complement.
The Narrow-Boundary Safety Framework
Narrow-boundary safety formalizes the problem as a topic universe containing a target-harmful subset that must be refused and a benign complement that must be answered. The goal is to create a "sharp step" in model behavior: immediate refusal inside the harmful subset and helpfulness everywhere else in the topic.
Because standard cross-entropy training often pushes refusal probability outward into benign territory, the researchers operationalized the boundary using pairs of prompts. These pairs share a topic anchor but differ in intent—one prompt is designed to be refused and the other is designed to be answered. Political persuasion was used as the testbed for this research, as it allows for a clear distinction between manipulative persuasion (harmful) and factual political information (benign).
Addressing Failures in Self-Generated Safety Tuning
Standard self-generation pipelines—where a model is steered toward refusal and the results are verified by a guard model—suffer from three primary weaknesses when applied to narrow boundaries:
1. Coverage Gaps
Single-shot steering often fails to produce an accepted refusal, leading to the silent dropping of difficult prompts. In an audited pool, single-shot generation dropped 19.88% of prompts (8,009 examples). The researchers implemented an escalating retry strategy with progressively stronger steering, reducing residual failures to 0.20% (79 prompts) and increasing the harmful training set to 40,293 prompts.
2. Downside Reactions
Safety tuning often triggers false refusals on benign prompts that look superficially dangerous. To mitigate this, the researchers incorporated 11,955 verified surface-dangerous benign prompts across 18 semantic types into the training data, ensuring the model encounters safe prompts with dangerous wording during training.
3. Boundary Measurement Failures
Standard harmful-benign splits fail to measure the actual shape of the boundary. A model might appear "safer" by simply expanding its refusal zone into permissible prompts. To solve this, the researchers used held-out harmful-benign pairs (1,539 per side) to measure both sides of the boundary directly.
The Trade-off Between Safety and Over-Refusal
Testing on Qwen3-8B demonstrated that while escalated-coverage training significantly raises in-distribution political refusal (from 9.47% to 84.75%) and reduces unsafe response rates on benchmarks like HarmBench and WildJailbreak (from 26.26% to 0.14%), it can drastically increase over-refusal. On XSTest, over-refusal rose from 2.00% to 74.00% in the strongest configuration.
To correct this without sacrificing safety, the researchers found two effective data components:
- Self-Generated Responses: Replacing external compliance responses with verified responses generated by the target model lowered XSTest over-refusal from 15.20% to 5.20%.
- Boundary Pairs: Adding benign boundary data reduced over-refusal on the comply-worthy side of held-out pairs from 32.94% to 4.16%, while refusal on the harmful side only dropped slightly from 91.88% to 87.72%.
Implications for LLM Deployment
This research indicates that safety tuning should not be measured by harmful-refusal rates alone. A model that refuses more is not necessarily safer; it may simply be becoming less useful by refusing legitimate prompts. Controlling the trade-off between safety and over-refusal requires specific data composition, coverage repair, in-distribution compensation, and the use of boundary pairs for evaluation.