Ant Group open-sources SingProbe, embedding safety checks into LLM decoding with under 0.5% overhead
The AMW Read
Open endogenous decode-time guardrail with measured sub-0.5% overhead and 29-model coverage meaningfully updates how CN open-weight stacks ship inference safety.
Ant Group open-sources SingProbe, embedding safety checks into LLM decoding with under 0.5% overhead
Ant Group's AI Safety Lab released SingProbe, an endogenous safety guardrail that scores risk while the model is still generating tokens rather than waiting for a finished reply. In production tests on Ling-3.0-flash, the team reported less than 0.5% extra decode overhead. The flash build beat selected public baselines on answer-safety classification and streaming safety detection, and roughly matched baselines on hallucination detection. Alongside the release, Ant published SingStreamBench, a benchmark that measures when a streaming reply flips from safe to risky content. A medical variant, SingProbe-Med, corrected 25.03% of previously wrong answers in tests on AntAngelMed-100B by intervening only on high-risk spans. SingProbe already covers 29 open-source models spanning Ling-3.0, GLM-5.2/5.3, Qwen, and DeepSeekV4, plugs into SGLang and vLLM, and ships with open code, models, and benchmarks.
Most production LLM apps still bolt on separate input or output moderators, which either delay risk detection until the full answer is shown or burn extra compute with frequent external checks. Moving the probe inside the generation loop attacks that tradeoff directly and matters most where Chinese open-weight stacks are being productized for customer service, office assistants, and medical Q&A. Open adapters for Qwen, DeepSeek, and GLM plus vLLM and SGLang hooks make this less a closed Ant-only feature and more a candidate default for open-model serving pipelines.
Builders should treat endogenous, streaming risk scores as a deployment option alongside classic external guardrails, especially for high-stakes verticals where aborting mid-generation is preferable to scrubbing after the fact. Investors watching China foundation-model and inference stacks should note that safety differentiation is shifting from post-hoc filters toward low-overhead probes wired into the decode path, with Ant using open release and multi-model coverage to set that standard rather than keeping it proprietary.
