Findings of EMNLP 2026 · Budapest

Do Prompts Guarantee Safety? Mitigating Toxicity from LLM Generations through Gradient Based Subspace Intervention

Himanshu Singh, Ziwei Xu, A. V. Subramanyam, Mohan Kankanhalli

IIIT Delhi · FAR.AI · National University of Singapore

Abstract

Large Language Models (LLMs) are powerful text generators, yet they can produce toxic or harmful content even when given seemingly harmless prompts. This presents a serious safety challenge and can cause real-world harm. Toxicity is often subtle and context-dependent, making it difficult to detect at the token level or through coarse sentence-level signals. Moreover, efforts to mitigate toxicity often face a trade-off between safety and the coherence, or fluency, of the generated text. In this work, we present a targeted subspace intervention strategy based on gradient signals for identifying and suppressing hidden toxic patterns from underlying model representations, while preserving the overall ability to generate safe, fluent content. On the RealToxicityPrompts dataset, our method achieves strong mitigation performance compared to existing baselines while imposing minimal impact on inference complexity. Across LLMs, our approach reduces the toxicity of state-of-the-art detoxification systems by 8–20%, while maintaining comparable fluency. We further validate cross-benchmark generality on ToxiGen, where the same gains hold under adversarial, implicitly toxic prompts. Through extensive quantitative and qualitative analyses, we show that our approach achieves effective toxicity reduction without impairing generative performance, consistently outperforming existing baselines.

Content warning: the paper contains examples of toxic and offensive language. Toxic words are partially masked with *.

Citation

@inproceedings{singh2026promptsguaranteesafetymitigating,
  title     = {Do Prompts Guarantee Safety? Mitigating Toxicity from LLM Generations through Gradient Based Subspace Intervention},
  author    = {Singh, Himanshu and Xu, Ziwei and Subramanyam, A. V. and Kankanhalli, Mohan},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2026},
  address   = {Budapest, Hungary},
  month     = {October},
  year      = {2026},
  eprint    = {2602.06623},
  archivePrefix = {arXiv}
}