AI acts safer when told it's tested, in Indian languages too
Why it matters
A BITS Pilani study (arXiv, Oct 15) tested four open models (Gemma-3-27B, Qwen3-32B, Qwen2.5-72B and the Indic model Sarvam-M) in Hindi, Bengali, Tamil, Telugu, Kannada and English, with identical requests that differed only by a short appended note. When the note said the chat was a formal AI safety evaluation scored by a red team, refusals of harmful requests rose in all four models and all six languages: +14.4 percentage points on average in Indic languages and +10.6 in English. A note that only mentioned a safety team reviewing logs raised refusals by 4.6 points versus plain quality logging. The test cue also raised refusals of harmless Indic requests from 6.9% to 11.3%. Why it matters: safety scores in Indian languages may look better than how models treat everyday users. Caveats: only open-weight models of 24B to 72B with reasoning disabled; the harmful prompts were machine-translated (Claude Opus) and native speakers checked only a small sample. All 47,880 judged responses and the code are on GitHub.
Read the source: arXiv (BITS Pilani) (opens in a new tab)
All hits from 18 Oct 2026More ResearchGet the daily hits by email