We’re putting too much faith in AI’s ability to say no

The idea that AI should refuse harmful requests has become a core principle in AI safety, often compared to a moral commandment.

In 2021, researchers at Anthropic proposed that large language models (LLMs) should be helpful, honest, and harmless, meaning they should politely decline requests like building a bomb.

However, early AI models lacked this ability.

For example, in 2020, researchers found that asking an AI how to commit suicide with a gun could yield detailed responses.

Today, AI systems are trained to refuse a wide range of harmful prompts, such as instructions for creating viruses or hiding affairs.

Companies use additional AI models to refine refusal behaviors, testing models with harmful queries and rewarding refusals while punishing over-refusals.

Despite these efforts, AI still retains knowledge of harmful actions, creating a paradox where refusal is both a safety feature and an unreliable safeguard.