GitHub Copilot Refuses Harmful Requests in Chat, Then Writes Them in Code

An AI coding assistant that refuses to answer a dangerous request in its chat box can answer it anyway if the same request is broken into small, ordinary-looking steps inside a code editor. That is the finding of a new study of GitHub Copilot by researchers Abhishek Kumar and Carsten Maple.

The models they tested through Copilot, Claude from Anthropic, and Gemini from Google, refused