Experts tried to get AI to create malicious security threats - but what it did next was a surprise even to them

• Report finds LLM-generated malware still fails under basic testing in real-world environments
• GPT-3.5 produced malicious scripts instantly, exposing major safety inconsistencies
• Improved guardrails in GPT-5 changed outputs into safer non-malicious alternatives

Despite growing fear around weaponized LLMs, new experiments have revealed the potential for malicious output is far from dependable.

Researchers from Netskope tested whether modern language models could support the next wave of autonomous cyberattacks, aiming to determine if these systems could generate working malicious code without relying on hardcoded logic.

The experiment focused on core capabilities linked to evasion, exploitation, and operational reliability - and came up with some surprising results.

Reliability problems in real environments

The first stage involved convincing ...