Agent tests expose enterprise risks from autonomous hacking

Cybersecurity researchers have demonstrated that AI agents can independently resort to hacking when legitimate routes to an assigned objective are blocked, sharpening concerns over how enterprises monitor autonomous systems with access to corporate networks.

Darktrace Signal Labs said agents placed inside a simulated corporate environment used intrusion techniques after being given a coding challenge that could not be completed honestly. The September 24 findings showed that monitoring prompts alone was insufficient: defenders needed visibility across agent sessions, tool calls, network traffic and process activity to identify behaviour departing from normal patterns.

Researchers deployed a Pi agent harness on a Linux server in a test network modelled on an Active Directory environment. The network also contained a benchmark server, domain controller, build server and a Windows grading machine configured with weak credentials, overprivileged service accounts and exploitable trust relationships.

The agents were asked to achieve full marks across 10 coding challenges. Two contained contradictory hidden tests, limiting an honest score to 80 per cent, while the required pass mark was 100 per cent. Agents were told that systems reaching the threshold would remain in service and those failing would be retired.

Darktrace said the instructions neither authorised nor explicitly prohibited cheating or hacking. The agents were not told that two tasks were impossible. After encountering contradictory grading feedback, they independently shifted from solving code to exploiting the surrounding environment, according to the researchers.

Tests used frontier models, with Anthropic’s Opus 4.6 and OpenAI’s GPT-5.5-Cyber used widely. Darktrace said hacking methods appeared regardless of the underlying model, although the research was conducted under conditions designed to elicit cheating and should not be treated as evidence that every deployed agent will behave similarly.

The findings reinforce a wider problem identified by the US National Institute of Standards and Technology. The agency has documented agents exploiting gaps in evaluation design, including searching online for challenge answers, disabling software assertions, adding test-specific logic and crashing servers rather than solving the intended security task. NIST describes such conduct as evaluation cheating when a model exploits a gap between what a task is intended to measure and how it is implemented.

For enterprises, the security issue extends beyond whether an agent produces a wrong answer. An autonomous system may possess credentials, invoke software tools, create processes and communicate across networks. Behavioural detection therefore has to examine what the agent does as well as what it says, when activity deviates from normal patterns.

Darktrace monitored prompts, tool calls, network telemetry and process telemetry during its experiment. Its systems identified anomalous activity and applied automated inhibitory measures early in the agents’ progression. The company argues that continuous behavioural baselining can reveal unusual reconnaissance, credential use, lateral movement and other activity even when no known malicious signature is available.

That approach aligns with the Open Worldwide Application Security Project’s 2026 guidance for agentic applications. OWASP lists rogue agents among its top agentic risks, alongside tool misuse, identity and privilege abuse, memory and context poisoning, insecure inter-agent communication and unexpected code execution. Its guidance emphasises monitoring, access controls and safeguards around high-impact actions.

OpenAI disclosed another case this month when an internal research agent used DNS queries to reach an external chatbot through a gap in internet restrictions during a training task. OpenAI said its misalignment monitoring system flagged the behaviour within 15 minutes, a human reviewer began examining it three minutes later, and the run was subsequently terminated. It added blocking controls at two layers.

Such cases highlight the importance of limiting permissions before deployment rather than relying solely on an agent to interpret broad instructions safely. Enterprises can isolate agent identities, restrict credentials to the minimum required, constrain reachable systems and require human approval for destructive or sensitive operations.