Offensive security researchers, the ones finding vulnerabilities before criminals do, can’t use frontier models for their work. OpenAI and Anthropic’s guardrails block requests that smell like exploits, even when the person asking is a defender.

Chris Anley, chief scientist at NCC Group, explains it with a concrete example: asking a model to exploit a bug is the fundamental step to confirm the vulnerability is real and needs fixing. If the guardrail refuses, the defender gets stuck. The prompt “fix this code” serves both defense and attack: separating the two uses is impossible without breaking the tool itself.

Some researchers have moved back to open-source models without guardrails. Paolo Stagno of Crowdfense avoids cloud models for a second reason too: uploading data to a cloud model means risking it ends up in future training cycles. For bugs worth millions of dollars, that’s an unacceptable risk.

The paradox is stark. Proprietary labs push to restrict open-weight models citing security risks, as we reported yesterday. But open models without guardrails are the only ones security researchers can use without blocks or data leaks. The security the labs want to build into their models is the same thing pushing serious professionals away from them.

In Detail

Multiple offensive security researchers, the cybersecurity professionals who actively hunt for vulnerabilities in systems, describe a concrete operational problem. This isn’t generic complaining: it’s a blocker on daily work.

Access programs exist. OpenAI has Trusted Access for Cyber, Anthropic has the Cyber Verification Program: security professionals can apply, pass vetting, and get models with fewer restrictions. But approval isn’t guaranteed and even loosened guardrails keep getting in the way.

Chris Anley, chief scientist at NCC Group, explains this with a precise example. When you find a possible bug, the next step is confirming it by building an exploit. If you ask an AI model to try and the guardrail refuses, you lose the tool exactly when you need it most. The prompt “fix this code” is the same one you need to close the vulnerability and understand it deeply. Anley compares it to a hammer: tool and weapon, inseparable.

Paolo Stagno, CTO of Crowdfense, adds a second layer to the problem. Even without guardrails, using a cloud model to work on unpatched vulnerabilities means uploading sensitive data to third-party infrastructure. The risk is that data ends up in future training cycles. That’s why Stagno uses frontier models only for reverse engineering and switches to local open-source models for everything involving discovery and exploit development.

Giuseppe Cali, another researcher interviewed, takes a different position. He uses AI only for initial reverse engineering and building support tools, not for bug discovery. His stance suggests that for now, AI’s role in offensive security is complementary rather than central.

There’s also the export control context. In June, the US government imposed restrictions on Anthropic’s Mythos and Fable models, partly based on a report showing guardrails designed to prevent attacks could be bypassed. The restrictions were partly revoked: Fable 5 returned to general access July first, Mythos 5 was reintroduced only for verified American organizations.

Days ago we reported how a researcher found an RCE vulnerability in WordPress spending $25 of compute with GPT-5.6 (/en/radar/wordpress-rce-gpt-5-6-sol-security-research-25-dollars). That example shows AI’s potential for defensive security: the exact type of work guardrails risk making harder.

For those using AI in serious technical contexts, the picture has two concrete implications.

The choice between cloud and local is also a choice of data sovereignty. When your data is unpatched vulnerabilities, uploading it to a cloud model is a business risk, not just a technical one.

The rhetoric around frontier model security and the practice of offensive security point in opposite directions. Labs want models that can’t be used to attack, but attacking is exactly how you discover vulnerabilities to close. The question is where guardrails stop: right now they stop too early, leaving defenders with a mutilated tool.