AI penetration testing: the complete guide
Published
AI penetration testing is security testing where an AI agent plans the checks, runs the testing tools and investigates weaknesses in systems you're authorised to test. A good tool then proves impact with working exploits, so your team can see what happened and decide what to fix.
This guide is for CTOs, engineering leads and IT managers who are new to AI pentesting or weighing it against a traditional pen test. It covers how the testing works, what it can miss, how it compares with manual testing, what it costs and how to choose and run a tool safely. It's written by BreachLine Labs, the team that builds Nebula, so we'll be honest about where AI falls short.
Key takeaways
- AI penetration testing uses an agent that plans tests, uses tools and investigates the results. A vulnerability scanner runs a fixed set of checks and reports matches.
- A finding is only worth acting on if it comes with evidence. Look for a working exploit and an independent check before a result reaches you.
- AI testing is strong on breadth, repeatability and frequency. People still matter for complex business logic and unusual attack paths.
- A report with no findings isn't proof that you're secure. Always check what was tested and what wasn't.
- Pricing usually follows one of three models: per test, subscription or credits. BreachLine plans start at £399 a month, and retests use credits.
What is AI penetration testing?
A penetration test is an authorised attempt to break into your systems the way an attacker would, so you can fix weaknesses before someone else finds them. In AI penetration testing, an AI agent does much of that work. It decides what to try next, uses tools such as a browser, a terminal and security software, and changes course as it learns more about the target.
That last part is what sets it apart from older automation. The agent isn't working down a checklist.
AI-assisted, AI-augmented and autonomous: the differences
Vendors use these labels loosely, so here's how we use them.
- AI-assisted. A human tester runs the engagement and uses AI for specific jobs, such as summarising tool output or drafting parts of a report. The person makes the decisions.
- AI-augmented. People and AI share the testing. The AI might map the attack surface or chase several leads at once while the tester concentrates on the hardest problems.
- Autonomous. The agent plans and carries out several steps from a single task. You set the scope and permissions, follow the work and review the results.
Autonomous doesn't mean unsupervised. You still decide what's in scope.
How it differs from a vulnerability scan
A scanner usually runs a defined set of checks and reports anything that matches. That's useful for spotting known issues, but a match isn't the same as a weakness someone can actually use, and a scanner can't follow a chain of small issues to a serious one.
An AI agent can plan a sequence of tests, use several tools and investigate before it reports anything. If a sign-in form looks weak, it can try to prove it. If one weakness opens the way to another, it can follow that path. You should get fewer findings, each better supported, rather than a long list to triage. This difference matters a lot when customers send you security questionnaires, which we cover in working exploit vs scanner finding.
How does AI penetration testing work?
Most AI pentesting follows the same loop as a human test: agree what to test, investigate, prove what you find, then check the fix.
Scope and authorisation come first
Before any testing starts, you need written permission for every target. In the UK, testing a system without authorisation is a criminal offence under the Computer Misuse Act 1990, and that applies to AI-driven testing just as it does to any other kind.
A good scope lists the systems you're allowed to test, the environment, anything that's off limits and the test accounts the agent can use. API documentation and architecture notes help too. Clear scope keeps the effort on the things you care about.
Planning, tools and investigation
Once the scope is agreed, the agent plans its first steps, perhaps mapping what's exposed before moving on to features behind sign-in.
You don't have to wait for a final report. You can follow progress, supply context the agent is missing (a test account with admin rights, for example) and steer the work within the agreed scope.
Evidence: what makes a finding trustworthy
AI can make mistakes. It can misread a response, overstate a minor issue or report something that only looks like a vulnerability. So the key question for any AI pentest result is simple: how do I know this is real?
Two things help most. The first is a working exploit, which shows the weakness can actually be used, not just suspected. The second is an independent second pass, where the finding is checked again, separately, before it reaches you.
Human sign-off on findings is available when a client requests it.
A useful finding should also tell you:
- the affected asset, endpoint or feature
- the test account, permissions and prerequisites involved
- the steps to reproduce it and what was observed
- the impact that was actually demonstrated, a suggested fix and anything still uncertain
A severity label on its own explains very little. Separate what was shown to happen from what is only suspected before you decide what to fix first.
Fixing and retesting
When your team has made a fix, you ask for a retest against the changed system and compare the result with the original finding, including any access or coverage limits.
On BreachLine, retests use credits from your plan, so it's worth allowing for them when you plan your month. To see the full workflow from scope to retest, read about AI penetration testing with Nebula.
What can AI penetration testing find, and what can it miss?
Where AI testing is strong (breadth, repeatability, frequency)
- Breadth. An agent can work through a large number of endpoints and features without getting tired or skipping the dull parts.
- Repeatability. The same approach can run again after a release or a fix, so you're comparing like with like.
- Frequency. Because you don't need to book a tester's time, you can test more often. That matters if your code changes every week but your pen test happens once a year.
Where human testers still matter
- Complex business logic. An agent may notice that a discount can be applied twice. A person who knows your business is better placed to see why that matters.
- Unusual attack paths. Experienced testers notice when something feels off and try ideas no plan would suggest.
- Contract requirements. Some contracts ask for a test carried out by people. If yours does, AI testing alone won't meet that requirement.
Why "nothing found" isn't proof of security
A clean report tells you the agent didn't find a weakness in what it tested, with the access it had, in the time it spent. It doesn't tell you there's nothing there. A feature the agent couldn't reach, a test that didn't finish or a missing test account all leave gaps.
So read every report for what it covers as well as what it found: what was in scope, what was tested, what couldn't be reached and what's still unknown. That's true of human tests too. For a closer look at the gap between headline AI claims and real security work, read why pentesting still matters.
AI pentesting vs manual penetration testing: which do you need?
It depends on what you need the test for, and you don't always have to pick one. Here's how the two approaches usually compare.
Scroll sideways to compare.
| Topic | AI penetration testing | Manual penetration testing |
|---|---|---|
| How it runs | An AI agent plans and runs tests from a task, within your agreed scope. You can follow and steer it | Testers work through an agreed scope during a set test window |
| Proof | Varies by tool. Look for one that proves impact with working exploits | Testers validate findings and explain the impact. Quality depends on the team |
| Frequency | Can run whenever you need it, for example after each release | Usually a point-in-time test, booked in advance |
| Retests | Varies by provider. On BreachLine, retests use credits | Usually agreed and priced as part of the proposal |
| Cost model | Per test, subscription or credits | Usually a quote for each engagement |
When a scoped human engagement fits
A test led by people makes sense when you need depth on one critical system, when your business logic is complicated, when you want a red team exercise, or when a customer or contract asks for testing carried out by people. Read about scoped penetration testing services.
When ongoing AI testing fits
Ongoing AI testing suits teams that ship often and want to catch problems between formal tests. It's also a practical way for a small team without its own security engineer to test regularly.
What does AI penetration testing test?
Coverage always depends on the approved scope, the access you provide, the tools you connect and the time spent testing.
Web applications and APIs
Nebula tests whatever you authorise in scope.
This is where most AI pentesting starts. An agent can test sign-in, sessions and access control, and with a test account it can reach features behind sign-in, including user permissions and admin areas. APIs deserve particular attention, because every endpoint has its own permission checks to get right. For worked examples of the attack chains that matter here, see our research on JWT attack chains and SSRF to cloud takeover.
Cloud and networks
In the cloud, testing looks at exposed services, configuration and permissions. On networks, it looks at external exposure, identity controls and the paths an attacker could take between systems. Our Active Directory takeover walkthrough shows an AI security engineer moving from one low-privileged account to Domain Admin, step by step.
Mobile
When you scope a mobile test, include the APIs behind the app as well as the app itself.
AI systems and agents
AI assistants and agents are now part of your attack surface. Prompt injection, data exposure, retrieval permissions and unsafe tool use can all turn a helpful assistant into a way in. Our write-up on prompt injection as RCE explains how flaws in AI code editors became a route to remote code execution. If you're about to give an agent new skills or tools, use our AI agent security tools to review its instructions and capabilities first.
How much does AI penetration testing cost?
Prices vary between providers, so start with the three main pricing models.
- Per test. You pay for each engagement or asset. Easy to budget for a one-off, but costs add up if you test often.
- Subscription. A monthly or annual fee for ongoing access. Check what's included, as some plans limit assets, tests or retests.
- Credits. A plan includes an allowance that testing and retests draw from. Spend follows usage, so keep an eye on it.
Whichever model you're offered, ask how retests are charged and whether reports cost extra.
Here's how it works at BreachLine. Plans are priced in pounds, billed monthly and start at £399 a month. A small test covering one or two assets usually fits that plan. Retests use credits. Larger teams can choose the Team, Business or Enterprise plans, and you can compare Nebula plans side by side. Smaller UK firms can start from the automated penetration testing page.
How do you choose an AI pentest tool?
Plenty of vendors call themselves the best, so judge them on what you can check. Use this list with any provider, including us.
- Scope controls. Can you set targets, exclusions and test accounts precisely, and stop a test at any time?
- Proof of exploit. Does each finding come with a working exploit, or only a scanner-style match?
- False-positive handling. What happens to a finding before it reaches you? Is there an independent check?
- Evidence quality. Does each finding include reproduction steps, the test account used, the impact that was demonstrated and what's still uncertain?
- Data handling. Where is your data stored, how is it encrypted, is it kept apart from other customers' data and how long is it kept?
- Retest model. Are retests included, charged per test or paid for with credits? Is that clear before you sign up?
- Pricing transparency. Can you see prices without a sales call, and are they shown in your currency?
- UK presence. If you're a UK business, where is the provider based and does it work under UK GDPR? BreachLine is headquartered in London.
For a frank look at leaderboard claims and what actually works, read our AI pentest tool comparison.
Is AI penetration testing safe to run?
It can be, if it's set up properly.
- Authorised scope only. Test only systems you own or have written permission to test.
- Agreed exclusions. List anything that's off limits, such as third-party services you don't control or systems that can't tolerate testing.
- Start in staging. For your first run, test a staging environment rather than production, and use test accounts instead of real customer data.
You'll find more detail on our security and compliance page and in our Terms of Service.