Zero is the most dangerous number in security automation. Zero observed errors, zero incidents since deployment, zero complaints from the team — each one sounds like success and none of them proves anything, because the arithmetic behind rare events hides failure with a smile. The honest case for or against AIOps for Security is built from other numbers, nine of which decide nearly every argument in the field. Collect them like a checklist, ask for them in demos, and watch how quickly vendors change the subject when one of them goes missing. Each entry below states the figure, where it comes from, and what it should change in your decision-making.

Nine numbers to ask of AIOps for Security

1. 0.9%. Run a high-volume estate with a detector that misfires once per ten thousand healthy events, and barely one flag in a hundred is genuine — precision around 0.9%, which works out to more than a hundred queue reviews for every real catch. This is what ordinary sloppiness costs when attacks are rare. Note the uncomfortable implication: the detector can be individually excellent by test-set standards and still produce a queue no team can read.
2. 47.4%. Squeeze the same detector until it misfires once per million and nearly half its alerts are real. Identical traffic, identical adversary, a completely different working day. Any pitch that hides the false-alarm rate is hiding the only number that matters at your prevalence, so make it the first thing you pin down and the first thing you test on your own data.
3. 200 and 30. The evidence floor for trusting recommendations: two hundred scored alerts minimum, thirty or more of them genuine positives, all labelled by somebody with no stake in the answer. Below that floor, a ranker is guessing with confidence. Above it, you can at least argue about its judgement with evidence on the table — which is the whole difference between a tool and a toy.
4. 2% and 150. The auto-close gate: misses demonstrated at no more than two percent across at least 150 true positives, with planted canary cases running, a blind holdout already scored, and reopening available in a single click. Untested closure is not automation; it is hoping with infrastructure attached. The one-click reopen matters more than it sounds, because it converts a wrong closure from an investigation into a keystroke.
5. 236, then 313. Sample sizes, if you want to prove that 2% ceiling properly. One observed miss in your evidence demands 236 samples; two misses push the requirement to 313. Small pilot studies with tidy results cannot clear this bar, which is exactly why the bar exists. Teams that internalise this stop being impressed by “thirty alerts, zero errors” — the arithmetic bounds the true miss rate near ten percent on numbers like those.
6. 25.2%. In ITBench’s compliance-and-security scenarios, presented at ICML 2025, autonomous agents resolved one scenario in four. Independent benchmarks exist precisely because vendor-selected test sets flatter their own products. A quarter solved is useful help and a genuine sign of progress; it is not a handover, and budgeting as if it were one is how teams discover the other three quarters.
7. 23-34% and 43-53%. CyberSOCEval, run by Meta with CrowdStrike in 2025, scored agents in those bands on malware analysis and threat-intel reasoning respectively. Coin-flipping baselines managed 0.63% and 1.7%, so the machines are genuinely above chance — and still nowhere near a closed ticket without review. The spread between the two task families is also a lesson: capability is task-specific, and one good number never covers the whole job description.
8. 42.3%. The category that taught everyone to refuse blended reporting: one deployment’s overall miss line read a calm 1%, while cloud-IAM misses inside it ran at 42.3% behind a 9.1% phishing figure. Averages bury the exact places your estate gets hurt, and the buried categories are rarely random — they are where training data was thin and attacker behaviour moved fastest.
9. 93.5%. The share of analyst toil measured as honestly removable in a published triage evaluation — the number that justifies the whole exercise. Not “percent automated”, which measures enthusiasm; verified workload removed at a known error cost. Pair it with that study’s other findings — 99.8% precision on closures, 86% on genuine-threat verdicts — and you have the complete picture automation vendors should be able to produce about their own systems.

Three objections you will hear, and the numbers that answer them

Expect pushback when the list lands, and prepare the counters. When a dashboard claims 99% accuracy, remember the security ML literature: near-perfect scores have repeatedly deflated once evaluation bias was removed, and near-perfect anything in this field is now read as a sign of evaluation trouble rather than excellence. When someone says there is no labelled data to evaluate with, point out that every deployment starts in the same hole and climbs out by exporting dispositions and running blind shadow scoring — labels are built, not born. And when the reply is that the tool already closes cases automatically, the answer is the 2026 market fact that products have shipped without an observe-only mode, which means the canaries and the per-category counts need to exist today, not after the evaluation the vendor never offered. Each objection dissolves into the same request: show the measurement, on your population, at this prevalence, per category, with the numbers dated and signed.

How to deploy the list in a meeting

Sequence matters as much as the figures themselves. Ask for prevalence first, because every other number depends on it and nothing can be interpreted before it is known. Then ask which action class each claim covers, since closure of phishing earns nothing for identity events and a register that maps every class to its own evidence is the mark of a serious deployment. Then ask who labelled the evidence and when it was last refreshed, because attacker behaviour drifts and stale labels certify a world that no longer exists. A vendor who answers all three in writing is rare and worth shortlisting; one who answers with a customer logo montage has answered, in its way, and saved you both some time.

What these AIOps for Security numbers add up to

Read together, the nine describe a coherent discipline rather than a product category. Rarity punishes imprecision, so prevalence comes first. Small samples hide large errors, so minimum counts and canaries carry the proof. Public benchmarks sit far below the brochure language, so independent verification is not optional. Blended metrics bury the failing categories, so reporting splits or it lies. Risk teams already fluent in this arithmetic — the same habits behind risk-managed delivery practices — will recognise the pattern from where the DevOps profession is heading: measurement moving earlier, authority gated on evidence, dashboards demoted below registers. The full reference, thresholds and worked examples included, is maintained in the Ops Atlas under the heading AIOps for Security, and learners mapping DevSecOps competencies and certification blueprints will find its evaluation chapters the most career-relevant reading in the field right now. Print the nine numbers. Carry them into every call. The one that gets dodged first is the one your estate cannot afford.