AI in Cybersecurity: Matching the Pump and the Pipes

AI & Plumbing

Systems Integration 

In my youth I worked as a plumber's helper for a short while. This experience taught me the difference between being smart and being experienced. I was very much smarter than my boss but my boss was very much better at plumbing.

Talent is a thing. Motivation is a thing. Experience is a thing. All of these things are important and all of these things come in varying degrees in the form of human beings. Or AI agents.

Back to my experience as a plumber's helper. On the plus side, not only was I being humbled about the relative value of expertise and intelligence, I also spent lots of time thinking about dynamic systems so this brief employment was quite helpful when I moved into distributed information systems and then into cybersecurity.

A key principle in either domain is that systems are interactive: their parts work together. This is little benefit in putting a giant V8 engine in your VW bug. There is likely great harm in raising the water pressure in your home by a few orders of magnitude. Yes, your toilet tank will fill with pleasing rapidity but only until it explodes under the unplanned-for load.

AI & Software Development

A Cautionary Tale

So it is with AI being applied to writing software: you can raise the rate of producing lines of codes by a few orders of magnitude. Yes, your feature set will swell with pleasing rapidity but only until it overwhelms your pipline under the unplanned-for load.

Software development is not simply a matter of generating lines of code. In many cases the process looks like this:

1. Conceive of a product
2. Gather requirements
3. Write specifications
4. Design the software architecture
5. Code to the design
6. Test the code
7. Document the behavior
8. Market the product
9. Sell the product
10. Support the product

If you combine steps 2 and 3 into "prompt an AI" and then leave steps 4 and 5 to the AI, you will experience a leap in productivity with respect to code generation. Your coding toilet will flush at shocking speed. But your testing, documenting, marketing and selling pipelines will buckle under the onslaught. And support will be a rather tricky proposition, probably ending up in a spiral of being beyond humans in scope but also beyond AI in capability. You will have the software equivalent of AI slop on the internet: bloated software that is a bit idiosyncratic and likely buggy to boot which changes at an annoying rate. Could you try to fix this by applying AI to every step after coding? Sure, but there are a couple of issues: (a) AI coding is a well-defined problem well-suited to AI and is relatively mature and (b) all that change all at once is just asking for trouble. 

Is it possible to apply AI sanely to augment the software development process? Absolutely. It is possible to do this in a way that works. Is it easy to do it badly? So easy. So very easy.

AI & Cybersecurity

I expect that Software Development's experience is highly predictive of what we can expect when as we rush to apply AI to cybersecurity. And worse, there is a big difference between the two domains: the consequences of not really understanding what the technology is doing are greater in cybersecurity than they are in, say, games for mobile phones.

Consider the NIST CSF model of cybersecurity: (let's ignore Governance for now; unless you are intrigued by the concept of AI CEOs there isn't good way to apply AI to Governance)

1. Identify your digital assets
2. Protect those assets
3. Detect attacks and problems with those assets
4. Respond to attacks and problems
5. Recover from attacks and problems

Can AI help with the Identify pillar? Sort of: it can analyze activity logs and give you accurate information about what your networks and servers are being asked to do. This will help support your assessment of your assets. But I doubt that the value judgments inherent in deciding what to protect (you can't protect everything all the time) would not be better if done by an AI.

Can AI help with the Protect pillar? Yes! You can do a due diligence scan for known vulnerabilities with confidence, except that the AIs tend to lag a bit behind the state of the art. You can even teach an AI about your particular set up and see if it can find weaknesses, but I would exercise great care in doing this because you are, in effect, creating a deadly virus so that you can create a cure. This is a fine idea until the virus gets out. Also remember that such an AI would be a vital digital asset and that protecting it would a high priority. This is non-trivial since it is the only digital asset capable of deciding to escape.

Can AI help with the Detect pillar? Yes! This kind of sophisticated pattern-matching is a real strength of this technology. Again, assuming proper training. This is the closest analogy to coding: this is an area that is pretty safe, pretty well-suited to the AI and maturing quickly.

Can AI help with the Respond pillar? Yes, but only if your responses are very well thought out and very well laid out. Otherwise the wrong things will happen at super-human speed. Remember the relentless bloody-mindedness of the machine. I was once part of a two man team creating an expert system to analyze health claims data to rate providers. Early version kept picking terrible doctors and terrible hospitals because if you kill your patients quickly through incompetence you save a bunch of money. In hindsight, this is an obvious mistake, not to weigh outcomes very highly. But "find the most cost-effective providers" did not, at first blush, seem like a fatally flawed command. Live and learn, but make sure that your experience doesn't come at too high a cost.

Can AI help with the Recover pillar? Probably, but recovery is a more nebulous concept than response. In 2026 asking an AI to make the judgments about what to recover and how much time and effort should be expended seems like a dereliction of duty. Perhaps in 2036 this will be standard.

To Marry AI to Cybersecurity Or Not?

There is no getting around the bottleneck of human attention unless and until you trust the AI to do the job at least as well was a cadre of talented, experienced humans would do the job. It not clear that we have enough expert human attention now: increasing the workload by a factor or 10 or 100 or 1,000 sounds like a terrible idea.

This is what goes through my head as the C-Suite clamors for ever-more automation, for faster turnaround times and better results with ever-shrinking overhead. Can we crank up the pressure in the hose? You bet! Will that make us safer and more cost-effective? Not without careful planning and oversight.

Figuring out how to balance AI's ability to do specific and boring tasks with AI's inability to actually be a person with responsibility and judgment is hard. Keeping track of where that line is on any given day is harder. We can help. Ask us how.