How AI Finds Cybersecurity Vulnerabilities

Sometimes a question gets asked so much that it gets a blog post, even if that question isn't at the center of what we do here at Pythia Cyber. Lately, one such question is "how does AI find cybersecurity vulnerabilities?"

(We are also going to answer the underlying concern, which is usually "what can I do about this?")

Generative AI has significant pattern-recognition capability. This means that AI can find not merely simple matches, such as matching "golden apple" with "This is the tale of the Golden Apple" but also subtler, deeper matches, such as "golden apple" with "Greek myths."

Generative AI can only find patterns for which it has been trained. Once trained, that AI can only find patterns in data that has been fed into it. But after these two conditions have been met a decent AI is superhuman in its ability to find the patterns you taught it in the input that you give it.

Let us say that you spun up a copy of your favorite generative AI and called it Flawfinder. Then you trained Flawfinder on any of the many programming code bases available to demonstrate various kinds of mistake, for example the dreaded Buffer Overflow bug. Now you have an AI tool that knows how to find patterns of bad coding, misconfiguration or idiosyncratic flaws in various environments. This is great if you are using Flawfinder to make sure that your company's products are free of this kind of error. This is not great if you are a hacker looking to break into systems.

If you are a hacker then you turn Flawfinder loose on whatever source code you can find. Many versions of the popular server Operating Systems that derive from Unix (Linux and others) are Open Source, which means that their code is there for Flawfinder to review. This is on purpose: white hat hackers are supposed to do this kind of thing and the report their findings to keep bugs from becoming known vulnerabilities. But if you are a bad guy you keep your findings to yourself so you can use them for your own evil purposes.

Does this mean that you should avoid Open Source software and still to proprietary systems? Not necessarily. Flawfinder will likely find flaws in proprietary software as well because many products are based on or modeled after or just happen to work like their Open  Source counterparts. This is particularly true of drivers which are hard to write and easy to adapt from someone else's driver.

Note that Flawfinder, like all generative AI, is looking backwards at known issues with familiar and popular development environments. But alas huge chunks of the Internet were written in C decades ago and not many people are drawn to the unpaid drudgery of replacing this code with newer, more modern code. Especially if that old code is fast, functional and seems to have all its bugs removed.

I say "seems to have all its bugs removed" because most of us use software for its intended purpose in the expected way. Most of these vulnerabilities comes from edge cases and novel use. Sometimes deranged, deliberately destructive use. So a venerable piece of software can spend decades in quiet and error-free use because no one thought to enter specially-crafted malware as a password.

Does this mean that you should only use new, fresh software whose issues have not yet been gathered into sample code bases? Not necessarily. New, fresh software usually has new, fresh bad code, misconfiguration and idiosyncratic flaws. Newer is not always better.

So what should you do about the fact that generative AI is going to have a brief moment of malicious success reviewing lots of old-but-still-used code at superhuman speed for known kinds of weakness? You should keep up with warnings as they come out. You should warily apply patches and carefully accept updates to old systems that you thought were beyond all that. You should be aware but not panicked. This is a problem, but it is not a disaster. This is another example of having to row against the current in order just to stay still: they use AI to find issues and you use system administration to address those issues. And so it goes, on and on because a cyber defender's work is never done. Which doesn't mean that we don't keep doing that work.