Cybersecurity Talent: Tests That Pass The Smell Test

We at Pythia Cybersecurity focus on cybersecurity talent. It's one of our core areas of expertise. In today's post we will discuss how you would test for cybersecurity talent in an individual contributor.

Cybersecurity talent is not about "hard skills" -- it's harder. 

Let's say you want to hire someone to work on your cybersecurity team. How would you go about developing an ad and then sorting through the candidate pool?

Your first stop is to understand what competencies you're hiring for. A good place to go is ONET, which is high-quality and freely available. (You could copy job ads from Big Tech Company Players but their situation may not be your situation.) ONET does not have job information for "cybersecurity" but it has related titles, such as "penetration tester," that seem reasonable. It suggests that required software credentials include those for development environment software, object or component oriented development software, operating system software, transaction security and virus protection software, and web platform development software. You can include all of these as part of your ad.

So far so good: hard skills are easy to specify, they have defined credentialing criteria, and the credentials are accepted industry-wide.

What about academic history? Do you only hire candidates from brand-name elite universities? That's a choice of course though you should be clear about why you make that choice. If the answer is vanity or that's what the other players do...well...that's probably not working out well for you.

OK so now you have candidates with desired certificates and academic history. How do you differentiate within that pool?

You have three remaining steps.

First, you can require that your candidates take a skills-based test. This is generally a test of coding or systems analysis. These tests can be purchased or maybe you can contract to develop them. The point is to have verification that the person's skills align with their resume. Instead of a skills-based test you could use a test of intelligence (see a recent article about why you would do that); while this would be better in a lot of cases, it is very sensible in this situation to test for specific coding skills.

Second you should have them take a "soft-skills" assessment. There are a lot of off-the-shelf tests you could use, many with four letter names or that have nine facets or -- not making this up -- involve choosing your spirit animal. Do not use those. Your realistic options include a test of the "big five factors of personality" (Big 5) or something specifically designed to measure cybersecurity work style, learning styles, motivations, teamwork ability, and interpersonal skills. (We at Pythia are empirically creating one of these and should be able to announce its availability soon.)

We do not recommend administering an off-the-shelf Big 5 test because it won't tell you what you want to know: how does this person "show up" at work, what motivates them, how do they work with others, and can you trust them.

All candidates who pass the phases so far -- certificates, academic history, technical test, and soft-skills assessment -- are then eligible for the final test: the interview. Note: you should have the interview be live and in-person because AI is being used by some candidates during interviews to game the system. (If you're going to have a remote workforce, then you have to make decisions.) Maybe an option is live videocall, but pay attention to odd candidate behavior. 

There are two basic types of interviews ranging along a continuum of more structured ("Talk about a time when...") to less structured. Use both. Have a panel do a structured interview with pre-determined questions and guidelines for scoring responses, then everyone who passes that gets a sit-down informal interview with the hiring manager.

What should you ask about in that structured interview? This is about experiences they may encounter at your company and also the way they conceptualize cybersecurity challenges and problem-solving. For example do they need to go to the manager, or do they prefer to solo to solve issues; do they focus on systems, or do they approach vulnerability assessments on a one-at-a-time basis; do they "rage to master" or do they decide it's just a job?

Note: context matters when interpreting interview responses. Saying "I'd go ask my manager" for a brand-new right out of college hire is probably good while the same response maybe would not be what you're looking for in a seasoned hire.

Not all qualified candidates are good hires. Quality of hire is not a given. Also, remember that if you focus solely on skills then you are hiring for a set of skills and that may end with you firing a flaming dumpster fire of personality problems.

Measure up. As us how you can measure cybersecurity talent accurately and reliably.