The Most Important Quality for a Penetration Tester

15 January 2016 – The recent news that two federal agents diverted several hundred thousand bitcoins to their personal accounts during the investigation of the Silk Road web site prompted me to consider what I think is the most important quality for a pen tester to possess–integrity.

A Google search for “integrity” produces a collection of links to dictionary sites and, at least here in Colorado Springs, a link to a local bank’s web site. But what does integrity have to do with pen testing? Plenty.

The most obvious reason why you must hire pen testers with integrity is the fact that they will likely be exposed to some of your organization’s most sensitive information and/or have administrator (super user) access to your most critical information systems. Do you doubt this? You should not. Modern operating systems, applications, and networks are extremely complex and users are often lazy. This combination, even in the best of organizations, allows for large security problems that can go undetected for a very long time. Pen testers find these problems and explore exposed data in order to communicate the business impact of the issue. In some cases, they may download or exfiltrate that data right out of your network to check your detection capabilities. Starting to get nervous? You should be. If your pen testers don’t possess the highest level of integrity, they may be tempted to work both sides of this cyber equation–collecting a pen testing fee from you and leveraging your most sensitive information and vulnerabilities for personal profit.

As scary as that sounds, there’s a more obvious issue. You pay pen testers to tell you whether your security protections are working as they’re currently configured. Although there are many pen testers that take great pride in smashing through your security and reporting as many problems as possible, there’s also the temptation to “soften the blow” in reporting pen test results and omit security issues that will be unpopular or difficult to explain (especially among more senior pen testers who understand the importance of recurring revenue and building relationships with their clients). Does your pen tester have the integrity to tell you things you don’t want to hear (but need to)? Will your pen tester report security issues that he himself can’t exploit, but are likely exploitable by more skilled testers (or hackers)? If after the final pen report is issued, would he contact you if he discovered he missed an important security finding?

I don’t have a foolproof litmus test for pen tester integrity, but here are a couple questions you might ask your prospective pen testers next time you’re shopping for a pen test:

  • Can you describe a recent situation where you felt a conflict of interest during or after a pen test and what you did to work through it?
  • Could you tell me about your procedures for protecting client vulnerability information during and after a pen test?
  • Do you have a process to get rid of pen test data (client authentication credentials, sensitive information, etc.) gathered during pen tests? How does that work across a team of testers? Has that process ever failed?

In my experience, junior pen testers and those lacking in integrity will gloss over these issues in answering the questions. Experienced pen testers (if they’re honest) will be able to talk at length about the difficulties these issues pose. The pen tester with integrity may not answer perfectly, but he should answer candidly. Great testing experience, polished communications, and competitive pricing are what usually sells pen tests. But for my security budget, I’d be shopping for pen testers with demonstrated integrity.

In case you want to read up on the theft of the bitcoins, here’s one of the many news articles about the case: