Choosing a pen test provider

This article introduces the basics of pen testing and provides pointers on how to choose a suitable supplier to perform your testing. Whilst managing testing can be straightforward there are some challenges that it is useful to be both aware of and prepared for.

What is pen testing?

According to the NIST SP 800-53 pen testing is:

"A specialised type of assessment conducted on systems or individual system components to identify vulnerabilities that could be exploited by adversaries."

Pen testing goes beyond automated vulnerability scanning where testers assess, identify and validate vulnerabilities in systems. Testers will utilise automated tools to assess the in-scope systems but will also manually check any potential findings to ensure they are valid.

Following the completion of a test you will be provided with a report. The report will include:

• Executive summary (high level view) of the findings;
• Detailed breakdown of each finding including its severity rating;
• Recommendations for remediation.

Finding severity ratings (i.e. low, medium, high, critical) should be calculated using the Common Vulnerability Scoring System (CVSS). CVSS takes into consideration factors such as how easy it is to exploit and the likely impact of the vulnerability. Use of a standard rating system helps to ensure that ratings given are consistent and specific to the context of the system in your organisation.

You need to be aware that there different versions of CVSS. Its important to stick with one version across all methods you use for vulnerability assessment. This will help to ensure your vulnerability ratings are consistent and enable more effective prioritisation of remediation efforts.

Why is it an important control?

Systems and the infrastructure around them are subject to continual change. Through a combination of change and evolving threats you should expect to find vulnerabilities within your systems. Pen tests help you to find and remediate vulnerabilities within your systems before an adversary can exploit them.

To supplement pen testing you also need to consider security controls earlier in the change lifecycle as part of a strength in depth approach to security. If you can develop systems that are secure by design this will be far more cost effective than trying to retrospectively secure systems and remediate findings.

Examples of other supporting controls include:

• Secure code standards – define standards that set out secure coding requirements;
• Static code analysis – help your developers identify vulnerabilities early;
• Training – upskill the capability of your developers to design and develop secure systems;
• Vulnerability scanning – an automated tool to assess your systems for vulnerabilities;
• Web Application Firewall (WAF) – block common attempted exploits of your applications.

When should you run a pen test?

Pen testing is an expensive control to operate and requires a significant administration overhead. It is standard practice to perform pen tests on systems (at least those considered as business critical) on at least an annual basis or after material change. By material change this needs to be something that can affect the security posture of the system.

Example material changes may include:

• Adding or updating a security control – i.e. user authentication, encryption;
• Developing new products, services or features.

Whilst this is an important control it can be time consuming and expensive to operate especially where your organisation has adopted a Continuous Integration – Continuous Delivery lifecycle (CI-CD) (i.e. small and frequent deployments).

You’ll need to determine what constitutes a material change. For non-material changes look to implement automated tools that provide you with a suitable level of assurance that help to remove unnecessary friction in your company delivering change.

How can you identify suitable pen testing companies?

For those companies that don’t have the budget / specialism in house you’ll need to find a trusted third party. CREST are an international not-for-profit accreditation and certification body that recognise accreditations for organisations and professional certifications for individuals in security testing.

They provide a list of CREST accredited pen testing suppliers on their website. This is an ideal source of potential companies to perform your testing.

There are several factors you should consider in your selection process.

Stakeholder requirements

If you work with large clients, they are likely to require you to use a CREST accredited testing company. Some of your clients may even provide you with a short list of third parties that they require you to use. Make sure you understand stakeholder requirements to avoid having to duplicate any testing.

Your clients may even want to undertake or manage pen testing of your applications / systems. Be careful in this scenario as it will introduce a raft of challenges that should be avoided.

Technology

Pen testers will specialise in different technologies making them more proficient at testing certain types of applications / systems (i.e. web, mobile, cloud).

For highly specialised test requirements you will need to identify a company that has the specialist knowledge required to perform the testing effectively.

How many pen testing companies should you work with?

The answer very much depends on the number of applications / systems you have along with the requirements related to the type of testing. As with any type of supplier its useful to have a few options to choose from. I’d suggest having between two and three testing companies on an approved panel.

There are several factors you should consider.

Attrition rate

From experience there can be a high attrition rate (turnover) of testers. The pen testing companies are only as good as the capability of the testers they have working for them. Ensure the testers assigned to you are experienced and proficient at working with the required technologies.

Limited capacity

Pen testing companies are often small to medium sized businesses. They will have limited resourcing capacity to service the testing requirements of their clients. If you’ve got a lot of testing to be performed over a short time period you may struggle if you rely on one company.

Prioritisation

You will be in a pool of clients that the company provide services to. Some of these clients will commit to a significant amount of testing and may be given a higher priority over your testing needs.

Rotation

It’s useful to rotate use of pen companies as this allows you compare the effectiveness of the testing being performed.

Cost considerations

Day rates

Tests are charged according to the day rates of the testers. Given the manual and specialised nature of pen testing, engagements are expensive. Go to tender with several suppliers to understand the average day rate you’ll expect to pay.

Don’t just except the standard day rate offered to you. Always look to negotiate to see if you can achieve a more preferential rate. There are a couple of factors that will help to lower the rate:

• Purchase or commit to a volume of testing;
• Purchase on the potential of a future increase in business.

Cancellation penalties

Situations can arise where you need to cancel or rearrange testing engagements. Make sure you understand the cancellation notification timelines as well as the potential costs incurred. There are typically cut off times (i.e. 5 working days prior to the test) where you’ll incur the full or partial cost even if the test doesn’t go ahead.

If you do need cancel look to repurpose the days to a different test rather than incur cancellation charges.

Final thought

This article provides a high-level summary of factors to consider when choosing third parties to perform your pen testing. Selecting one or more capable third parties is vital to be able to run and maintain an effective security testing programme.