People Centric Security – reframing your perception

Making people a strong asset in the management of risk

During my time working in information security, it's not been unusual to hear comments like "people are the weakest link in security". I suspect, in part this is caused by the knowledge and experience of those working in the industry often coming from a background in technology. This can lead to an unconscious bias towards technology and process over people. It's true that people make mistakes but in the same sense technology controls can be poorly designed / configured and badly operated.

People represent one of the core pillars in running an effective security programme along with process and technology. Rather than viewing them as the weakest link, change your perception to consider how they can become your strongest asset.

In this article I introduce topics on security behaviour and psychological safety. I then provide some suggestions of what you can do improve attitudes, perceptions and engagement with the security team.

It can be hard to know where to even get started improving people related security. Companies typically start by providing compliance focused awareness initiatives to meet the needs of regulation or standards. If you want to make a genuine impact on improving people security you need to focus on behaviour / culture change and the delivery of secure behaviours.

You don’t need to be an expert in behavioural psychology to get started!

Secure Behaviour

A good security behaviour is a combination of walking the walk and talking the talk. Saying and doing. To change people's behaviour, you need to consider the factors that are influential in its delivery. The following three factors are from the COM-B model. Consider what you can do to improve each of these factors in your organisation.

Capability (need to know)

Do staff have the knowledge, skills and abilities required to engage in a particular behaviour?

They can't just be expected to know what you want of them. Invest in your staff so that they can develop their capabilities (i.e. report incidents, identify phishing emails) to act securely.

Motivation (need to feel)

Do staff have a reason to act in a certain way? Do they perceive a benefit in performing a behaviour overriding the competing behaviour to not do it? What is their attitude towards a particular topic?

If staff have a poor attitude to security this increases the likelihood that they won't perform them. Consider a situation where an individual perceives a security control as adding little to no value. In this scenario the individual has low motivation and is unlikely to follow the required security behaviour. This may lead them into circumventing it and perhaps even encouraging others to do so.

Opportunity (need to have)

Do external factors make the execution of a particular behaviour possible?

Communication

How are you communicating the behaviour? What scope of your target audience are you reaching and are they engaging with your communications?

Usability

How easy is it to do the right thing? Do what you can to reduce friction and make it easy for staff to perform in the right way. Removal of friction increases the likelihood of staff behaving securely.

Leadership

Do senior leaders' impact, support and lead by example? Its difficult to deliver a culture of security without having senior level support. If senior leaders undermine security initiatives or don't follow security behaviours this will influence a poor culture of security.

Psychological safety

Most staff go to work to be effective in their role. Security by its very nature can add friction into business processes making it more difficult for staff to achieve their (non-security) objectives. Information Security doesn’t exist in a silo and is there to enable the organisation to operate effectively whilst balancing that against the management of security risk.

For your security programme to be effective you want to develop a psychologically safe environment where people are comfortable in expressing and being themselves. Most people want to look smart, capable and helpful. In contrast they don’t want to look ignorant, incompetent or disruptive. This fear over perception can lead to people taking the safe option of staying quiet rather than raising their thoughts and opinions.

Making it safe

There are different approaches that you can take that will either encourage engagement or lead to avoidance of the security team.

General engagements

It is the role of the team to engage with a wide range of stakeholders across the organisation. The way you manage these engagements will impact staff attitudes and perception of security which in turn influences staff motivation (positively or negatively) to behave securely.

It is important to note that senior staff can often feel more safe engaging with others. In contrast those in lower status roles can feel less safe especially when dealing with more senior staff. Make sure you treat all staff with the same level of trust and respect regardless of their status in the organisation.

The following are some suggestions on what you can do. These may seem obvious but in my experience they are often lacking.

Suggestion	Comment
Develop a culture of listening and actively seek feedback.	Understand people's attitude / perception of security. This will help you to understand what does / doesn’t work and adapt.
Respond in an appreciative, respectful and productive manner.
Invite participation and a sharing of knowledge.	Be open to people raising concerns, questions, mistakes and ideas.
Provide constructive feedback.	Ensure you avoid being critical of the individual.
Be willing to accept when you are wrong.	This provides you with a valuable opportunity to learn / develop.
Be open to discussion / debate.
Treat staff with trust and respect.	Avoid embarrassing or belittling staff.
Act on constructive feedback.	Consider feedback and act on the information provided to you. Failure to take or be seen to take any action can lead to people feeling a sense of futility in reporting.

Incidents / breaches

The information security team need staff to be willing to report incidents and breaches. Its only possible to contain / minimise the impact of those that you are aware of.

In an incident / breach scenario there will be an increased level of fear from the individual reporting. This is due to the fact that what they are reporting often relates to personal mistakes. In this situation people can perceive it to be safer to cover up the situation rather than admit to any level of incompetence.

In a situation of heightened stress it can be easy to jump to conclusions and attribute blame to others. We each frame situations based on our own knowledge and experience. Take the time to reframe and understand situations from the perspective of the reporter rather than being quick to base it on your own assumptions.

From experience the key change you need to make relates to blameless reporting. Shift away from the belief that incompetence was to blame for an incident or breach. This will help to address staff fear of reporting.

Summary

Focus on making people a strong asset in the management of security risk rather than perceiving them as a weakness. Target your security programme at delivering secure behaviour and endeavour to make people feel psychologically safe when engaging with the security team.

A combination of these approaches will help to transform people security and develop a security oriented culture within your organistion.

For further information on delivering behaviour focussed security take a look at a previously posted article.

Choosing a pen test provider

This article introduces the basics of pen testing and provides pointers on how to choose a suitable supplier to perform your testing. Whilst managing testing can be straightforward there are some challenges that it is useful to be both aware of and prepared for.

What is pen testing?

According to the NIST SP 800-53 pen testing is:

"A specialised type of assessment conducted on systems or individual system components to identify vulnerabilities that could be exploited by adversaries."

Pen testing goes beyond automated vulnerability scanning where testers assess, identify and validate vulnerabilities in systems. Testers will utilise automated tools to assess the in-scope systems but will also manually check any potential findings to ensure they are valid.

Following the completion of a test you will be provided with a report. The report will include:

• Executive summary (high level view) of the findings;
• Detailed breakdown of each finding including its severity rating;
• Recommendations for remediation.

Finding severity ratings (i.e. low, medium, high, critical) should be calculated using the Common Vulnerability Scoring System (CVSS). CVSS takes into consideration factors such as how easy it is to exploit and the likely impact of the vulnerability. Use of a standard rating system helps to ensure that ratings given are consistent and specific to the context of the system in your organisation.

You need to be aware that there different versions of CVSS. Its important to stick with one version across all methods you use for vulnerability assessment. This will help to ensure your vulnerability ratings are consistent and enable more effective prioritisation of remediation efforts.

Why is it an important control?

Systems and the infrastructure around them are subject to continual change. Through a combination of change and evolving threats you should expect to find vulnerabilities within your systems. Pen tests help you to find and remediate vulnerabilities within your systems before an adversary can exploit them.

To supplement pen testing you also need to consider security controls earlier in the change lifecycle as part of a strength in depth approach to security. If you can develop systems that are secure by design this will be far more cost effective than trying to retrospectively secure systems and remediate findings.

Examples of other supporting controls include:

• Secure code standards – define standards that set out secure coding requirements;
• Static code analysis – help your developers identify vulnerabilities early;
• Training – upskill the capability of your developers to design and develop secure systems;
• Vulnerability scanning – an automated tool to assess your systems for vulnerabilities;
• Web Application Firewall (WAF) – block common attempted exploits of your applications.

When should you run a pen test?

Pen testing is an expensive control to operate and requires a significant administration overhead. It is standard practice to perform pen tests on systems (at least those considered as business critical) on at least an annual basis or after material change. By material change this needs to be something that can affect the security posture of the system.

Example material changes may include:

• Adding or updating a security control – i.e. user authentication, encryption;
• Developing new products, services or features.

Whilst this is an important control it can be time consuming and expensive to operate especially where your organisation has adopted a Continuous Integration – Continuous Delivery lifecycle (CI-CD) (i.e. small and frequent deployments).

You’ll need to determine what constitutes a material change. For non-material changes look to implement automated tools that provide you with a suitable level of assurance that help to remove unnecessary friction in your company delivering change.

How can you identify suitable pen testing companies?

For those companies that don’t have the budget / specialism in house you’ll need to find a trusted third party. CREST are an international not-for-profit accreditation and certification body that recognise accreditations for organisations and professional certifications for individuals in security testing.

They provide a list of CREST accredited pen testing suppliers on their website. This is an ideal source of potential companies to perform your testing.

There are several factors you should consider in your selection process.

Stakeholder requirements

If you work with large clients, they are likely to require you to use a CREST accredited testing company. Some of your clients may even provide you with a short list of third parties that they require you to use. Make sure you understand stakeholder requirements to avoid having to duplicate any testing.

Your clients may even want to undertake or manage pen testing of your applications / systems. Be careful in this scenario as it will introduce a raft of challenges that should be avoided.

Technology

Pen testers will specialise in different technologies making them more proficient at testing certain types of applications / systems (i.e. web, mobile, cloud).

For highly specialised test requirements you will need to identify a company that has the specialist knowledge required to perform the testing effectively.

How many pen testing companies should you work with?

The answer very much depends on the number of applications / systems you have along with the requirements related to the type of testing. As with any type of supplier its useful to have a few options to choose from. I’d suggest having between two and three testing companies on an approved panel.

There are several factors you should consider.

Attrition rate

From experience there can be a high attrition rate (turnover) of testers. The pen testing companies are only as good as the capability of the testers they have working for them. Ensure the testers assigned to you are experienced and proficient at working with the required technologies.

Limited capacity

Pen testing companies are often small to medium sized businesses. They will have limited resourcing capacity to service the testing requirements of their clients. If you’ve got a lot of testing to be performed over a short time period you may struggle if you rely on one company.

Prioritisation

You will be in a pool of clients that the company provide services to. Some of these clients will commit to a significant amount of testing and may be given a higher priority over your testing needs.

Rotation

It’s useful to rotate use of pen companies as this allows you compare the effectiveness of the testing being performed.

Cost considerations

Day rates

Tests are charged according to the day rates of the testers. Given the manual and specialised nature of pen testing, engagements are expensive. Go to tender with several suppliers to understand the average day rate you’ll expect to pay.

Don’t just except the standard day rate offered to you. Always look to negotiate to see if you can achieve a more preferential rate. There are a couple of factors that will help to lower the rate:

• Purchase or commit to a volume of testing;
• Purchase on the potential of a future increase in business.

Cancellation penalties

Situations can arise where you need to cancel or rearrange testing engagements. Make sure you understand the cancellation notification timelines as well as the potential costs incurred. There are typically cut off times (i.e. 5 working days prior to the test) where you’ll incur the full or partial cost even if the test doesn’t go ahead.

If you do need cancel look to repurpose the days to a different test rather than incur cancellation charges.

Final thought

This article provides a high-level summary of factors to consider when choosing third parties to perform your pen testing. Selecting one or more capable third parties is vital to be able to run and maintain an effective security testing programme.

Information Security - management reporting

As with any department within a sizeable organisation you need to produce reporting to communicate information to the board and senior management. You need to be structured and intentional with the reporting you produce to ensure it is delivering the right outcomes for your department as well as the wider organisation.

Reporting is an important tool that is required in enabling the delivery of a successful Information Security programme. This article provides some guidance on how you can be more successful in the delivery of your own reporting.

Purpose of reporting

Management reports provide a means of communicating information upwards. When producing these reports make sure you are clear about the objectives you are working towards to achieve your goals and are tailoring to the needs of your audience.

Focus on risk / reward and the outcomes for your organisation. This helps aid the understanding of the intended audience and will prompt / enable decision making to stimulate action where it is required. Your audience are the decision makers in the organisation and can positively or negatively influence in the delivery of your programme.

Structure your report

The following sections detail the content that you should consider including within your reporting.

Executive summary

Executives / senior management will receive a multitude of reports / communications. They are typically time short and want to know quickly if there is anything they need to be concerned with. Make sure your summary highlights any increasing risk exposures especially where they require decision making / action from the reader. Be aware that this may be the only portion of your report that they read.

Overview of security controls

The Information Security team are delegated the responsibility for operating security controls to enable the management of business risk. Whilst the team operate the controls the accountability for the risk is with the risk owner who delegates out the operation of those controls.

In larger organisations security controls are likely operated on behalf of many risk owners who are accountable for their department or entity. From a regulators perspective the regulated entity is accountable for their risk even if they have outsourced controls to their parent organisation.

Risk ownership will be with the board or senior management who have sufficient influence in the organisation to be able to manage that risk effectively. This report provides you with an opportunity to inform the risk owners how their controls are performing (through measurement / trending) and provide them with sufficient information to take decisions and drive action where it is required.

What to include	Why should it be included?	Example/s
Control Scope (KCI)	Be clear about what is / isn’t included in the scope of your security controls. If the reader isn’t comfortable that the scope is sufficient this will help to justify increased investment. It doesn’t matter how effective your security controls are if they only cover 5% of the overall scope!	We provide annual security assurance for 30% of our high security suppliers / vendors.
Control Effectiveness (KCI)	New threats emerge and existing ones evolve. This changing threat landscape will require you to adapt or replace your controls to meet with the latest threats faced by your organisation. The quality of your existing controls may reduce over time. Failure to adequately resource them or invest in their development may lead to them not being fit for purpose. Call out where controls are no longer adequate for effective management of risk. The controls themselves may still be effective to address the original threat but less effective against new or changed threats.	Our email security tools block 70% of malicious traffic. This has reduced from 90% in the previous quarter. We are seeing a growing threat in malicious applications targeted at our organisation. There have been 5 instances in the last 3 months. We lack an effective control to mitigate this growing threat.
Control Performance (KPI)	Make sure you include details relating to where your KPIs are failing to meet the agreed minimum-security requirements. You will need to correlate changes in your KPIs with the actual risk to the business.	10% of critical vulnerabilities are not addressed within the standard defined timeline. 15% of staff click on phishing emails. 5% of staff fail to complete their security training.
Security Risk (KRI)	Security controls exist to enable organisations to manage business risk within a set risk appetite or at least within a defined risk tolerance. Failing to manage risk within the overall risk capacity has the potential to threaten the viability of the organisation. Your objective is to identify increasing risk exposures to enable effective management of the risk.	We have experienced a 10% increase in security incidents. There has been a 10% increase in data breaches. We have experienced a 15% increase in spear phishing attacks leading to a 5% increase in malware incidents.

Important to note

An effective control is one that enables management of risk within risk appetite or risk tolerance. This means a partially effective control can be seen as adequate where it is enabling the effective management of the risk even if this isn’t ideal from the perspective of the control operator.

Security events / incidents

Provide an explanation of significant security events and incidents. Incidents represent realised risks and can be a good indicator of new threats and risk trends. These can be internal or external to the organisation such as:

• Compromise of a supplier network
• Incident experienced at another organisation
• Vulnerabilities receiving significant media attention

Where its internal make sure to detail what happened and what has / is being done to manage the incident. Where its external detail the mitigations that exist within the organisation or highlight the need for control improvements to address this new threat.

An incident, even one experienced by a third party can be an opportune time to get the buy in you need to deliver your initiatives!

Security programme / initiatives

Provide an overview of progress towards your objectives or highlight where the team are supporting in the delivery of wider business objectives. This is an opportunity to offer assurance that the Information Security department is adding value and meeting the needs of the organisation.

Where you are experiencing challenges / problems call these out and highlight the actions you are taking as well as detailing the actions required by the reader.

Your programme needs to be forward looking (working towards a desired state) and not just focussed on fighting fires.

Recommended actions

This is an opportunity to detail the actions that are required to deal with incidents, respond to emerging threats, correct any decline in control effectiveness / performance and respond to increasing risk exposure.

The information you provide needs to aid the understanding of the intended audience. This needs to prompt / enable decision making to stimulate action where it is required.

Important factors to consider

These are some key factors that you need to consider when producing your reports:

• Understand your audience – whilst this may be targeted at the board consider which other stakeholders (such as your regulators) may have visibility of them
• Make sure you communicate to the right stakeholders
• Provide clear and concise content that is easy for the intended audience to understand
• Produce reporting at consistent / set intervals – this is typically produced monthly or quarterly
• Avoid noise / padding as this just detracts from what is important
• Report on information that is important to achieving your objectives / goals
• Supply the right information to enable required decision making
• Make sure to highlight and recommend required actions that will lead to the required outcomes

Internal benchmarking

Whilst is can be difficult to benchmark against external companies it is possible to do a direct comparison of security indicators across your internal departments or entities. Consider use of gamification within your reporting through the introduction of game elements such as use of points and leader boards / tables.

Gamifying the reporting makes it easy to do internal direct comparisons and provides a level of competition that can help to drive improvements to your overall security posture. Make sure the metrics / indicators you include support in the delivery of your objectives. Avoid focussing resourcing in the wrong areas.

Looking beyond your organisation

Where the information is available consider benchmarking against other companies. This can provide a good measure of the capability / maturity of the organisation’s security programme. Benchmarks can be sourced from:

• Security rating platforms
• Security health check assessments
• Security metrics sourced from your controls

I hope this article proves helpful in making your reporting more effective!

Cyber Security - professional training options

Throughout my time working in Cyber Security, I have attended a variety of different security training courses that have helped me to successfully attain a selection of professional qualifications. There are four main training options available to you. Before committing to one you’ll want to consider what options are available and assess the suitability of each to your needs.

In the following article I have assessed each of the training options and rated them according to cost, speed and convenience factors. These have been ordered with the lowest cost options first.

If you are unsure if security qualifications justify the commitment I suggest reading a supporting article I produced covering this topic.

Self-study

In 2017 I successfully passed the Cloud Security Alliance CCSK qualification and more recently the ITIL Foundation exam in 2020. This option is well worth considering but despite the low cost may not be suitable for everyone.

	Cost	Speed	Convenience
Rating	Low	Slow	High
Description	This just includes the cost of the study materials.	This is driven by your self-motivation and the amount of time you can commit to study. From personal experience it has taken up to 6 months to prepare for an exam through self-study.	You get to take the training at times that suit you rather than committing to set dates / times.

Suitability

This is ideal if you:

• Are looking for a low-cost option
• Are able / willing to commit the time and effort required
• Can balance the training around your other commitments
• Can self learn without support / guidance from an instructor
• Don't have time pressures to achieve a qualification quickly

Assessment

This is by far the most cost-effective option but its highly dependent on your ability to motivate yourself and commit to undertake the study.

If you're struggling for motivation I suggest booking the exam in advance but give yourself enough time to complete your preparation. This gives you a timeline to work towards and the deadline acts as a great motivating factor!

Whilst you don't have support from an instructor there is likely to be wealth of materials available to you and online communities that provide a level of support / guidance.

On-demand training

I have attended several on demand training courses through SANS including web application security, ethical hacking and incident management. Of the three courses I only took an exam in web application security and passed this in 2012. I had an incredibly positive experience in the use of the SANS platform.

As an alternative to specialist training providers like SANS I have experience working with learning platforms such as those provided by Pluralsight and Percipio. They cover a broad range of training topics at a far lower cost. These type of platforms are ideal for supplementing your self study but be aware that the quality of training offered can be highly variable.

	Cost	Speed	Convenience
Rating	Low to Medium	Medium	High
Description	The cost of this option will vary according to the vendor you attend the training with. Dedicated training such as that provided by SANS is comparable in cost to instructor led training. Learning platforms come at a much lower cost but should be considered as a supplement rather than replacement for self study.	You can do the courses at your own set speed. This is likely to be at a slower pace than instructor led training especially if you are having to balance multiple commitments.	This is the main selling point of this option. You get to take the training at times that suit you rather than committing to set dates / times. Learning platforms provide you with access to a range of courses for an ongoing monthly fee. Specific courses you sign up to will give you access to the on-demand training materials for a set duration (i.e. 3 months) with the potential to extend access at a cost.

Suitability

This is ideal if you:

• Are able / willing to commit the time and effort required
• Can balance the training around your other commitments
• Can self learn with minimal support / guidance
• Don't have time pressures to achieve a qualification quickly

Assessment

This is an ideal alternative to self-study as the training is far more engaging. There is a significant difference between booking a particular course on demand and subscribing to a learning platform. If you book a particular course the costs can be comparable to instructor led training.

If you opt to use a Learning Platform I would advise using this as a supplement to rather than replacement of self study.

Instructor led (in person or virtual) training

I have had the opportunity to attend a selection of in person and virtual training courses. Most recently I attended ISACA hosted training for CRISC in 2020 and went onto successfully pass the exam.

	Cost	Speed	Convenience
Rating	Medium	Medium	Medium
Description	The cost of this option will vary according to the vendor you attend the training with and whether its in person or virtual. Even at the top end cost this is likely to be at a lower cost to the bootcamp equivalent.	The courses are less intensive than doing through a bootcamp equivalent. From experience they tend to span a typical working day (i.e 9 – 5). For the major cyber security qualifications expect to do significant self-study to supplement what you learn in the training sessions.	The shift to remote / virtual training has improved the overall convenience of attending this type of training although you do lose the additional benefits of learning in person in a class room setting.

Suitability

This is ideal if you:

• Want to supplement your own self study with instructor led training
• Need support with understanding / learning the material
• Can commit to attending fixed time / date sessions
• Struggle to motivate yourself through personal study

Assessment

Instructor led training will help you to understand the more complicated topics and will support you in preparing for the exam.

You should consider this as a supplement to self-study. From experience I have had to commit to far more personal study prior to the exam than was required after attending bootcamp training.

Bootcamp

I have only had the opportunity to attend one training bootcamp. This was back in 2016 when I was preparing for the ISACA CISM exam. The course was held by Firebrand. Overall, I had a positive experience and went onto successfully pass the exam a week after finishing the course.

	Cost	Speed	Convenience
Rating	High	Fast	Low
Description	Given the intensive nature of the courses you should expect to have to cover the cost of food and accommodation as well as the course fee. This is the most expensive option.	Bootcamps provide an intensive experience that forces you focus and study in preparation for the exam. This is by far the quickest option. It would be difficult to personally motivate yourself to emulate this through self-study. Well, certainly from my own experience!	The likes of CISM and CISSP have respective bootcamps spanning between 4 and 6 days. These courses require you to dedicate your time across long days with the addition of self study in the evening.

Suitability

This is ideal if you:

• Can cover the higher expense, or this is being picked up by your company
• Need to achieve the certification quickly
• Can commit to dedicating up to 6 days intensive study
• Struggle to motivate yourself through personal study

Assessment

This is a great option to pass an exam quickly but comes at significant cost. It also requires a high level of your commitment over a period of up to a couple of weeks.

Even though these courses will cover all the topics within the exam I would still advise you to do some study in advance of the course as this will help you to maximise the value of your training.

Final thoughts

The below table summarises the training options and shows you the trade offs between the cost, speed and convenience factors.

	Cost	Speed	Convenience
Self-study	Low	Slow	High
On-demand	Low to medium	Medium	High
Instructor led	Medium	Medium	Medium
Bootcamp	High	Fast	Low

In reality I have always taken a hybrid approach combining multiple training options rather than any one in isolation. The key is in finding an approach that works for you whilst balancing each of the factors to fit your needs.

Some of the training options will include the cost of the exam. The exam cost alone can be considerable.

Its important to note that the quality of study materials, training courses and instructors can vary considerably even within one particular vendor. I advise doing some research on a given vendor and ideally speaking to others in the Cyber Security community to get an idea of who's good and who should be avoided.

How to identify people related phishing vulnerabilities

Phishing is a significant threat to organisations and remains a common vector that threat actors used to compromise organisations. Whilst traditional email defences will block most malicious emails from reaching your employees there will always remain a portion that will get through. This is where the security capability of your employees is key in the detection and reporting of phishing-based threats.

In this article I’m focusing on how to gain visibility of people related phishing vulnerabilities to support in increasing the security capabilities of your own employees. Whilst technical controls remain important the people related aspects are often overlooked and under resourced. This is not surprising with many referring to people as being the weakest link in company security. A paradigm shift to seeing them as a significant asset in your defence in depth approach to security will deliver significant value and increase the effectiveness of both your technical and process related controls.

What do you want to achieve?

Your goal is to reduce the risk phishing poses to your organisation. Whilst you will never eliminate the risk, you can take significant steps towards achieving your goal through the delivery of the below objectives:

• Building visibility of people related vulnerabilities;
• Increasing the capability of staff to spot phishing scams;
• Increasing the willingness of staff to report phishing.

The following sections look at each of these objectives and describe what actions you can take to achieve them.

Building visibility of people related vulnerabilities

Undertake phishing testing against all or targeted individuals / groups at frequent intervals. Whilst they need to be operated at set intervals make sure these aren’t done too frequently (i.e. more than once every 6 weeks to the same individual) and check the timings aren’t predictable.

You will need to vary the lures, difficulty and types of phishing (link, attachment, credential harvesting) to identify which employees are susceptible to certain types of threat. Prioritise testing according to the genuine threats’ employees are proving vulnerable to.

Increasing the capability of staff to spot phishing scams

As you increasingly identify people related vulnerabilities, you will need to deliver bespoke / targeted training and awareness to help increase staff capabilities. Whilst bulk training may help to improve general capabilities around basic phishing threats, it will not help your staff to identify the more sophisticated threats that are being specifically targeted at individuals in your organisation.

Everyone is susceptible to phishing threats but at varying degrees of difficulty and lures. You need to identify these and specifically focus on addressing the needs of individuals.

Increasing the willingness of staff to report phishing

Build a culture of security where employees know the importance of their role in keeping the organisation safe. You want employees to report phishing emails quickly to give the security team the opportunity mitigate the threat before a wider audience has the opportunity to be compromised by it.

Building visibility

Visibility of people related phishing vulnerabilities can be achieved through a combination of operating phishing testing and through the analysis of genuine threats. This will provide a great insight into the types of emails individual employees are vulnerability to.

When running phishing testing against your employees you will find it challenge to understand individual vulnerabilities when the main measures you have to work with are:

• Click rate - based on links;
• Compromised rate - based on staff giving away sensitive information (data harvesting) or clicking on suspicious attachments;
• Average Failure Rate – benchmarked failure rate across different organisations.

Whilst these are useful indicators in trending progress at an aggregated (high) level, they are not particularly suited to explaining vulnerabilities at a granular level.

NIST Phish Scale

In 2020 NIST published a research article introducing a means of categorising human phishing difficulty using a method called Phish Scale. The method uses a scoring mechanism to calculate the difficulty according to the number of cues visible in the email combined with the premise (applicability, alignment or relevancy) to the organisation. Premise considers the threat within the context of both inside and outside of the organisation.

One of the key failings of using a benchmark figure to compare organisations is that the premise (context) rating of the phishing threats will vary across organisations. For instance, a phishing email themed on a technology has a far greater likelihood of being effective if a given organisation is using that technology. So what is a difficult threat in one organisation may lack relevance and be perceived as easy within another.

As with any research paper the challenge is to take it from an academic concept and apply it to provide beneficial outcomes in a real-world scenario. The method (unlike many others) can be fairly easily translated into a workable assessment tool even if this is just via a Spreadsheet.

Whilst the fundamentals behind the method are great there are still opportunities for refinement.

Phish Scale - potential improvements

I have personally supplemented the existing cues to include unfamiliar tone, overly vague and unusual request as well as updating cue names, descriptions and criteria to make them easier to understand / apply.

The common tactic section of cues suffers from being very specific. In this form, for the list to be effective it would need to be actively developed and would require ongoing maintenance to keep it relevant with changing tactics.

The list can be more effectively represented through utilisation of the 6 types of social power. Each of the existing tactics can be matched to at least one social power. I recommend reading the linked article to understand more about these. They are useful in building an understanding of the techniques used by threat actors to persuade people to undertake their desired actions.

The premise calculations provide a good articulation of context but it is valuable to detail the lure/s (something that tempts or is used to tempt) used in the email as these are important factors in understanding why certain individuals are proving susceptible to it.

Summary

The NIST Phish Scale method has helped me to fill a gap in understanding people related vulnerabilities. It can be applied to both test phishing campaign emails as well as genuine phishing emails. Through combining your existing indicators with those within the Phish Scale method you can help to build actionable intelligence that can enhance the security capabilities of staff in your organisation.

Creating a good security culture

This article introduces the concept of a security culture and provides guidance on how you can positively influence and evolve the culture in your organisation.

According to ENISA “Cybersecurity Culture refers to the knowledge, beliefs, perceptions, attitudes, assumptions, norms and values of people regarding cybersecurity and how they manifest in peoples behaviour”.

Every organisation has a security culture whether this is considered to be good or bad. Security culture is intertwined with organisational cultures and is shaped by messaging across all-levels of the organisation.

Large organisations are unlikely to have one culture. This will often differ across the organisation within different functions, departments, entities, business lines, offices, and jurisdictions. This cultural variation creates a challenge when seeking to communicate security messages and drive behavioural change as the message will be subject to differing interpretations.

Having a good security culture is fundamental in helping you to deliver an effective security programme within your organisation. This is key to building a good security foundation and will increase the probability that your initiatives deliver positive outcomes for the organisation.

Culture takes time and a concerted effort to evolve and must be formed with staff rather than being imposed upon them.

What does a good security culture look like?

A healthy or positive culture will actively contribute to support and enable the business to achieve its goals and objectives. To achieve this, you need staff to:

• Remember the things they are supposed to do for security and do them at the right times and in the right circumstances;
• Prioritise doing things in secure ways;
• Know how to report any concerns or suspicious activity and feel empowered to do so;
• Question processes in a constructive manner;
• Contribute to shape security policy;
• Understand the importance of cybersecurity measures and what they mean for the organisation;
• Understand risk associated with their day-to-day activities;
• Know and be confident in the mitigation and handling of risk.

What does a bad security culture look like?

A poor or negative culture will undermine efforts to manage security risk and has the potential to hinder the operation of the business. A negative security culture can result in:

• Staff not reporting any concerns or suspicious activity through the fear of blame or reprisal;
• Staff bypassing security tasks, taking unnecessary or high levels of risk or seeking to cut corners;
• Staff being cynical about security often due to a lack of influence to deliver the outcomes for which they are accountable;
• Staff only seek to do just enough;
• Company leaders not following the rules or seeking exceptions or special treatment;
• Staff not engaging with the security team or not contributing to security initiatives / programmes;
• Security team members feeling undervalued and separated (isolated) from the rest of the business;
• High attrition rate of security staff.

What blockers exist to developing a good security culture?

There are a multitude of factors that will hinder delivering a good security culture. These factors need to be addressed to support building the foundations for delivering a good culture of security:

• Security budgets are not keeping pace with the rising threat level;
• Staff do not feel their contributions are considered or valued;
• Staff lack the knowledge or confidence to do the right thing;
• Security is too insular and focused in a silo (such as technology) creating a boundary between security and the wider organisation;
• Rules make it hard for people to do their jobs encouraging staff to find workarounds or unofficial ways of working;
• Security seen as a blocker rather than an enabler;
• Leaders fail to lead by example;
• Operating a blame culture.

How can you influence security culture?

It is important to note that in general people want to do the right thing. There are often influencing factors involved that can inhibit staff following the desired security behaviours. Through understanding the behaviours, you can design and deliver targeted behaviour interventions that can help to overcome inhibiting influences.

The following are some examples of positive changes that you can make to support the development of a good security culture within your organisation.

Recognise that people are integral to successful security

Support people to get their job done as effectively and securely as possible. Develop capabilities and cues to make delivering secure behaviours easy. The less resistance to delivering a change in behaviour the easier it will be to support in the development of security as a habit.

Security Education, Training and Awareness (SETA)

Commit resource to the delivery of your SETA programme. Utilise the wealth of available resources to help you implement and deliver an effective programme. If you’re unsure what level of resourcing you require or what an effective programme looks like a good place to start is by reviewing the latest SANS Security Awareness Report and SANS Awareness Planning Kit.

Security as a business enabler

Support the business in achieving its goals and objectives. Enable people through helping them to effectively manage security risk. This will encourage staff to engage with the security team rather than finding ways of circumventing them.

Lead by example

The board and senior management need to lead by example and champion security. People will look to the leadership to provide an example. Ensure they receive tailored and targeted training, awareness and reporting. Help them to understand and manage the security risk to the business whilst enabling in the delivery of the organisational goals and objectives.

Avoid a culture of blame

Focus on the enforcement of good security behaviours through positive acknowledgment. Measure individuals progress and recognise their effort and improvement. Understand the root cause of bad security behaviours and design interventions to help address them. Fostering a culture of blame will encourage people to protect themselves often at the detriment of the organisation.

Make policies and standards fit for purpose

Engage with stakeholders from across the organisation. Be willing to recognise and address where they aren’t fit for purpose or are having a detrimental effect on peoples work. Support people to perform their role effectively and securely. If people don’t feel they are fit for purpose they are more likely to feel it is acceptable to bypass them.

Summary

Culture is developed at all levels of the organisation but needs to start from the top through the championing of security and cascading down through the organisational structure.

Assess the security culture in your organisation and understand what blockers exist in developing a good culture. Understand the root cause of these blockers and design initiatives / interventions to overcome them.

You can positively influence the security culture in your organisation but be careful to plan for this to take time and a concerted effort to deliver. Culture is something that needs to be managed and sustained over a prolonged period.

Further reading

There is a wealth of readily available information on developing a culture of security. The following are well worth reading to support you in the development of your own security programme:

• ENISA Cyber Security Culture in Organisations
• NCSC You Shape Security
• KnowBe4 Security Culture Report

Keeping track of application security flaws

This article provides a granular view into how to track and visualise application security flaws. It builds upon a previous article that provided a high-level overview of how to keep track of your application security posture. We'll do a deep dive into a few metrics identified within the original article. This will help you to understand how to visualise the metrics and what to look out for when analysing / trending the data.

Before getting started its important to be aware that garbage in (flawed input) will lead to garbage out (flawed output). Ensure the flaws identified by your security controls are genuine. A combination of false positive (incorrectly identify a vulnerability) and false negative (incorrectly identify that a vulnerability does not exist) will distort your findings.

Reporting on flawed data can be particularly problematic as you may incorrectly prioritise and resource unnecessary mitigations or fail to act in situations where mitigations are required.

Finding / Flaw Creation Rate

Track the rate of newly created flaws over a set period. Flaws are often introduced due to:

• The deployment of changes;
• Newly identified vulnerabilities in utilised technologies;
• A failure to maintain technologies or adhere to security best practice standards.

What to look out for

Be aware that both upward or downward trends require further investigation as they can be considered positive or negative.

No change	If the number of identified flaws remains consistent this indicates that the security posture of your application/s is being maintained.
Upward trend	Positive This can indicate improvements in your capability to identify flaws. This may include the increased: Effectiveness in flaw identification tools / techniques Scope of systems covered in your programme Negative This can indicate declining security standards.
Downward trend	Positive This can indicate improving security standards. Negative This can indicate a reduction in your capability to identify flaws. This may include the decreased: Effectiveness in flaw identification tools / techniques Scope of systems covered in your programme

Finding / Flaw Remediation Rate

Track the rate of flaw remediation over a set period. Flaws are remediated due to:

• The deployment of changes;
• Patching of vulnerabilities in utilised technologies;
• Maintenance of technologies or alignment to security best practice standards.

What to look out for

Be aware that both upward or downward trends require further investigation as they can be considered positive or negative.

You will need to ensure that flaws marked as remediated have been fixed. Incorrectly closing flaws distorts the remediation rate as well as the overall security posture of the application.

No change	If the number of remediated flaws remains consistent this indicates that the resourcing level is being maintained.
Upward trend	Positive This can indicate an increased level of effort / resourcing or more effective enforcement of security standards. Negative This can indicate the potential gamification of the vulnerability management process. Check to ensure that flaws are not being incorrectly closed or suppressed.
Downward trend	Positive This can indicate a reduction in the total number of outstanding flaws. Negative This can indicate a decreased level of effort / resourcing or a decline in enforcement / adherence to security standards.

Flaw Growth Rate

The growth rate is derived from the flaw creation and remediation metrics. This is calculated by:

Flaw Growth Rate = Flaw Creation Rate - Flaw Remediation Rate

What to look out for

Upward and downward trends have a clear positive and negative correlation. From a security risk perspective, you want to see either no change or a downward trend to ensure that the associated level of risk is at least being maintained.

No change	A flat growth rate indicates that the security posture is being maintained at a consistent level. This may be an issue if you have a significant backlog of flaws.
Upward trend	Negative This indicates that the number of open flaws is increasing. An upwards trend can be a good indicator of increasing risk exposure.
Downward trend	Positive This indicates that the number of open flaws is decreasing. A downwards trend can be a good indicator of decreasing risk exposure.

Visualising the data

The following charts help to demonstrate how you can visual this reporting to support with the analysis of your data. The charts have been created based on the below table.

Flaw Creation & Remediation Rates

The below bar chart summarises the flaws identified and remediated over a period of six months.

If you were seeing this within your own data, you would want to determine why there is such a disparity between the rate of flaw creation and remediation. This is highlighting a concerning upwards trend.

Flaw Growth Rate

The below waterfall chart demonstrates the flaw growth rate over a period of 6 months. This clearly identifies a growth trend and shows that the total number of flaws have grown by 46 over that period.

Whilst the chart demonstrates a lagging (historic) indicator this can also be used as a leading (future) indicator in projecting trends. Given the identified average monthly growth rate of 8 flaws, you can predict that based on the current trajectory the backlog of flaws will end up doubling to 92 within the next 6 months. This provides a clear indication of increasing risk exposure.

What actions should you consider taking?

On the basis that the reported data is correct there are a couple of actions that you will want to take.

Reduce newly created flaws

Its always easier and more cost effective to address the flaws early in the software development lifecycle. You will want to consider:

• Defining and enforcing a secure coding standard;
• Integrating security tools (i.e. SAST, DAST) into the development lifecycle;
• Improving the security capability of your developers / testers
• Improving the security team engagement into the development workflow.

These actions will help to reduce the number of identified flaws within new deployments.

Remediate the flaw backlog

The flaw growth rate has led to 46 flaws in the production environment. The existing remediation priority / resourcing is insufficient to maintain the flaw backlog let alone reduce it. Investigate what can be done to increase the rate of remediation.

Correlating the flaw growth rate with the underlying risk will help you to indicate where the level of risk is outside of your company’s appetite. In doing so this may help you in getting increased resource allocation to address the flaws.

Delivering a behaviour focussed security training programme

Companies often start out delivering a security training program to meet compliance requirements driven by standards and regulation. This helps to tick the compliance check box but is unlikely to deliver the security goals and objectives your organisation requires to protect its information.

A significant proportion of those who work in information security are often keen to focus resources on delivering technical solutions. Technology alone cannot solve the security challenge and needs to be balanced along with consideration of people and process. An effective behaviour driven training and awareness program can transform your staff from a perceived security weakness to a key security strength.

This article considers what you need to deliver an effective behaviour driven programme.

Identify different target groups and training topics / needs

The threats and vulnerabilities associated with individuals and job roles will vary. Consider how vulnerable, attacked and privileged the staff are in your organisation. Use this information to target content to achieve the greatest impact (i.e. reduction of security risk). The following provides an explanation of these risk factors.

Vulnerability

Has an individual proven vulnerable to specific threats in the past such as installing malware or clicking phishing emails? If they have historically been vulnerable to a threat, without a change in behaviour (intervention) they have a higher probability of being susceptible to comparable threats in the future.

Attacked

Is an individual being actively targeted? This provides an opportunity to see which individuals or groups are being targeted and the types of attack they are experiencing. Targeting training to specifically address identified threats will deliver a far great impact than providing generic / non-specific content.

Privileged

Consider the level of privilege an individual or target group have? By privilege I mean the authority possessed by a particular individual or group. In information security this relates to the level of access they have to systems or information (read, modify, and delete). Those with a higher level of privilege are more likely to be targeted by attackers as they impact of compromising them is often far greater. The below are an example of roles that would be considered to have a higher level of privilege:

• Finance – ability to make / approve payments
• Directors – access to highly confidential intellectual property
• IT – administrative access to systems

Consider these three factors in determining the risk related to individuals or groups. You are going to need to prioritise targeting and tailored content towards those that are highly vulnerable (increased likelihood), have significant privilege (increased impact) and are being actively targeted (increased likelihood).

Whilst you have little control over whether an individual is being attacked you can reduce the likelihood and impact of compromise by increasing their capability through training / awareness and managing privilege through adherence to the least privilege principle.

Continual reinforcement of training

By delivering targeted training frequently you can increase the likelihood of improving your staff capability at identifying and responding to threats. Continual reinforcement helps to address the following challenges.

New and changing threats

The threats your organisation faces will evolve over time. Your program needs to be responsive to address new and changing threats as they happen. Failure to adapt will reduce the overall effectiveness of the training you deliver.

Forgetting curve

Hermann Ebbinghaus (a 19th century German psychologist) introduced the concept of a forgetting curve. He identified that people forget 90% of what they have learned within a few hours after learning it. This is down to the information remaining in short term memory. Continual reinforcement of training has been proven to increase the likelihood that information learnt will persist within long term memory.

Deliver positive security behaviour change

Behaviour is the “way in which one acts or conducts oneself”. If an individual has historically demonstrated bad security behaviours, they have an increased probability of repeating those behaviours. Throughout your organisation staff will demonstrate both positive and negative security behaviours. You will need to design and deliver interventions to deliver a change to those behaviours.

Getting Started

It can be daunting to know where to start. I would suggest using the Cybsafe Security Behaviour database to get started. This provides a comprehensive cyber security behaviour database that is maintained by a global community of security professionals and academics. In time you will want to supplement this list with behaviours that are unique to your organisation.

Delivering change of behaviour

To change behaviour you will need to look into designing and implementing interventions to support in the delivery of change. Interventions are specifically designed to address factors such as capability, opportunity and motivation that are currently impacting delivery of the desired behaviours by individuals in your organisation.

Analysing behaviours and designing interventions are a significant topic that I will address within a follow up article.

Resources

If you want to understand more around changing behaviours, there are some great resources available. There is a practical guide available covering the Behaviour Change Wheel. This will help you to analyse / define behaviours and design / deliver interventions.

The Information Security Forum (ISF) have produced a number of whitepapers covering Human Centred security. These are well worth a read and are specifically targeted at Information Security. They provide detail related to initiatives that can help deliver effective security interventions.

Summary

I often hear a level of frustration from those working in Security Education, Awareness and Training (SETA) due to being afforded limited resources to operate their programs. By shifting the focus to delivering security behaviours you will be able to demonstrate the value that your program is delivering and in doing so can produce a stronger business case for greater levels of investment.

Human-centred security is a hot topic in the information security world. Its great to see those in security learning from other professions such as psychology in order to increase the effectiveness of their own initiatives.

By building out measures to track behaviours you will also be creating an invaluable data set that will not only help you identify past behaviours but will also support in the prediction of future behaviours.

The fundamentals of Information Security

Information Security is a specialised risk management function that supports the business to understand and manage security related risk. As a team they provide advice and help to design, implement and operate security controls to bring security risk into the businesses risk appetite or at least within risk tolerance. I will explore this statement throughout this article as it can be confusing to understand what this actually means.

The Information Security team often consists of differing specialisms to enable security management across diverse subject areas. The function is often perceived to be technical in providing IT/Cyber Security. Information Security is wider in scope and seeks to equally manage security risk relating to people and process in addition to technology. Through effective security risk management the team seek to enable (rather than block) the organisation to take advantage of opportunities. This is achieved through balancing risk and reward.

In NIST SP 800-59 Information Security is described as:

“The protection of information and information systems from unauthorized access, use, disclosure, disruption, modification, or destruction in order to provide confidentiality, integrity, and availability.”

The goal is to protect information and information systems to provide the three key components that make up the CIA triad. These are:

• Confidentiality
• Integrity
• Availability

Confidentiality

Access to information should be restricted to only those who need access to it. This protects information to ensure it is only accessable to those who have a need to know it as part of their role.

Consider the scenario

A long standing member of staff has worked across multiple departments / functions within the organisation. Their level of access has increased to allow for their new role but previous access has been retained. They have access to more information than they require and as a result pose a far greater risk to the organisation through both the malicious or accidental exposure of data.

Integrity

Assurance that information is accurate, and reliable. This protects against unauthorised modification of both data at rest (in storage) or in transit (in transport).

Consider the scenario

A member of the customer service team are involved in making payments to customers. A lack of adequate controls around making changes to customer bank details increases the risk of internal fraud on the customers account.

Availability

Information is available to authorised staff as and when they need it. This protects against destruction or loss of data and disruption to services.

Consider the scenario

The organisations systems are subject to an external Denial of Service attack. A lack of adequate controls leads to the systems being unavailable to customers at their point of need.

To help understand the initial statement its important to understand the fundamentals of risk and how it applies in the management of security risk.

Risk Management

Risk itself is defined by ISO as:

"The combination of the probability (likelihood) of an event and its consequence (impact)."

The definition of risk used within Information Security is often bias towards negative consequence. Risk management requires a more balanced view between security risk (CIA impact) and opportunity risk as risk itself can also be positive.

The fundamentals of risk management require an organisation to define its risk appetite. This is the amount of risk that an organisation is willing to accept. Risk is a balance between the positive opportunity (what could we gain) and the negative consequence (what could we lose). The level of appetite is determined using a risk matrix that is typically based on likelihood (expected frequency of event) and impact (often determined through categories such as finance, reputation, regulation). Appetite states that this is the level of risk we as an organisation are willing to operate at.

The level of acceptable appetite will vary considerably between organisations with those in heavily regulated areas often more adverse to taking risk. An organisation may set out that it has a low appetite for risk but there will be situations where it is willing to take higher levels of risk (tolerance) as this is justified by the potential reward. This risk tolerance is the level of variation management are willing to accept.

There is a limit to the level of risk tolerance that an organisation can take which is the risk capacity. This is the level of risk that can be tolerated without potentially compromising the existence of the organisation.

Managing Information Security Risk

The Information Security team perform risk assessments and provide advice / consultancy on how to effectively manage risk. The team articulate the security risk and advise on how it can be mitigated. This information supports management in making an informed decision balancing risk and reward in pursuit of the organisation’s goals. Where the level of risk being taken is above appetite an escalation process needs to be followed to ensure the risk is owned and effectively managed. The stakeholders involved in the escalation process should be expected to increase in seniority as the level of risk being taken increases.

For risk to be effectively managed it needs to be documented to ensure the organisations overall risk posture is understood. Failure to identify or acknowledge risk or its blind acceptance undermine efforts to manage risk and can lead to an organisation taking unjustified risks or even exceeding its capacity for risk.

Security risk needs to be considered in terms of both existing risk (current status) and emerging risk (future status). There will always but short to medium term challenges but its important to balance these out with longer term strategic planning to enable organisations to adopt new innovations securely and within appetite / tolerance.

Security Controls

Security controls are implemented to be able to bring security risk to a level acceptable by the organisation. Risk management does not require controls to be effective, it simply requires controls to manage risk sufficiently within an acceptable threshold.

The controls themselves need to be in proportion to the risk. Where the cost associated with the realised risk is less that the cost associated with implementing and operating the control this cost cannot be justified. A role in information security requires a level of pragmatism to design and implement controls that are proportionate to risk whilst allowing organisations to take advantage of new opportunities.

Lawrence Gordon and Martin Loeb are economists at the University of Maryland. They published a study on “The Economics of Information Security Investment” in 2002. In this they suggest that the optimal amount to spend on information security should never exceed 37% of the expected loss resulting from a security breach. As with anything in risk this is subjective but provides a potential guide as to what level of spend is considered proportionate in the management of security risk.

Security Resources

Information Security resources are often limited in both staff numbers and budget. This makes it important to understand where the greatest levels of security risk exist so that resources can be appropriately prioritised. This can be achieved through assessing the value of assets to identify their risk to the organisation. For each asset this process often involves associating a risk rating against each component of the CIA triad. This approach seeks to proportionately apply resources according to risk rather than attempting to protect all assets equally.

Hopefully this post has helped build your understanding of the fundamentals of Information Security as well as appreciate the wider dependency on risk management within the organisation.

Keeping track of your applications' security posture

There are so many different indicators that you can use to track the security posture of your applications. The challenge is to determine what to track and what good looks like. Make sure that the indicators that you track offer value and aren’t just creating noise. You’ll also need to differentiate between increased risk and improved effectiveness of security controls.

The intention is to comprehensively implement controls and ensure they are efficient and effective at meeting the desired outcome. Do your indicators help you do this?

In this article I’ll be discussing application security indicators and some wider considerations around how to use them to monitor the ongoing effectiveness of your security initiatives and programs.

General Considerations

Some basic considerations:

• Check which CVSS version they utilise, where possible use a consistent version;
• Check comparable findings across tools to ensure vulnerabilities have consistent severity ratings;
• Question the validity of the findings, not all findings will be valid;
• Tools require refinement to reduce false positive and false negative results;
• Watch out for exceptions being creating around false results, check that these are and remain valid.

Quality of findings

Its important to note that the quality of findings will vary across controls. Penetration testing will involve an individual proving the existence of a vulnerability. Automated tools especially those that are unauthenticated have a higher probability of identifying false results.

Of the total findings identified check how many are:

• True positive;
• False positive – incorrectly identify a vulnerability;
• False negative – incorrectly identify that a vulnerability does not exist.

You’ll need to go through a process of refinement to reduce the number of false findings. This may involve recording respective false findings as exceptions within your tools.

Potential Indicators

There are lots of potential indicators that you can track. The following are some indicators for you to consider. These are all impacted by the quality of findings and the consistency of ratings. Refine the tools used to reduce any false results and ensure consistent severity rating of findings.

Control Implementation Indicators

According to NIST 800-55 implementation measures demonstrate “progress in implementing programs, specific security controls, and associated policies and procedures.”

Number of automated tests and tools

Determine what proportion of your applications are in scope for your security services. From that scope check how many are covered by your services. It doesn’t matter how effective your controls are if the scope of implementation is limited.

Assessment Coverage

Check what proportion of the application is covered by the assessment. Seek to maximise the coverage of your controls.

Assessment Frequency

Check how frequently security assessments are being performed against the in scope applications. Does this frequency adhere to your or industry defined standards?

There is a balance to be found according to your organisations risk profile and the resources available to you. Testing:

• Infrequently will leave vulnerabilities undetected on your applications for longer durations;
• Frequently will provide you with a more frequent snapshot of your security posture but is more resource intensive to operate.

 

Control Effectiveness / Efficiency indicators

According to NIST 800-55 Effectiveness / Efficiency measures “monitor if program-level processes and system level security controls are implemented correctly, operating as intended, and meeting the desired outcome”. The following are some potential indicators to consider.

Number of application vulnerabilities

At a very basic level you can track the number of vulnerabilities across one or more applications. The number is typically tracked along with the respective severity ratings:

• Critical;
• High;
• Medium;
• Low.

Ratings are reflective of the vulnerabilities exploitability and impact. The vulnerability score is typically derived using the Common Vulnerability Scoring System (CVSS). It is important to note than a combination of vulnerabilities can represent a higher severity to the application than the individual ratings considered in isolation. For instance two medium findings combined may constitute a critical vulnerability.

Type of vulnerability

Tracking the type of vulnerability helps to apply context to the vulnerabilities identified across applications. Example types include:

• Cross Site Scripting (XSS);
• SQL Injection;
• Cross Site Request Forgery (CRSF);
• Insecure Cryptographic Storage.

For a more comprehensive list of web application vulnerabilities see the OWASP Top 10.

Look out for consistent / repeat findings across one or more applications. These can be an indicator of weak standards or poor implementation of standard requirements. Seek to understand the root cause rather than trying to address findings on a case by case basis.

Source of discovery

It is common to have multiple controls that are used to identify application vulnerabilities. Consider correlating the number, type and quality of findings with the source tool. This helps to determine which tools and processes are most effective and offer the greatest return on investment (ROI).

Average Time to Fix or Defect Remediation Window (DRW)

From the initial identification of a vulnerability track how long it takes in days until it is verified closed. Companies will typically define time to fix requirements in accordance with the finding severity rating.

Make sure you:

• Refer to the original identified date as findings often come from multiple sources;
• Verify that findings have been closed.

This indicator will help determine if vulnerabilities are being addressed within the timescales set out in your standards. Repeated failure to meet the agreed timescales can be a good indicator of increased risk exposure.

Finding / Flaw Creation Rate

Track the rate of new findings over a set period. Its worth noting that an increase in findings may relate to an improvement in your ability to identify findings rather than a drop in development standards.

Finding / Flaw Remediation Rate

Track the rate of remediated findings over a set period. Make sure you verify that findings are closed. It’s possible that an applied fix isn’t sufficient to address the finding.

Finding / Flaw Growth Rate

If the finding creation rate exceeds the finding remediation rate this is a key indicator that vulnerabilities are increasing. This is a good way to identify increasing risk exposure.

Flaw Growth Rate = Flaw Creation Rate – Flaw Remediation Rate

Factor in the severity rating of findings. An increased growth rate driven by low findings alone may not suggest an increased risk exposure.

Density

Not all applications are created equal. Larger applications have a greater attack surface that can be exploited and its reasonable to expect them to contain a greater number of findings.

Consider tracking the number and severity of findings according to a defined density such as lines of code, number of pages or screens. This will be a more effective indicator of the security standard of an application.

Rate of findings per lines of code = Lines of Code / Number of Findings

It may be difficult to source sizing details for applications. Investigate what tools are available to you to support gathering of meta data relating to your applications.

Weighted Risk Trend (WRT)

It is difficult to articulate the actual risk of applications based on just the volume and severity of their findings. WRT makes it possible to derive a single measure which uses the business criticality of the application to determine risk in the context of the business.

WRT = ((critical multiplier x critical defects) + (high multiplier x high defects) + (medium multiplier x medium defects) + (low multiplier x low defects)) x Business Criticality

Choose severity multipliers that work within your organisation and make sure the calculations you use are transparent to the intended audience. Multipliers could be assigned scores on a simple basis of a range of 1 (low) to 4 (critical) or with wider weighting increasingly afforded to the more significant severities such as:

• Critical = 9;
• High = 6;
• Medium = 3;
• Low = 1.

Rate of Defect Recurrence

Identify the reoccurrence of previously remediated findings. Look out for:

• Version control related issues that lead to loss of security fixes in the production environment;
• Remediation of issues without solving the root cause of the problem;
• Developers that lack security awareness may continue to produce insecure code overwriting that which was previously secure.

Consider Business Context

Security findings need to be considered in the context of the wider organisation. For instance, an application handling confidential / regulated data will pose a greater risk to the company than a brochureware site containing publicly accessible data. Look to prioritise your resources to effectively manage the security risk of your applications.

Application criticality

Determine the criticality of your applications to the organisation. This is often used to determine / prioritise the scope of security services offered. Typical factors often include:

• Type and volume of data;
• External availability;
• Compliance or contractual requirements.

Business Impact Assessments (BIA) are a useful source of information.

Gamify reporting

In isolation it can be hard to articulate what a good application security posture is. Organisations typically own / operate multiple applications. Within a large organisation its viable to expect this to be in the hundreds or potentially thousands.

Consider the comparison across regions / business lines / legal entities / departments / functions within your organisation. Identify those with the best and worst security posture and encourage some friendly competition. You could even create a league table so that the different areas can easily see how they compare to each other.

A mean of the overall rating will enable you to trend indicators over time and track increasing / decreasing trends in your organisations overall security posture.

Summary

Identify a select number of key application indicators that will support you track the effectiveness of your controls. Be careful to avoid tracking data that does not provide an indicator of implementation, effectiveness or return on investment (ROI). Avoid tracking data that provides no clear indication as this can create noise and detract from those indicators that are important.

Target indicators to respective audiences to ensure that stakeholders have access to the right information. Identify where indicators are increasing / decreasing and determine what actions are required to address any increasing risk exposures.

Its important to work with stakeholders involved in the delivery / operation of your security controls. Work closely with your development / technical teams to educate them in the usage of the tools and provide guidance around remediation. Make sure that you recognise stakeholders for improvements that are being made and support them in their on going journey to improve.

Let me know what application indicators you track and what you have found to work effectively within your organisation.