This is a guest article written by Amrit Dhanoa. The author’s views are entirely his own and may not reflect the views of IT Governance.
If, like me, you are an avid fitness enthusiast, you’ll probably have a gadget of some sort to measure your running or cycling performance. With the recent proliferation of wearable technology (activity trackers, wrist bands and smart watches), we’re spoilt for choice. For the serious runners and cyclists out there, GPS-enabled devices have been available for many years to record speed, distance, pace, heart rate, etc.
This is the sort of thing that we, as information security and compliance folks, should be applying to our information security management systems. How do you measure the effectiveness of your information security processes, controls and activities? And why do you need to measure them? The answer, unfortunately, isn’t to purchase a new GPS watch.
Where are we going?
Let me start with an analogy. Like many of you, I enjoy cycling and running as a means of keeping fit as well as emptying my wallet each year on expensive carbon cycling wheels or the latest performance gadget!
I’m now reliant on these GPS gadgets to help measure my performance. While I could simply enjoy a run or cycle without measuring anything, I want to improve and go faster than before, so I employ measurements to understand my current level (baseline) and set targets or goals to achieve. As I plan for a race or training session, I set a goal or objective supporting my aim of being faster or fitter than before. I’ll measure progress against the goal by recording data on my GPS running watch.
The watch analyses the data and presents information (metrics) in a format I can understand. I’ll review the metrics as I run to ensure I’m on target for the objective. Without the metrics, I have little idea of how I’m performing or what, if any, corrections I need to make. Sure, you can still go for a run without your GPS watch, but if you are aiming for a goal you need a structured training programme involving the recording of measurements to track performance.
That’s a direct analogy to almost any other management process including information security: simply putting in processes, technology and procedures doesn’t mean you will achieve your goals. You need to measure them. But first you must know the objective of measurement.
Objective of information security
With the wealth of security data at our disposal, we can often be overwhelmed by new management consoles and flashing lights without understanding the core purpose of relevant security data. I guess this is similar to the sensation I have when I buy a new pair of expensive shiny carbon fibre cycling wheels. They look so good that I have to them, but I often fail to understand the benefit or outcome of the wheels and whether I have the means to make use of them. This is no different from information security data metrics.
What is the objective of information security? Apart from keeping us in jobs, information security is there to protect the interests of the business. It is sensible to ensure that security itself has an objective that is aligned with business outcomes. Not new stuff, I hear you say? Yes, that’s true, but how many organisations are actually measuring the real value of information security rather than being overwhelmed with rich data from across the many technology systems they have in place? Here are three questions to ask when considering security metrics:
- What is the underlying driver or benefit to the business of all this security stuff?
- Can we identify objectives for our security systems and processes supporting the overall benefits?
- Can we identify processes, controls, activities for measurement and, hence, determine how well security is meeting its objectives?
A good example is the need to reduce information security risks to an acceptable level. As an outcome, it’s not measurable until we develop an objective supporting the outcome. In this case, it could be to reduce the impact of unexpected events on business activity. To achieve this, we could set an objective to ensure that the business continuity and disaster recovery processes are adequate, maintained and tested on a regular basis. Measuring test schedules, issues and timely closure of post-testing findings provides ample data metrics.
Meaningful strategic security metrics
We have a wealth of data at our disposal from the multitude of security systems in place across many organisations. While monitoring patching, firewall alerts and IDS notifications is important, it does little to inform our senior management of the real value or benefit of security. Consider a motor vehicle: operational metrics are akin to the indicators we have on the dashboard (speed, oil pressure, battery condition, fuel status, etc.). They tell us how well the engine is running but not where we are going or if we are going to get there (okay, perhaps the fuel level is a useful operational and strategic metric!). We can often be consumed by all the detailed information at our disposal without considering the strategic view.
Why we need to measure security
Demonstrating the value and benefit of information security is vital in gaining trust and buy-in from business managers. It provides a vehicle for security professionals to demonstrate what they do in a meaningful and business-aligned perspective. Without measuring security it’s akin to someone just going out for a run or long bike ride: it does you good and you might even enjoy it, but if you were aiming for a marathon or five-hour endurance cycle event, you’d have little chance of knowing how you were progressing towards your goal.
While measuring patch and vulnerability status and firewall alerts is an essential part of security metrics, we need to roll operational metrics up into a strategic view for senior management and the Board of Directors. Two strategic security metrics I like to measure are:
- Effectiveness of risk management at reducing risk to an acceptable level
- Overall strength and maturity of internal controls
Figures 1 and 2 provide examples of how security metrics could be utilised to help provide a strategic view on these metrics.
Figure 1: Information security risk exposure
Figure 1 indicates the mean risk level over time against the organisations risk appetite. We can see a trend in reducing information security risk over time, indicating that risk treatment is effective. This provides the Board of Directors with far more valuable insight than looking at patch and vulnerability trends.
Figure 2: Control maturity
Figure 2 displays control effectiveness against time using the Capability Maturity Model (CMM) to indicate the level of control maturity. There is progression towards a desired outcome, perhaps through improvements in control effectiveness and closure of non-compliance issues.
SIEM systems can correlate data from many sources for operational metrics. In some cases, however, you may want to resort to other semi-automated means to record data over time. The data for Figures 1 and 2 could be extracted from a risk register or control catalogue (e.g. ISO 27001 Statement of Applicability).
So, next time you see a runner checking their watch constantly, you’ll understand that they’re measuring data for the purposes of improvement and goal attainment. Or they could just be checking the time… Happy measuring.