The Shortcuts Trap – Key Indicators Under Fire

  • Subscribe to updates

  • Privacy
  • This field is for validation purposes and should be left unchanged.

How common key indicator errors can increase risk and reduce resiliency.

This is the second in a series of four blogs about the ways in which common shortcuts can undermine overall operational risk management success within organizations. You can access the first blog here Risk and Control Self Assessments.

Everyone loves metrics, and particularly key indicators such as key performance indicators (KPIs), key risk indicators (KRIs), and key control indicators (KCIs). In a perfect world, you’ll be using risk management software and each data point should provide the organization with importance intelligence. However, most operational risk teams wind up burdened with hundreds of KPIs, KRIs and KCIs, and then find they cannot see the forest for the trees. At some point, most operational risk teams will find themselves needing to sit down and shake up their stock of key indicators. And that’s where the temptation to use shortcuts can arise. Common shortcut mistakes include:

  • Deeming the KPIs to be KRIs – Often the business will have a list of KPIs that they track for operational and financial purposes. There can be a temptation to just adopt all of these KPIs as KRIs, and call it a day. However, KRIs are just what they say they are – and so they need to be identified with a specific risk. For example, a KPI that everyone looks at is profitability, and this is an excellent KPI. However, it tells the operational risk team almost nothing specific about the way the firm is managing any of its operational risks or controls. Operational risk teams should challenge the use of KPIs as KRIs by asking which specific risk or control a KPI relates to.
  • Regarding KCIs as KRIs – Once again, it’s important to distinguish between what is a KRI and what is a KCI, and to treat them differently. KCIs are vitally important for an operational risk program – they measure the strength of the control environment. Treating them as KRIs leads to failing to consider the control environment on its own. KCIs should be paired with a relevant KRI, for maximum operational risk framework robustness. As well, they should be tested regularly through the control testing that the operational risk team performs.
  • Using the data to set the thresholds – Although using the data might seem to be a straight-forward way to set the thresholds of key indicators, it is deeply problematic. For example, some operational risk teams will look at the data for a key indicator and say to themselves, “we’re generally where we want to be, but sometimes we’re not so that must be amber, and remember that really big one two years ago, that must be red.” However, this thinking doesn’t really capture what is happening on the ground within the business. In fact, the process the key indicator is capturing may be experiencing problems, so what looks like “green” from the data might actually be amber or red for the team. It’s important to have a conversation with the business for each KRI and KCI being used, to set the thresholds according to what is right for the business.

  • RCSA-Dangers-of-shortcuts

  • Believing all KRIs are early warning signals – Certainly some KRIs are early warning signals. In particular, those KRIs which relate to the likelihood of a risk happening can fall into this category. However, those KRIs which are indicators of the size of the impact of a risk event are not early warning indicators – they simply indicate the possible magnitude of the impact. As a result, impact KRIs are more of a lagging indicator. An example of a good operationa risk early warning signal is a staff turnover metric. Staff leaving a settlements processing team leads to a loss of operational history as well as the need to train new staff. Existing staff can become demoralized and not train new team members properly – and so risk can rise. Therefore, indicators of likelihood are good early warning signals because the right ones can provide an insight that a risk is more likely to happen. However, in this example, the number of settlement fails would be an impact indicator – it is an indicator of the impact of losing key staff – and not providing an early warning of anything.
  • Tossing out indicators that aren’t predictive – Most financial services organizations have far too many key indicators – they tend to accumulate over time. However, because there is a commonly-held belief that all key indicators are early warning signals, some operationa risk teams have simply put other types of key indicators in the bin. This is a big mistake. For example, KRIs that suggest the potential impact of a risk event, as well as KCIs are important for understanding operational resilience. Instead, operationa risk teams seeking to reduce the number of key indicators should look towards industry best practice and indicator effectiveness as benchmarks.

In short, it’s important to take a more thoughtful approach to restructuring an ecosystem of key indicators. Taking shortcuts can reduce the ability of certain KRIs to be predictive, and damage the firms’ understanding of its operational resiliency, through other KRIs and KCIs, as well.  

Related Posts

The Shortcuts Trap – Considering stress testing and scenarios, part 2
Ways in which popular shortcuts can result in problematic understandings of the levels of risk a firm can face. This is the final blog in a series of seven about how frequently-used shortcuts can undermine operational risk management programs within financial firms. It is part two of a blog looking at stress testing and scenarios …

The Shortcuts Trap – Considering stress testing and scenarios, part 2 Read More »

The Shortcuts Trap – Considering stress testing and scenarios, part 1
How frequently used shortcuts can result in poor understandings of the levels of risk within an organization This is the sixth in a series of seven blogs about the ways in which common shortcuts can undermine operational risk management success within financial services firms. It is the first of two blogs that focus specifically on …

The Shortcuts Trap – Considering stress testing and scenarios, part 1 Read More »

The Shortcuts Trap – Focusing in on Risk Appetite
For risk appetite, it pays to think more strategically This is the fifth blog in a series of seven about the ways in which frequently used shortcuts can deliver poor outcomes for operational risk management teams within financial services firms. It is surprising how many firms short-change themselves by taking a minimalist approach to risk …

The Shortcuts Trap – Focusing in on Risk Appetite Read More »