Ways in which popular shortcuts can result in problematic understandings of the levels of risk a firm can face.
This is the final blog in a series of seven about how frequently-used shortcuts can undermine operational risk management programs within financial firms. It is part two of a blog looking at stress testing and scenarios shortcuts in particular.
When undertaking scenario exercises, some operational risk teams engage in shortcuts that result in either limited perception or a lack of perception about risk, even if using risk management software. In many cases the shortcuts do not appear, at first blush, to be particularly wrong as far as op risk practice goes. However closer examination shows that each shortcut will result in limited understanding of the organization’s risk and control environment, or of what happened within the scenario exercise itself. These shortcuts include:
Focusing on just the effects of the scenario rather than secondary effects – When conducting the scenario exercise, it’s easy to fall into the shortcut trap of simply looking at the immediate effects of the scenario, rather than so-called secondary effects. These are increases in either the likelihood or impact of risks that are the result of the first-level risk materializing. It’s important to consider this risk cascade effect, and how the whole risk and control environment of a firm could change as a result of these interconnections. One example might be how increased external fraud risk might lead to increased data privacy risk.
Not considering the preventative control failures that must have happened for the scenario to occur, and only considering the risk(s) – For risks to have turned into loss events, preventative controls must have failed, either in part or entirely. Yet many firms don’t bother to look at controls when they do their operational risk scenarios. Even more worryingly, they don’t consider the possibility that the control failure that led to the initial loss event could result in other loss events, too. It is then possible for the unrepaired control to result in more loss events. For example, an internal fraud may initially seem like it just impacts one business line in a certain way. However, for the fraud to have taken place, there must have been a failure in the organization’s ethics training, which is a control. So, senior management might want to invest in improving the firm’s ethics training across the entire organization, as a means of repairing the control or enhancing it.
As well, the workshop should look at the scenario as a whole for the risk profile, rather than just at a limited set of particular risks. In this example, where ethics training was identified as a potentially weak control, the organization might want to also consider what other kinds of related training could also be weak, such as the part of sales training that focuses on mis-selling.
Failing to consider the effect on the whole of the firm – Often in scenario work, the scenario will only consider the impact of a loss event on a particular business. Even if this is a major business for the organization, it’s important to think about how the loss event could impact the entire firm. This is a particularly significant issue when it comes to reputational risk – a significant loss event such as a mis-selling scandal may only happen in one jurisdiction, such as the US. However, with the advent of social media, the reputational damage could impact operations in the UK, Germany, Australia and Hong Kong.
Expecting a quick recovery within a scenario – It takes (many) months to recover from a significant scenario, and what other type is there? When doing a scenario it can seem straight-forward to say that a disaster recovery site will be up and running within 24 hours, enabling the business to be fully functional. While it’s true that the DR site may be up quickly, it can take months – or even years – for a company to truly recover. For example, one large international bank had a headquarters fire about a year ago, and this organization is still recovering. In particular, although so much of business is conducted electronically today, many paper documents were destroyed in the blaze which had to be replaced, such as mortgage contracts. So, in a significant scenario, it’s important to consider knock-on impacts and the challenges that could emerge from the periphery, which could present the business with difficulties over a longer timeframe.
ORM ‘doing’ the scenario work, rather than the business – Sometimes, op risk teams try to “help” the business by actually doing the scenario exercises for them, as a shortcut. The op risk team will then present the result of the scenario to the business for sign off. This is always a disastrous shortcut – the business will not have bought into the scenario workshop outcomes. For example, they may discount the idea that a certain control could fail. No matter how good the operational risk management team are, they will never know the business as well as the business itself does. And these scenario exercises are not “just for the regulator”. They are in place so that the business has to think through what might happen and how it should react in advance of a significant loss event or series of events.
Documenting the scenario too little or too much – It can be a shortcut to do either of these two things. On one end of the spectrum, perhaps very little documentation was produced during the scenario exercise itself – just two or three bullet points. On the other end of the spectrum, some op risk teams produce a Hollywood movie script for their scenario exercises and retain binders of information, as a kind of “brain dump”, without really properly analysing it. All organizations should have a scenario policy that outlines the steps that need to be taken and the outcomes achieved for a scenario exercise. This includes the documentation that should be produced – reports that are based on regulatory requirements as well as intelligence needs of the business and senior management.
Failing to take account of biases – Every person that take part in a scenario exercise will come to that exercise with biases. Sometimes this can be a good thing – if an individual has experienced an office fire, for example, they will have a better idea about the risks that will materialize in an office fire scenario. However, this experience can also result in biases – as a result of the individual’s expertise in office fire because of his or her experience, the group might weight the likelihood and impact of an office fire more highly than would actually be the case. So, biases are not necessarily a bad thing, but those running the scenario workshops need to be aware of them, and to take account of them.
In summary, op risk teams need to pay attention to just how shortcuts can circumscribe the perception of risks and controls. Scenario analysis is meant to open up the minds of both the op risk teams and the business to potential negative outcomes as a result of risks materializing and controls failing. Taking shortcuts is therefore dangerous – it can leave a firm with less operational resilience, damaging the ability to deliver value to shareholders.
For more information about stress testing and scenario modelling best practices, contact RiskLogix on 0207 377 2250.
