How frequently used shortcuts can result in poor understandings of the levels of risk within an organization
This is the sixth in a series of seven blogs about the ways in which common shortcuts can undermine operational risk management success within financial services firms. It is the first of two blogs that focus specifically on stress testing and scenarios.
Stress tests and scenarios have proven problematic for operational risk teams since they were mandated under Basel II more than a 15 years ago. This is partly because they are an attempt to imagine situations in which exceptional risks materialize within the business, and then to quantify both the likelihood and impact of those exceptional risks happening. This translation of qualitative insight into quantitative information has repeatedly proved to be problematic.
In this first blog, we’ll look at practices that have perhaps outlived their usefulness. Over time, as the discipline has evolved, some practices that were used early on are now viewed as damaging shortcuts. These include:
Ignoring reputation damage – Often firms forget to include reputational damage in their scenarios, or leave it out because they consider it too difficult to work out how much the impact of it could be. Indeed, technically reputational risk sits outside of the strict Basel II definition of operational risk, as does strategic risk. This is a very bad shortcut to take – quite a lot of the significant damage that firms can suffer today during a loss event can be in reputational damage, particularly as a result of social media. So, when considering a scenario, think about what the impact would be if the loss event appeared on social media and went viral – if social media could have an impact on the size of the loss then it’s a good idea to include reputational risk in that scenario.
Using only one factor (risk) rather than multiple factors – Modelling based on a single factor is a stress test. Modelling multiple operational risks at the same time is considered to be a scenario by regulators. So, for example, when an operational risk team and the business decide to look at a big internal fraud, and together they simply ask – How big could that internal fraud be? – that is a stress test.
Certainly, firms are expected by regulators to do stress tests for key operational risks, but focusing on stress tests at the expense of scenarios is a shortcut that many firms take. A more realistic and useful way to model risk – and to prepare for operational resilience – is to do scenario testing.
For example, model an internal fraud by saying, “OK, we’ve lost our building’s IT capabilities, so we will go down to the disaster recovery site, only to arrive and discover that the site isn’t functioning because the monthly tests that were supposed to be performed were not completed. So, there is no IT system functioning for the organization. As a result, someone takes the opportunity to commit an internal fraud.” Such a scenario is much more aligned with the way risk can unfold in an organization over time, and will “kick the tires” on more risks and controls – which is the whole point of doing this kind of exercise.
Using each Basel Loss Event Type (BLET) for the single factor – These are the seven event-type categories that were created during the development of the operational risk framework under Basel II, and include items like internal fraud, external fraud, and damage to physical assets. A frequent short cut is to simply use these seven BLETs for seven straight-forward stress tests, each associated with a single risk. This is problematic in the same ways that using only one risk factor is – the outcome of such a straight-forward risk analysis is unlikely to produce a realistic assessment of either likelihood or impact of real-world risk situations. This is because using a single factor approach requires risk teams to ask the business to guesstimate the size of both likelihood and impact at points far into the future – an approach that has its weaknesses.
Using a confidence level that is beyond normal management’s experience, such as 1-in-100 years – Although it might seem easy to ask management to guess if an event is a “1-in-a-100 year event”, the truth is that such a speculation is beyond the actual experience of even the most seasoned manager. A 1-in-25-year event estimate is just about the span of an individual’s career, so people are able to make an estimate on this. A one-in-50 year event estimate is only really appropriate for individuals who are nearing retirement. Avoid asking individuals to estimate beyond 1-in-50 or 1-in-100 years during exercises – in fact in some jurisdictions, regulators have explicitly asked firms to stop using 1-in-100 years this because it is meaningless.
Using two or three confidence levels only (1 in 5 years; 1 in 25 years; 1 in 100 years) – Asking the business to estimate just two or three confidence levels, and then calling it a day, when doing stress tests or scenarios, is a very dangerous way to undertake these exercises. Once again, the information the business provides will not accurately reflect the levels of risk, particularly at the outer end of the time spectrum. And having only a very small number of confidence levels is not enough information to base correct risk analysis on.
Today, firms should have moved beyond these practices. As suggested above, one way to model operational risk avoiding these shortcut traps is to combine several risks into a plausible scenario, using conditional probabilities. Building these scenarios ideally with the help of risk management software, will help firms develop a better understanding of the robustness of their overall risk and control environment.
For more information about stress testing and scenario modelling best practices, contact RiskLogix on 0207 377 2250.
