Using all data sets for risk management modelling
Our second in the series on modelling, this time Tony and John look at input data. Operational Risk Software can be key to supporting this discipline.
Taken from: Mastering Risk Management
Having looked at the business benefits and uses of modelling output, it is important to look at the other end of the process, i.e. the input data. Modelling risk management is an inherently difficult process as the quality and quantity of data varies enormously. For example, a firm will have large amounts of data on the creditworthiness of its customers. It will have very little information on internal fraud and probably some, but not a large amount, of data on transactional errors. In short, while the data relating to the financial side of the business are likely to be of sufficient quantity and quality of modelling, the data relating to the non-financial side are clearly suspect and require special attention.
There are four types of non-financial data available to a firm:
- Data on the business environment and the firm’s controls that mitigate the environment (i.e. risk and control self-assessments)
- Data from the losses that the firm has suffered (i.e. internal losses)
- Data on scenarios and stress tests that the firm has used to protect itself from exceptional events
- Data on losses that competitors have suffered when the competitors’ control environment has failed (i.e. external losses).
We consider each type in turn.
The business environment and the firm’s mitigants
This type of data is typically represented by a firm’s risk and control self-assessment. It has a relatively small amount of data on each risk as the risk is assessed only once. However, it does have the advantage that it is data from the people who run the firm, i.e. management and senior management. This gives the data great significance for looking into the future as the risks stated are those events that may cause a firm to be unable to meet its business objectives. In addition, there will be an assessment of the controls that mitigate each of the risks.
The use of subjective, management forward-looing data is very common in business. It is used every time a firm develops and implements its five-year business plan. It is also very common to use this type of data in modelling, particularly when combined with objective actual data. For example, an oil firm will model its reserves through a combination of actual geological data and the management’s knowledge and experience. An engineer will model the likelihood of an oil rig leg collapsing through a combination of data on previous rigs’ collapses and the engineer’s knowledge and experience of similar structures.
Losses that the firm has suffered (i.e. internal losses)
This type of data has the advantage of being actual data, although it has occurred in the past and any control failures that are outside appetite will have probably been fixed. As it contains past data it requires challenging for any losses that are no longer relevant and which must therefore be eliminated from the data set. For example, if a firm has sold a business the losses from that business will no longer be relevant to the firm’s current economic capital needs. In addition, it is quite common that not all losses are captured by risk management and therefore the data set is incomplete.
Scenarios and stress tests from exceptional events
Again, this is subjective information and is management’s view of exceptional but plausible events that could happen to the firm. As such it is a valuable dataset showing management’s reactions to extreme events and how controls perform when under stress. Clearly this dataset forms only a small part of an economic capital calculation as it represents extreme situations. However, it is a vital part of economic capital as it also represents the capital required for the firm to survive exceptional events.
Losses that competitors have suffered (i.e. external losses)
Competitors’ events are probably he most difficult dataset to use in modelling. It is much more difficult to obtain full data on the event and the competitor’s control environment is largely unknown in detail. However, they form one of the four datasets available to a firm and can be used to augment and enhance the firm’s other datasets. It is good practice to use competitors’ loss data as a representation of a deterioration of the firm’s own control environment.
From a modelling perspective, cleansing of external data is particularly important so that the external data better represents the firm’s risk profile. The term ‘cleansing’ denotes the process of checking that the losses are relevant to the firm and determining an appropriate size of the loss with respect to the firm.
While the appropriate size can be determined through some form of scaling, the relevance of the loss to the firm is the first step in the process, as there is no point scaling a loss which is not relevant. To understand the relevance, it is important to have a narrative in the external data which comments on the cause of the loss. A description which is as full and accurate as possible is therefore required. Unfortunately this is often difficult.
It is clear that a loss made by a competitor through employment practices such as strike is relevant, at some size, to other firms. Equally, this loss is unlikely to be relevant to a small retail firm whose employees are not in a union. However, a transactional loss suffered by a large retail firm through poor documentation standards may be conceptually relevant to a smaller firm. Such documentation standards are equally applicable to many other types of firms.
In addition, there may be losses made by firms outside their industry sector which are directly relevant to their sector as well. For example, the loss of the IT system and outsourcing risk, are directly relevant to almost all firms.
It is therefore important for external data to be carefully challenged, both in terms of relevance, and size, before putting such data into a model. This challenge does not have to be carried out every time the model is run, although it is appropriate to review previous challenges on a periodic basis, such as an annual review. Similarly, it is also appropriate to challenge internal data when the firm’s business model changes, when there are significant changes in the marketplace and as it degrades over time and may become only partially relevant.
What about KRIs?
Key risk indicators (KRIs) and key control indicators (KCIs) can be used instead of RCSAs as data for the business and control environment. Although these are not forward-looking they are actual objective data in the present rather than the past. In addition, trend analysis of the recent values of the KRI may give some indication of future values. KRI data are very valuable from the perspective that they represent the risks which are considered to be key. However, this is also their main disadvantage in that they only represent the key risks. The risk profile of the firm is more fully represented by the complete set of risks. The inclusion of KCIs will allow controls to be explored as well as the firm’s business environment.
Comparing the four data sets
The table below compares various attributes of the four types of data. While RCSAs and scenarios have most attributed in common, internal losses and external losses are subtly different. For example, for external losses the collection time is shorter than for internal losses (particularly if those data are from a consortium) and there is generally a higher quantity of losses available compared to internal losses. It should be noted that by combining the data of all four datasets there is at least one positive item in each row. This is as a minimum and balances out the negative items in the other datasets and overall contributes to a more complete overall set of data.
Add table here: Comparing the datasets
Combining all four datasets
Clearly if we are able to take advantage of the different positive points of all four sets of data we must combine them in some way. The typical way of combining the data is by using your own business lines and loss events types. This has the obvious advantage that your internal data (losses, RCSAs and scenarios) is almost certainly categorised in this way already. Unfortunately, although understandably, in some industries firms are forced into using regulatory classifications so the regulator can compare across firms. This is not ideal for embedding in the firm even though it should lead to better regulation of an industry as a whole.
One of the challenges in combining all four datasets is to ensure that the appropriate weighting is give to each type of data. This weighting should be changed by business line and event type according to the quality and quantity of the data in each of the four datasets.
In our next blog Tony and John discuss confidence levels and holding periods and their effect on modelling.
Mastering Risk Management by Tony Blunden and John Thirlwell is published by FT International. Order your copy here: https://www.pearson.com/en-gb/subject-catalog/p/mastering-risk-management/P200000003761/9781292331317
For more information contact us today on sales@risklogix-solutions.com
