How do you challenge a risk model?
In our next series of blogs Tony and John look at challenging risk models with a business rather than a mathematical approach, something that Operational Risk Software can help with. In this first blog, we look at parameters that affect the economic capital.
Taken from: Mastering Risk Management
There are many ways to challenge a model that are non-mathematical. In reality, the mathematical challenge to a model is only one type of challenge and is very narrow. What goes into a model, the parameters around a model, what comes out of a model and the documentation and governance are all as important as the mathematics. And yet, mathematics appears to be the be all and end all of so many articles on modelling. This next series of blogs will look at various ways to challenge a model that are grounded in a business approach to modelling.
Parameters that affect the economic capital
There are a large number of parameters that can affect a model’s output. They fall into three main groups:
- Pre-modelling parameters, that is items that are completely independent of the model being used;
- Parameters relating to the modelling of the qualitative data, that is the modelling of the subjective non-financial data such as that relating to risks, controls and scenarios;
- Parameters relating to the capital model.
In addition, and as part of good governance, it is common to have a change authority relating to the parameters. This is because the parameters detailed below can result in significant capital movement and should therefore be within the normal governance structures relating to strategic decisions. Having said this, it is common to allow a change to parameters which result in a capital movement of less than, say, 10% to be approved by a model development committee.
The change should be documented in the same way as a greater than 10% change. All such changes are often presented to a risk management committee annually. However, any parameter change which results in a capital movement of greater than 10% should be approved by the risk management committee before its implementation, and after review and approval by the model development committee. Any such change will include the reasons for the change and before and after examples.
Pre-modelling parameters
These parameters relate to the challenges that can be made outside of the models and before uploading the data into the models. They are:
- Number of risks
- Number and level of controls
- Risk assessments
- Control assessments
- Risk assessment matrix ranges
- Use of events – by business line
- Use of events – by each event
- Scaling of relevant events
We shall look at each of these in detail, examining the parameter itself, the challenge that can be made to the parameter and how capital is affected by changes to the parameter.
Number of Risks
Parameter description – this is the number of risks in the risk and control self-assessment (RCSA) that is being modelled (Data Requirements – Using all four data sets for your modelling), The business environment and the firms mitigants)
Challenge – Risks stated in the RCSA should be independent of each other. It is common for risks to be interrelated. However, when modelling the elements are always assumed to be independent of each other (unless a correlation matrix has been constructed). They should therefore be carefully challenged so that the RCSA being modelled reflects a full set of independent risks (Risk management and risk and control self-assessments).
How this parameter may affect capital – Clearly, a higher number of risks will require a higher level of capital. It is tempting therefore to weed out risks and so reduce the amount of economic capital required. However, without a full risk profile, the firm will be short of the capital that it needs to maintain and grow its business.
Number and level of controls
Parameter description – this is the number of controls mitigating each risk in the RCSA. In addition, the controls should be at the same level as the risks (e.g. strategic risks should be mitigated by strategic controls and department risks should be mitigated by department controls).
Challenge – Again, controls should be independent of each other. This is even more difficult than with risks. It is common to see several controls listed as mitigating the risk when those controls are either dependent on each other or all but one of the controls are dependent on that particular control. For example, for the risk ‘Loss of key staff’ the mitigating controls may be listed as: remuneration package, basic salary, benefits, bonus. Clearly, the remuneration package is the independent control which comprises basic salary, benefits and bonus.
Controls should be at the same level as the risks, or there is the possibility of identifying controls at a lower level than the risk. Often there are a number of controls at a lower level that can be aggregated into a single control at a high level. The example above of a remuneration package is a control that is likely to be listed in a strategic risk assessment, whereas the HR department may list the three controls that make up remuneration package.
How this parameter may affect capital – A model assumes that the controls are independent and at the same level. If three controls are listed, the assumption will be that there are three independent controls mitigating the risk. Clearly the risk will be better mitigated and will happen less frequently. Therefore lower capital will be required than if the risk was mitigated by a single control.
Risk assessments
Parameter description – These are the inherent risk likelihood and impact scores in the RCSA that is being modelled.
Challenge – The inherent scores should be consistent with the residual scores combined with the control scores, bearing in mind that preventative controls tend to affect likelihood and detective and corrective controls tend to affect impact (Risk management and risk and control self-assessments) As the modelling is carried out using the inherent score and the control scores it is more difficult to game the inherent score when the three scores must be aligned with each other.
How this parameter may affect capital – Consistent scores are required in order to give a credible economic capital value. It should be noted that lower scores will give a lower loss and therefore only a smaller amount of capital is required.
Control Assessments
Parameter description – These are the control design and performance scores of the RCSA that is being modelled.
Challenge – They should be consistent with the inherent risk scores combined with the residual risk scores, bearing in mind that preventative controls tend to affect likelihood and corrective controls tend to affect impact.
Note that various categories of staff have different biases about controls. Risk owners tend to understate the quality of controls by remembering only the times when the controls have failed. Control owners tend to overstate the quality of controls by noting that controls generally work. In addition, control scores can be challenged with internal audit reports on controls and with the actual losses suffered by the firm.
How this parameter may affect capital – High control scores will reduce the need for capital as the control will fail less frequently, there will be fewer losses and therefore less capital will be required.
Risk assessment matrix ranges – likelihood and impact
Parameter description – These are the time ranges given to likelihood and value ranges given to impact for the RCSA which is being modelled.
Challenge – The ranges should be consistent with the area being assessed. The time range for an executive committee or board is likely to be in years, whereas a time range for Accounts Receivable is likely to be in months and for Accounts Payable is likely to be in weeks.
Similarly, the impact range for an executive committee or board is likely to be in millions, whereas a value range for Accounts Receivable is likely to be in tens of thousands and for Accounts Payable is likely to be in thousands.
How this parameter may affect capital – Shorter time values will lead to higher capital requirements as the risk occurs more frequently. High impact values will lead to higher capital requirements as the impact is more costly to the firm.
Use of events – by business line
Parameter description – Losses and events are categorised by the firm’s business lines.
Challenge – Are each of the business lines still relevant to the firm’s business profile today? As a firm’s business changes so does its risk profile and therefore the economic capital required by it. Business lines that were relevant a few years ago may no longer be so large or may have been sold and therefore not relevant at all. This challenge applies to both internal and external data.
How this parameter may affect capital – Business lines that are not relevant will wrongly increase the firm’s economic capital by requiring capital for businesses that are no longer material or relevant to the firm.
Use of events – by each event
Parameter description – Any recorded event or loss.
Challenge – Even within relevant business lines, events may not be applicable as the risk profile of the firm may have changed from when the event occurred. The events relating to a business that has been sold or reduced in scope may no longer be relevant. Conversely, events relating to a business that is expanding aggressively should be considered for scaling up.
How this parameter may affect capital – Events that are not relevant will wrongly increase the firm’s required capital by requiring capital for events that are no longer applicable to the firm. Both frequency and severity will be affected.
Scaling of relevant events
Parameter description – Any recorded event that is either too large or too small in respect of a firm’s current risk profile.
Challenge – All events must be challenged to assess whether or not they are of appropriate size. If not, they must be scaled using relevant factors such as, for example, revenues, staff numbers or products dispatched. This challenge is largely appropriate to external events, although it may also be applicable to internal events when circumstances have significantly changed.
How this parameter may affect capital – Events that are too large with respect to a firm’s current risk profile but are still included in the modelling data will lead to more economic capital required than is appropriate.
In our next blog Tony and John talk about the parameters relating to the modelling of qualitative data.
Mastering Risk Management by Tony Blunden and John Thirlwell is published by FT International. Order your copy here: https://www.pearson.com/en-gb/subject-catalog/p/mastering-risk-management/P200000003761/9781292331317
For more information about how Operational Risk software can help your organisation, contact us today on sales@risklogix-solutions.com
