Scenario Governance

  • Subscribe to updates

  • Privacy
  • This field is for validation purposes and should be left unchanged.

Governance points to consider when developing risk scenarios 

Tony and John discuss how operational risk software can manage risk scenarios, and how to avoid availability and motivational bias.

Taken from: Mastering Risk Management 

As with any risk methodology or procedure, it is vital to ensure that the governance relating to the methodology is documented and understood. Good governance will enable the board and senior management to guide and direct the risk scenario strategy and to review its effectiveness. From a practical perspective, this will involve: setting the scenario objectives; defining the scenarios; discussing and promoting the discussion of the results of the scenarios; assessing potential actions and making clear decisions based on the results; fostering internal debate on the results of the stress tests and scenarios programme as a whole; and challenging prior assumptions such as cost, risk and speed for raising new capital or hedging/selling positions. All of these governance points may be taken up by the board in its meetings or may be delegated to a board scenarios subcommittee, which reports back to the full board. 

Risk scenarios should enable a firm to understand the sensitivities of all of the elements of the firm’s risk exposure, as set out in the risk framework. .This includes: 

  • Clarifying interactions and causal relationships between risks and controls
  • Acting as a challenge to the subjective nature of risk and control self-assessments
  • Compensating for the lack of internal loss data
  • Allowing adjustments to the likelihood and impact assumptions in risk assessments
  • Allowing adjustments to the design and performance assumptions in control assessments. 

Developing risk scenarios

Points to consider before development 

The crisis management team

When management focuses on a major event, there is a loss of focus on other controls so that the firm is much more likely to experience another major event. A typical mitigant is the firm’s crisis management team, which should preferably exclude the CEO, and allow essential business line management to continue focusing on the business, confident in the knowledge that other members of the senior management team are sorting out the crisis. 

Combinations of events over a period of time

When developing a set of scenarios it is important to consider more than one major event happening over the period of the scenario. Scenarios do not involve a combination of events at one point of time, but should be generated on the much more realistic assumption that one major event may be followed by another within a matter of months. 

Recognising and mitigating natural biases

In a working paper published in September 2007, the Australian. Prudential Regulation Authority (APRA) notes that there are conscious or subconscious discrepancies between a participant’s response when developing a scenario and an accurate description of their underlying knowledge. There are many biases, but they probably resolve themselves down to two generic types: 

  • Availability bias
  • Motivational bias

Availability bias refers to the ease or otherwise with which relevant information is recalled. A subset of it is overconfidence bias, where undue weight is given to a very small set of perceived events. Interestingly, it can be overcome by using two other forms of availability bias, partition dependence and anchoring. Partition dependence arises when respondents’ responses are affected by the choices they are asked to make, or the buckets into which their answers have to be put. Anchoring is the bias towards information presented in background materials to survey questions or within the questions themselves. 

The use of external loss data can help in inspiring scenarios which might otherwise have been overlooked and can therefore mitigate availability bias. However, availability bias can also affect the frequency assessments. The likelihood or frequency of an event may be overstated if the relevant event has occurred recently or if it has been personal experience. Conversely, the likelihood may be underestimated if the event has not been previously experienced. For example, someone who has previously been involved in a fire is more likely to overestimate the risk of a fire. On the other hand, firms often significantly underestimate the frequency of internal fraud since relatively few internal frauds are actually detected. It is therefore important to bear in mind availability bias and, if necessary, adjust for it, especially when using external loss data. Taking in these data may give a false sense of having covered most eventualities. 

Motivational bias arises when a participant has an interest in influencing the result. It can lead to the understatement of frequency and impact, the understatement of the effectiveness of controls, and the understatement of the uncertainty surrounding the assessment made.  It is very common, for example, for control owners to overstate the efficiency and effectiveness of the controls for which they are responsible. When a control assessment is presented to the risk owner and business line manager, a very different view of the capability of the controls mitigating that risk often emerges. There is also an incentive to understate potential losses in order to reduce the capital required to run the business line of the firm; or simply to provide a rosier view of the riskiness of the business line to the firm. Making scenarios subject to peer review, in addition to the formal challenge process carried out by risk management, is a good way to reduce the influence of motivational bias. 

The influences of these biases can be seen in likelihood assessments. Estimates for likelihood can be particularly difficult when considering rare events. It is difficult to distinguish between a 1 in 1000 chance in one year for the event and a 1 in 10,000 chance. Both events are beyond most people’s comprehension. Availability bias is almost inevitable in these circumstances, particularly when using external likelihood data, of which there will be relatively little. 

Impact assessments of scenarios are also prone to problems as most people find it difficult to think in terms of probability distributions. Ideally, several impact values for the scenario will be helpful at specified percentiles along the distribution. This is known as the percentile approach. 

An alternative but more difficult approach is the interval approach, which consists of frequency estimates for a series of distinct impact ranges. This is conceptually similar to a risk and control self-assessment approach, although obviously different in details. 

Bias matrix

An easy way to identify that a bias exists in your set of scenarios is to develop a bias matrix. This is a spreadsheet with each column representing a scenario and each row representing a key risk. The key risks that make up part of a particular scenario are marked on the spreadsheet. When the set of scenarios has been analysed, it will be clear if there is a bias in that set. 

Assumptions

Scenarios will be used in conjunction with the other techniques used by the firm such as risk and control self-assessments, forecasting and strategic analysis, resource allocation and business planning. As a result, the assumptions which form the base case for the scenarios should be consistent with the assumptions in the other techniques and should broadly reflect events envisaged in the long-term plans made by the firm. 

Environment

Scenarios should also take account of the broader business environment. Political, financial/economic, social, technological, environmental and legal factors will inevitably affect the scenarios over the period they cover. The scenarios should be challenged by each of these factors to ensure they have been fully incorporated. 

Historic or hypothetical data

Scenarios can be developed using either historical real data or hypothetical data. When using historical data, care must be taken to reflect changes to the internal and external environment within which the scenario is planned. When using a hypothetical approach, care must be taken to devise a scenario which is sufficiently extreme but still plausible. Either way, the scenarios must be consistent with the firm’s risk and control profile as there is no value in analysing stresses which will not apply to the firm. 

Spurious accuracy

There is a temptation when undertaking any calculation to be as accurate as possible. This is also the case when calculating the effect on a firm of stress tests and scenarios. However, scenarios are exceptional but plausible events. Any representation of a scenario will inevitably be approximate in terms of its impact on the firm as exceptional or extreme events are impossible to predict with accuracy. 

The monetary impact of a particular stressed key risk on a firm may well be approximated to the nearest million monetary units by senior management or subject matter experts. It is therefore pointless attempting to be accurate to the nearest monetary unit when calculating the overall impact. If subject matter experts have used millions or tens of millions, risk management should not attempt to be more accurate. 

In our next blog Tony and John explain how to develop a set of practical scenarios.   

Mastering Risk Management by Tony Blunden and John Thirlwell is published by FT International. Order your copy here: https://www.pearson.com/en-gb/subject-catalog/p/mastering-risk-management/P200000003761/9781292331317    

For more information contact us today on sales@risklogix-solutions.com