Operational risk and control assessments are often the first process that a firm uses to conduct operational risk management. Frequently the assessment is carried out without an operational risk management framework in place and without much thought being given to good corporate governance around the multiple interlocking processes of operational risk management.
Objective
Few now doubt the advantages of having a documented operational risk policy. It allows senior management to communicate to all staff the approach of the firm to operational risk management. As such, the policy should be approved by the Board of Directors. Alternatively, in some firms, the Executive or Management Committee may wish to approve the policy document or at a minimum, review and comment on it prior to Board approval.
Performing an assessment
There are a variety of views on how to perform an operational risk assessment. Options include:
- A third party review, which uses a central understanding of critical objectives and processes together with an independent validation of assessments.
- Facilitated assessments (conducted by an outside consultancy, risk management and business managers), which uses the central understanding to identify and agree the business risks with the business. The effectiveness of internal controls is also documented and action plans are agreed where necessary.
- Self assessment (conducted by the business managers), which uses the detailed knowledge of people in the business to identify the business risks and to agree on their monitoring. As with facilitated sessions, control effectiveness is also assessed and action plans put in place to enhance ineffective controls.
The three methods of operational risk assessment above have an increasing level of business benefit although these are balanced by an increasing level of process sophistication. In particular, a self assessment (being conducted by the business itself) gives the best platform for cultural change. (It should be recognised that most firms will, necessarily, go through a period of cultural change whilst embedding operational risk management into the structure and decision making of the firm).
Any of the methods above can be used for risk assessment, control assessment or risk and control assessment. Commonly, firms start with an assessment of risk (initially evaluating the risk after allowing for the mitigating effect of the controls). Both stand-alone assessment methods give some value although neither gives the value that can be derived from a combined risk and control assessment. For example, there is generally very little shared assessment in control self assessments, even when the business reviews the process for the assessment of control effectiveness. By contrast, in risk and control assessments carried out by the business there is usually a natural element of co-assessment in order to ensure consistency.
Possible methodologies
There are a variety of practices that can be used to carry out any of the three methods of assessments. These include:
Workshops, which can be very effective and efficient in a firm that is open to discussion and challenge. However, the drawback is that a first risk and control assessment generally takes a full working day to complete and it is therefore necessary for all workshop attendees to be absent from their desks for the day.
Interviews, which work very well in a firm that is used to one-to-one discussion of issues. Interviews are relatively inefficient as a certain amount of iteration is necessary in order to obtain agreement on the risks and controls. They are nevertheless effective when an entire cadre of staff cannot be spared or is not available for a full day workshop.
Questionnaires, which can be easy and quick although these generally need strong management and significant communication skills in order to achieve a cohesiveness to the wide ranging results that can be a consequence. Good design of the questions is fundamental to obtaining an outcome that has business benefits. This is often harder than it may appear as risks, control failures and indicators can easily become confused in the mind of the person answering the questionnaire.
Why do assessments go wrong?
Given the variety of views on who should perform a risk and control assessment and on what method to use to carry it out, it is not surprising that many firms unknowingly chose a sub-optimal approach. After a period of attempting to implement their chosen approach, a frequently asked question is “What is wrong with our approach to risk and control assessment?”.
There are a number of reasons why risk and control assessments go wrong. At a high level, these include cultural issues, administrative hurdles and value perception.
1. Cultural Issues
As noted in previous articles, a common risk language is important for a consistent approach to operational risk management across the business. It is impossible to aggregate risks, compare risk exposures or analyse control profiles without an agreed view of common risk terms. All three actions are typical uses of a risk and control assessment. An inconsistent quality of identification can also be a result of a lack of understanding of risk terms or alternatively it can result from a lack of application of a risk audit process to the risk and control assessment results.
Another common cultural issue is the lack of support from senior management for the risk and control assessment process. This is often characterised by a lack of attendance by senior management at risk and control assessment workshops or by sudden departures after 30 minutes or 1 hour. Alternatively the firm’s appraisal or review mechanisms may not take into account good (or bad) risk management by the employee being evaluated.
A further typical cultural issue is the use of operational risk management to reduce risk rather than managing it appropriately to the organisation. Some firms aim for a perceived level of best practice, whereas operational risk management should be focussed on managing risk at a level suitable to the firm’s size and substance.
2. Administrative Hurdles
Risk and control assessments are often unnecessarily paper intensive. The implementation of this type of assessment is very difficult across regions of the world and particularly across different cultures. It is also burdensome to maintain and can be orientated towards a policing role, looking for a fault and assigning blame rather than forward looking and proactive.
3. Value Perception
Sufficient thought must be given to the reporting of risks and controls so that they can be monitored. This will be addressed further in later articles although it should be clear that inadequate reporting provides limited business value. Additionally, if the results from the risk and control assessment are not linked to other users of the information there will be limited leverage possible. There is also a much greater perception of the value from a risk and control assessment when the action plans generated (either to enhance controls or add new controls) can be seen to be followed up and implemented. The greatest value to be obtained from operational risk and control assessments is from linking them to losses, key indicators and mathematical models. These links will be addressed in later articles.
Carrying Out Assessments
1. Level
The level at which an assessment is to be carried out should first be decided. Many organisations first look at the major processes undertaken and assess the risk and controls over these. Other organisations leave the major process risks until the strategic risks and controls have been assessed. This second practice has the advantage that the major processes can then be placed into the context of the business objectives and their risks and controls, rather than trying to back fit the process risks into the strategic level at a later date. The other major advantage of starting with the business objectives as the first level risk drivers is that there is rapid buy-in from the most senior management in the firm as they are responsible for achieving the business objectives and any obvious assistance is always appreciated.
2. Approach
Risk and control assessments can be carried out at using two different assessment approaches which can also be combined. The most common starting point is to assess the risk after the controls (i.e. after taking into account the mitigating effect of the controls). This is known as net or residual risk assessment. However, losses generally occur after controls have failed and therefore net risk assessment by definition does not give any values for the likely loss that the firm will suffer when the risk event occurs. Only values for ‘expected’ losses are measured. This problem can be overcome with the use of gross risk assessment followed by an assessment of the controls. The risk is assessed before taking into account the effect of any controls imposed by the firm, as these will have failed when the risk occurs.
3. Enhanced Approaches
As a firm progresses along the risk and control assessment path, it sometimes combines the above two approaches by assessing risks at a gross and net level as well as assessing the mitigating controls. Often an assessment of the risk at a ‘target’ level (i.e. after any remedial action) is also made. In any of the approaches, the action plans for enhancing the perceived defective controls are also identified. The owner of each action plan is identified together with a brief description of the plan, its expected completion date and any cost involved.
4. Owners
As well as risks and controls being assessed, the owner of each risk is generally identified as is the owner of each control. It is common in a first pass through of the owners of the risks in a strategic risk assessment that the CEO is the owner of the majority of the risks. However, once the Board has been challenged, the CEO normally owns a number but not the majority of the risks to the business objectives.
5. Scoring
Following the identification of the risks and their owners, the risks are usually scored. Five years ago, a risk would have been scored for its severity – a one dimensional value. Today, almost all firms use two dimensions – likelihood and impact. Controls are also today often scored in two dimensions (typically, design and performance) rather than simply the effectiveness of the control. The scores of the risks and of the controls are usually arranged on a scale. Some firms use 1, 2 and 3 or low, medium and high. Others use up to ten levels. It is useful to use an even number of levels so that there can be no sitting on the fence by using the middle level for most risks and controls. Probably the most common number of levels is four or six – with four levels being high, medium high, medium low and low.
The scale for likelihood is linked to the likely rate of occurrence of the risk and that for design and performance is linked to the likely failure of the control. However, the impact scale requires some thought as different firms use different impact criteria such as the impact to annual revenues, three year plan profits or the share value.
6. Cause, event and effect
Another consideration when carrying out a risk and control assessment is to isolate the risk events (i.e. what you want to capture) from the risk causes, the risk effects and the control failures. Most methodologies for risk assessment (see the previous article) will produce a combination of all four risk types unless some guidance is given. It is the risk event that is required in a risk and control assessment as the risk event is immutable whereas risk causes and effects change over time. If controls are applied to changing circumstances, the controls may become less effective because of the shifting conditions rather than the efficiency of the control itself.
7. Control assessment
The assessment of the controls can be carried out either on the cluster of controls that mitigate a risk or on each control within the cluster. The greatest business benefit is derived from assessing each control as a control may operate on several risks and its varying effects can therefore be judged. Additionally, controls are often identified as either preventative or detective controls to aid the design of action plans over the further mitigation of a risk.
Monitoring Assessments
Assessments are monitored in various ways by firms. Tables, heat maps and radar charts are common methods. The monitoring is also for varying reasons, from identifying the highest risks and the poorest controls through the effectiveness of controls to the degree of over and under control of the risk. The scoring used in the assessment is also used in the monitoring. Typically, the likelihood and impact scores are linked together to give a composite value that can be used for comparing one risk with another.
There are varying levels of sophistication in risk monitoring, even when simple concepts such as heat maps are involved. The first figure below shows the drawbacks of using fixed values for the boundaries of the impact and likelihood. A risk can be categorised as a major risk (because it falls within that square) even when it is calculated as a minor risk (and therefore requires significantly less attention).
1. Heat Map of fixed boundaries with four levels of impact and likelihood
2. More flexible heat map with varying boundaries
Tables and radar charts are also becoming more sophisticated and the reporting article later in this series will contain further details of risk assessment monitoring.
Software Tools
No article on risk and control assessment would be complete without reference to the many software tools that exist today for capturing risk and control assessment data. There are several types of risk management software:
- Purpose built risk and control assessment software that focuses mainly or solely on risk and control assessments using any methodology (sometimes together with process mapping and action plans);
- Questionnaire based products that are focused on completing the assessment through questionnaires;
- Analytical tools that are mainly based around complex mathematical algorithms; and
- Fully integrated operational risk products that capture risk and control data as part of an overall approach to operational risk involving indicators, losses and modelling as risk, controls and action plans.
The choice of tool will depend on how comprehensive an approach is required by the firm. However, it is easier to buy a comprehensive tool and grow into it than it is to change software part of the way through an operational risk management programme. There are a number of points to consider when implementing operational risk software:
-
- fit with the firm’s business culture: select software that works in the same way as the firm, e.g. if the firm generally uses workshops and brainstorming, do not select a questionnaire based tool;
- fit with the firm’s IT culture: support will be required from IT so ensure that it is involved early in the process; and
- fit with the firm’s risk culture: risk management software that focuses on controls is not going to work well if the firm is committed to a combined risk and control assessment.
Summary
Risk and control assessment is a fundamental part of operational risk management. Although there are many hurdles to carry it out well, it can be done and has been done by many firms. The best implementations of risk and control assessment are giving real business benefit and are fully supported by the Board and senior management of the firms involved.
Tony Blunden
