How speed can be the enemy of usefulness when it comes to reporting
This is the last in a series of four blogs about the ways in which common shortcuts can undermine core operational risk elements within financial services firms…..more to come
You can view the other blogs here:
The Shortcuts Trap – Loss Events Reconsidered
The Shortcuts Trap – Risk and Control Self Assessments
The Shortcuts Trap – Key Indicators Under Fire
Of course, a very important reason for collecting operational risk data is to be able to report on it to the business, senior management, and the board. However, even when using risk management software not enough thought is put into reporting, the process of reporting, or who the reports are for.
Op risk teams use a variety of shortcuts to make reporting easier to produce, or quicker to get out the door. Sometimes op risk teams see reporting as a less important deliverable to the organization. All of this leads to suboptimal risk management outcomes. A significant chunk of the value that an op risk team delivers to its organization is the intelligence contained in the data that it collects.
So just what are op risk teams getting wrong when it comes to reporting? The following shortcuts are frequently encountered:
- Including too much data and having too many pages – It can be tempting to include almost every piece of op risk data in a report. There is so much interesting data, right? And who has the time to sift through it all? It’s not uncommon to find a thick bundle of operational risk reports – more than 100 pages – sent to the op risk committee each month. These are usually in a range of formats, including spreadsheets, slideware, and text documents. This is too much information for the op risk committee to digest. Instead find out what the op risk committee want to see each month, and structure professional reports around that. Deliver intelligence to them that they can use to better understand the risks the firm faces.
- Not thinking about the decisions being taken on the basis of a report – Operational risk reports will have different audiences, and those audiences will have unique responsibilities within the firm. When they read an op risk report, they are looking for information that will help them take those decisions. Structure op risk reports so that each audience is being given the intelligence it needs to make informed choices.
- Not considering the agenda of the person receiving the report – It’s important to consider the authority that an individual has when structuring a report. For example, what decisions is an individual actually allowed to make? What aspects of the organization might an individual have formal regulatory responsibility for? Alternatively, what areas might an individual have influence over? And what elements are beyond the formal scope of that individual – do they need to be receiving op risk intelligence on those areas, or not?
- Constructing a ‘dashboard’ that puts together the easy data, rather than the meaningful data – Frequently operational risk teams will create a dashboard that simply contains separate lists of the top 10 risks, top 10 KRIs, and top 10 losses for a month or a quarter. But how useful is this information for decision-making? Three individual, disconnected lists provide very little context for understanding the nature of a risk and how it might be better managed. Instead, op risk teams should consider providing information about the relationship between risks, KRIs and losses. For example, a list of the top risks should include, for each risk, information about the KRIs and losses that is relevant. It may take additional time to put together this more complex report, but not doing so could be dangerous for the organization – decision-makers just don’t have enough information to work with.
- Using report formats which are not suited to the data – One common shortcut is to shove data into a report format, such as a pie chart, whether or not the format helps to illustrate the story the data is telling. For example, pie charts work really well when there is a limited number of data points and there are significant differences between the points, so there are a few “slices” of the pie that are obviously different in size. Pie charts work less well if there are many small data points, or if the slices look to be all about the same size. Unless such a pie chart is heavily annotated, it’s difficult to be able to see the data in sufficient detail, for it to be useful in decision-making.
- Handing over op risk data NOW! even though it’s incomplete – Sometimes the business, or a senior executive, or another person in the business can demand seeing the op risk data immediately, disregarding the state that the data is in. The pressure to hand over the data – even though it is not complete – can be enormous. Turning over the incomplete dataset can seem to be the easy way out of a loaded situation. However, it’s best to hold out here and not turn over the report. An unfinished report could contain data that is incorrect, or have data missing. Any decisions that are taken on the basis of this report may therefore be poor ones, and could have significant repercussions for the firm.
- Waiting until the op risk team is absolutely sure that the data is correct – On the other hand, op risk teams can be so cautious about falling foul of the risk of submitting the wrong data that they double and triple check data. It is easier to hold onto the data than to risk turning in the report. The result is reports that are not handed over in a timely fashion. Boards and senior managers find out about risks too late to effectively mitigate them, or else they make take wrong decisions based on stale data.
- Failing to use operational risk data reports for risk appetite discussions – Operational risk reports should link the data directly back to the stated risk appetite. For example, what is the relationship between any red-amber-green traffic light system a report might use and the risk appetite? Or, which loss events take the firm beyond its risk appetite and which do not? How do the KRI thresholds reflect the risk appetite, and how is that reported? The board and senior management should have an idea about how well the organization is keeping within its risk appetite from the report. The report should connect the dots to form a picture of the risk appetite.
- Not taking report recipients through a new report – Reports change from time to time in format and content, and it can be easy to just make changes and call it a day. It’s really important to talk recipients through a report when there are changes to its format and content so that they understand what it is they are looking at. Some report graphics – for example, “spider-grams” or “bow ties” can hold extremely meaningful information, but at first glance appear indecipherable. As well, individuals might not understand the relationships between two different data sets. The organization will take better decisions with the data – and perceive the op risk team as adding more value – if they understand the intelligence in the reports.
In short, operational risk teams need to think more strategically about the reports they produce, and how they produce them. Op risk reports should deliver key intelligence to the business, and underscore the value that the discipline of operational risk, practiced well, can bring to the business.
