Efficient Model Lifecycle Management
8/10/2022
The key to good model risk governance is establishing an efficient model lifecycle management process. In the banking industry, small and medium sized firms may not have the infrastructure and resources to establish a dedicated model governance group. After the initial validation of a new or significantly updated model, the model becomes active, but model risk management does not end there. Models typically spend the majority of their lifecycle in the active state and need to be monitored on an ongoing basis to determine that they remain fit for use. Ongoing monitoring can be a time-intensive activity and it is crucial banks create an efficient workflow, especially given limited resources.
Ongoing monitoring
Ongoing monitoring refers to more than just monitoring the performance of a model. For example, a risk management team needs to track all stages of the model’s lifecycle. Among other aspects of model governance, this includes monitoring the status of model findings, timing and status of periodic assessments and revalidations, and approved model users and uses. In addition, as part of ongoing model governance, model risk management needs to periodically report on the status of both individual models and the entire model inventory to model risk committees at both the division and corporate levels.
To facilitate an effective ongoing monitoring process, we need to pay special attention to dates. When was the last performance report? When are the findings due? When was the last time the model was assessed? When was the last time the model had a revalidation? All this information must be tracked. For management reporting, you need to know: the number of open findings, whether the findings are overdue, and the status of finding resolutions. A model risk management team needs to track these important dates and to know when actions need to be initiated. Without a workflow built around this information, the monitoring process can become overwhelming, and can lead to violations of model risk polices and procedure. This, in turn, will lead to increased scrutiny and the issuance of findings by auditors and regulators.
Model inventory
A well-designed model inventory facilitates tracking of the information mentioned above. At a minimum, a model inventory should track the name and risk rating of the model and all approved uses and users of the model. In addition, the status and schedule of performance reporting, findings, and ongoing assessments and revalidations must be tracked. To assist the model risk management team, the inventory should be designed to allow reports on the status of a model or the aggregate inventory to be generated quickly. At the beginning of a quarter, a model risk manager should be able to generate all upcoming activities that require action by model stakeholders during the quarter. This would allow the model risk management team to prioritize and schedule activities to meet all due dates required by model risk policies. Often, small and medium sized firms rely on spreadsheets to implement their model inventories. Depending on the size and complexity of the model inventory, this might not be adequate to facilitate an efficient workflow. Overall, expending resources to design and implement a more sophisticated inventory at the start, could reduce costs and workload down the road.
Model lifecycle – active model
Most approved models spend 90% of their lifetime in the active state, undergoing ongoing monitoring. Once the model is developed and initially validated (see 1 -3 in Figure 1), it can be rejected (see 4 red arrow in Figure 1) in which case the model goes back to its development stage (see 2 in Figure 1) or approved in two ways: without findings (see 6 green arrow in Figure 1) or with findings (see 5 yellow arrow in Figure 1 ) that will need to be addressed. In the meantime, while the findings resolution is going on you have an Active Model (see 7 green box in Figure 1).
The model risk management team will spend a significant amount of their available time monitoring the model in its active phase (see A in Figure 1). A model typically has periodic revalidations (see B in Figure 1 ) at least annually depending on the risk of the model, as well as periodic assessments (see C in Figure 1 ), and reporting (see D in Figure 1). Reporting on the model must be provided to the Model Risk Committee at a minimum, and depending on the risk of the model, the reporting might also be provided to the Board. If model performance deteriorates, the model inventory should facilitate efficient reporting to all model stakeholders. The reporting should include findings resolution status, model performance, and the outcomes of revalidations of the model. In this phase of the model lifecycle (see 7 green box), where the model is active, there is lots of information that the risk management team needs to know: specifically, what to do and when.
The active model is a very important part of the model life cycle which needs to be tracked. The end of the model lifecycle will eventually arrive when you retire or redevelop the model completely, depending on its performance (see 9 yellow arrows in Figure 1).
Figure 1: Model lifecycle
Source: Kevin D. Oden & Associates
The key to well-designed risk management and governance is establishing an efficient workflow process and understanding the importance of managing the ongoing workflow and not just the initial validation. Therefore, small and medium sized banks with limited model governance resources need to consider whether a spreadsheet-based inventory is adequate for their needs.
RMA Model Validation performs many initial validations and re-validations for firms with limited resources, in partnership with Kevin D. Oden and Associates. Schedule some time with RMA to learn more.
Greg Brozak, Ph.D., Senior Quantitative Analyst
Greg Brozak is a quantitative finance professional with 25+ years’ experience in developing and validating financial and risk management models across a wide range of product types and risk factors. He has broad experience in model risk management and model governance. He has led model development and model risk teams at large financial organizations.
Greg’s areas of model expertise include mortgage and credit models, interest rate/ALM modelling, stress testing, operational and climate risk models. Greg also has extensive experience setting up and overseeing model governance frameworks. Greg has a PhD from Northeastern University, an MA from University of Buffalo, and a BS from Queen’s College CUNY.