JLT Re has reiterated its scepticism of risk modelling firms and questioned the reliability of modelling tools in the wake of natural catastrophes.
The reinsurance broker has again called into question the worth of modelling agencies after different firms put out vastly different loss estimates in the wake of Hurricanes Harvey, Irma and Maria (HIM) last year.
“The fact that such significant ranges were generated for HIM has raised questions over whether modelling tools can be relied upon to produce credible information for catastrophes in real time,” JLT Re said.
The broking house pointed to revisions made by AIR Worldwide for Hurricane Maria. The model provider initially estimated insured losses of between $40bn and $85bn but then it lowered the range to between $27bn and $48bn two months later.
Hurricane Maria, which devastated parts of the Caribbean, revealed the widest disparity between modelled estimates, with AIR’s initial top-end prediction almost three-times higher than the estimates published by rivals RMS and KCC.
There was also no overlap between the lower end of AIR’s loss picks and RMS’s top estimate.
JLT Re said that the disparity in estimates immediately after Maria showed that accuracy suffers when events bring unforeseen - and often unmodelled – consequences.
The broker said the gulf in estimates stemmed in part from differing judgements made over Maria’s windfield size at landfall, ground-up exposures in the Caribbean and resulting repair costs. It also pointed to differences in assumptions about insurance penetration in the region.
In addition, JLT Re said the race among modelling agencies to be the first to the market with estimates was having an adverse impact on the accuracy of those picks.
“The requirements and expectations of real-time information will only increase and it is important that catastrophe modelling companies strengthen their authority in this area: accuracy needs to be the focus so that decision-makers can be confident in the numbers,” the report said.
“Unfortunately for catastrophe modelling firms, failures endure far longer in the memory than successes and a significant credibility gap remains, justified or not.”
JLT Re said that practitioners in the insurance market should recognise this and “wait for a more considered view” rather than calling for immediate estimates.
However, the broker said that while the accuracy of modelled losses varied, there were encouraging signs given the levels of complexities involved. It added that the wealth of data collected during last year’s hurricane season - along with technological advancements - is likely to result in improved accuracy in the future.
“This highlights the inherent difficulties modelling companies face in predicting losses for complex hurricane events that strike highly populated urban areas, Josh Darr, lead meteorologist at JLT Re, said.
“These types of events often bring unforeseen consequences that cause losses to spiral,” Darr added.
“Vendor firms have in recent years drawn on important lessons learned during these events to recalibrate their models and incorporate a whole host of previously unmodelled perils.”