Edition 3 | Week 1 of Reserving | Actual vs. Expected | The Scatter | Framing & Baseline

The Decision Question

This edition looks at how to show the actual experience (or new ultimate) versus the expected ultimate (previous ultimate) claims when the question is:

Can I trust the reserving outcomes? Are predictions broadly calibrated or are there systematic bias?

Audience:  Management

Default Practice

Scatterplot with the expected / predicted ultimate on x-axis and the actual or updated expected on the y-axis.  Each point typically represents a total reserve for a line of business, allowing management to assess calibration and directional bias at a financially material level.

This scatter plot is used as a high-level assurance check on reserving outcomes. It compares the previously expected ultimate claims with the subsequently observed or updated ultimates to assess whether results are broadly aligned with expectations. By plotting expected outcomes against actual experience at an aggregated level, the chart allows management to judge calibration and directional bias without engaging in technical detail.

Points clustering around the diagonal suggest outcomes are broadly consistent with prior expectations, while systematic deviations above or below the line may indicate under-(engineering) or over-estimation (property) at a portfolio or line-of-business level. The purpose of this chart is not to explain why differences occurred, but to provide a first-pass indication of whether the reserving process is producing outcomes that can be trusted.

The table outlines upgrades with the improved chart following below.

Commonly Used Version (what is usually shown)

Best Practice Version (same chart, disciplined upgrades)

Typical characteristics Expected ultimate on x-axis, actual or updated ultimate on y-axis One point per line of business Single 45° reference line Minimal labelling No explicit decision framing Why it works Familiar and fast to produce Visually clean Signals reassurance without explanation What it silently assumes Aggregation is sufficient Proximity to the diagonal implies accuracy All deviations are equally informative

What changes (without changing the chart type) 1.      Explicit tolerance bands ±X% or ±Y currency bands around the diagonal to make “acceptable vs concerning” visible. 2.      Clear aggregation declaration In axis or chart title. 3.      Materiality-aware emphasis Larger points for larger % deviations. 4.      Outlier annotation In this chart there were few enough lines of business to annotate all points. When there are too many, only annotate points that breach tolerance. 5.      Decision-oriented caption States the question being answered: “Used to assess high-level calibration and directional bias.”

Best practice does not replace the scatter plot — it constrains it.

Where the Scatter Plot Breaks Down

Proximity to the diagonal is over-interpreted.

Closeness to the 45° line is often read as evidence of accuracy. In reality, reserve movements can cancel at an aggregate level, and the chart does not distinguish between stability and coincidence.  Timing is ignored - early adverse development and late emergence appear identical once aggregated, even though they carry very different implications for model adequacy and risk.

The chart invites conclusions it cannot support.

While it can indicate potential bias, it cannot explain causes, assess development behaviour, or validate assumptions. Used in isolation, it risks becoming a comfort chart rather than a diagnostic one.

Decision Boundary

The scatter plot is effective at signalling whether a question should be asked, but it is not sufficient to answer the question itself.

The scatter plot works when your goal is:	Assess high-level calibration of reserving outcomes against prior expectations Identify obvious directional bias at a portfolio or line-of-business level Provide management with a first-pass assurance view without technical detail Screen for outliers that warrant further investigation Confirm if/that results are not materially surprising in aggregate
You may need something else when your goal is:	Understand when deviations emerged during development Explain why reserves strengthened or weakened Distinguish structural issues from offsetting effects Assess model behaviour by accident year or development age Support governance, audit, or methodological decisions Communicate uncertainty rather than point alignment