# Teaching exercise: what a forecast map can and cannot tell you

About 30 minutes, for a class in climate science, forecasting or research methods. Uses the ENSOFORECAST review edition: the explorer (`index.html`, Figure 1) or Python 3 with `exercise/read_the_fields.py`. Credit the archive (Lamont-Doherty Earth Observatory climate group / Columbia IRI Data Library), the model (Zebiak and Cane 1987) and the transfer (Aaron Kaplan, IRI).

## Steps

**1. What is normal?** The maps show anomalies. Write one sentence defining an anomaly. The archive's reference period is not documented: name one comparison that is still fair and one that is not.

**2. Two observed winters.** In Figure 1, compare the observed maps for December 1982 and December 1997, or run:

```
python3 exercise/read_the_fields.py 1982-12 LDEO1 12
python3 exercise/read_the_fields.py 1997-12 LDEO1 12
```

Where along the equator is each observed peak, and how large?

**3. The tempting conclusion.** The LDEO1 12-month field shows strong warming for 1982 and little for 1997. Write the headline a hurried reader might write. Then list what you would need to know before believing it.

**4. Dates.** For the 1997 12-month field, which months did the six model runs start in, if the archive follows the convention in LDEO's 1997 bulletins? When was the field issued? What does the archive actually record?

**5. Two products.** What do LDEO1 and LDEO2 refer to, according to the December 1997 bulletin? Why is "LDEO2 is newer, so better" not a safe inference?

**6. Probabilities.** Why can't you report "the model gave a 20% chance of El Niño" from these fields? What would you need?

**7. Design a fair test.** Using the coverage table and `EVALUATION-PROTOCOL.md`, list the decisions a whole-record comparison must make before computing anything.

## Answer notes

1. An anomaly is the departure from the average for that calendar month over a reference period. Fair: comparing fields within the archive, on the same months. Not fair without re-centring: comparing magnitudes with a modern index on a different baseline.
2. Observed December 1982 peaks at about +4.06 °C near 128°W; December 1997 at about +4.59 °C near 104°W.
3. "The model predicted 1982 a year ahead but missed 1997." Needed: when each field was made (the 1982 fields are retrospective, by the published chronology); how the model and its correction were tuned; the full record, not two months; fair baselines; and whether the archived 1997 fields are the ones issued.
4. Under that convention, July to December 1996. The issue date is unknown; it was after the latest start. The archive records only the target month and nominal lead.
5. The original initialization and the improved initialization of Chen et al. (1995), which the authors called LDEO2. Newer does not mean better on every month or lead, the archive does not confirm the labels' meaning, and in September 1997 the authors reported both versions struggling with the 1997 event.
6. The fields are single continuous composites with no stated error distribution. A probability needs a calibrated error model fitted and checked on the full record, and a definition of the event.
7. Matched months (316 shared by all files) and masks; area weights; region definitions; baselines that use only information available at the start; season and lead breakdowns; dependence-aware uncertainty; separate reporting for retrospective and possibly real-time periods; and reporting every result.

## Sources

Zebiak and Cane (1987), *Monthly Weather Review* 115:2262–2278. Cane and Lee (2013), p. 72. Zebiak and Cane, *Experimental Long-Lead Forecast Bulletin*, September and December 1997 (NOAA CPC). Chen et al. (1995), *Science* 269:1699–1702.