# Teaching exercise: reading a priced distribution without inventing a percentile

About 30 minutes, for an undergraduate or early graduate class in forecasting, macroeconomics or statistics. Uses the Gürkaynak–Wolfers auction-only edition. Students need either the explorer (`index.html`, Figure 1) or Python 3 and `exercise/where_did_it_land.py`. Credit the original research when you use it: Gürkaynak and Wolfers, NBER Working Paper 11929 (2006), doi:10.3386/w11929.

The aim is to separate four things that headline forecasts blur: the probability of a range, the shape of a distribution, where an outcome sits in it, and what a single case can and cannot show.

## Steps

**1. Before the reveal.** Open the December 2003 payrolls auction with the result hidden.

```
python3 exercise/where_did_it_land.py gw2005-nfpay-2004-01-09-2004-01-09 --hide
```

- Which range would you call the market's centre? Why might two people answer differently?
- What probability did the auction put on a result below 25,000?

**2. Reveal.** Run the same command without `--hide`, or press *Reveal* in Figure 1.

- The announcement landed in a range priced at 1.03%. Is that range price the right measure of surprise?
- The script reports a cumulative interval rather than a percentile. Why can't the data give an exact percentile?

**3. An ordinary month.** Open `gw2005-nfpay-2005-01-07-2005-01-07`. It was chosen by a rule: the first payroll release after January 2004 whose result fell inside the range containing the market's 50% point.

- The result's range was priced at only 7.5%. Reconcile that with calling the month ordinary.
- Why does choosing an example by a rule matter, and why does it still not make the example evidence of accuracy?

**4. A tie.** Open `gw2005-iclm-2004-02-26-2004-02-26`.

- The result is exactly on the boundary between two ranges. Which quantity is still well defined, and which depends on an unrecorded contract convention?
- Suggest two ways an evaluation could handle the 23 boundary cases, and what each would assume.

**5. Mass and width.** Open `gw2005-nfpay-2002-10-03-2002-10-04` and switch Figure 1 to *Per unit of width*.

- Which range has the tallest bar? Which is most concentrated? Why do they differ?
- Why can't you compare the tallest bar across two auctions to compare confidence?

**6. Which number?** The auction settled on the December 2003 figure as announced: +1,000. Today's BLS series implies about +109,000.

- Which number should an evaluation of the auction use, and why?
- Name a research question for which the revised number is the right one.

**7. Discussion.** Three examples cannot show whether the auctions were calibrated. Using `EVALUATION-PROTOCOL.md`, list what a fair whole-sample check must decide before computing anything.

## Answer notes

1. The two heaviest ranges were 125–150k (13.0%) and 100–125k (12.8%); the market's 50% point fell in 150–175k (48.08% of the probability below 150k, 58.28% below 175k). "Centre" can mean the heaviest range, the median range or a mean, which requires within-range and tail assumptions. Below 25k: 2.00% + 1.03% = 3.03%.
2. A range's price depends on its width and on how many ranges there are, so it is not a surprise measure on its own. The cumulative interval is 2.00–3.03%: the announcement was in the bottom 3% of the market's distribution. An exact percentile needs an assumption about how probability is spread within the 0–25k range; the source prices only the whole range. Under the paper's assumption (uniform within the range, p. 24) the position would be 2.00% + 1.03% × 1/25 ≈ 2.04%, using where 1,000 sits in 0–25,000; the interval's midpoint, 2.5%, is a different and unsupported choice.
3. A distribution spread over 16 ranges leaves little on any one. The cumulative interval 49.0–56.5% shows the result was near the middle. A rule prevents picking an example to suit a story, but one case is still one case; and this rule selects on the outcome by design.
4. The cumulative mass of the ranges below the boundary, 63.28%, is well defined. Whether it is the probability of a release strictly below 350,000, and which range paid out (14.77% below or 14.17% above), depends on how the contracts settled ties, which is not recorded. Options: treat the boundary as a point in the cumulative distribution; or report results both ways as bounds.
5. Tallest bar: 50–70k, 11.56%. Most concentrated: 0–10k, 8.67% over a range half as wide (8.67% per 10,000 jobs against 5.78%). Bar heights depend on how the ranges were cut, which differs across auctions.
6. The announced number, because it determined payoffs (paper, p. 5). A revised number suits questions about the economy itself, such as how much employment actually grew.
7. At least: rescaling or not; within-range treatment; tails; boundary cases; by-indicator versus pooled results; dependence between nearby releases; what would count as a deviation; and that all results are reported.

## Sources

Gürkaynak, R. S. and Wolfers, J. (2005), FRBSF Working Paper 2005-26; (2006) NBER Working Paper 11929. Current payrolls: BLS series CES0000000001, retrieved 22 October 2026.