How Sure Are We? — October 2, 2026
What this is. A check on research/20 and the October 2 report: which claims are solid, which are judgment, how confident we are in each and why, and what we did when confidence wasn't good enough. Written after the pre-publication review (review_2026-10-02.md) and the fixes it forced.
1. Short answer
- The facts are solid. Every price print, rain total, canal flow, drought number and USDA price in the report was read from a saved primary file and checked a second way. An outside reviewer re-checked them against the same files and confirmed them.
- The price estimate ($275, range $210–340) is honest but not precise. We are confident the right number for tested Good/Premium large squares sold from a Southeast Colorado farm today is somewhere in $250–300. We are not confident it is $275 rather than $260 or $290. No sale of that kind printed, so nothing local can tell us.
- The forecast path (flat to January, then lower) is low confidence. It is close to "no change," which is the most our record supports.
- The first draft had real errors (17 review findings, 3 serious). They were fixed before publication. That the draft needed them is itself a reason to keep the LOW label.
2. Claim by claim
| Claim | How it was checked | Confidence | Why not higher |
|---|---|---|---|
| USDA August prices: CO alfalfa $235, all-hay $231, KS $127 | grading script + manual table read + reviewer | Very high | — |
| Sep 24 Colorado report: one SE per-ton farm sale, 25 t Fair at $250; no ≥100-t alfalfa sale in the state | two reads of the PDF text + parser + reviewer | Very high | — |
| Rain at Lamar Airport: Jul 3.18", Aug 4.47", Sep 4.50", Oct 1 0.96" | two independent ACIS queries + reviewer | Very high | a first chart draft showed July as 0.00 (saved file started Aug 1); caught on review |
| Fort Lyon Canal back on since Sep 21, 405 cfs Oct 1; Jun–Sep ~25% of normal | DWR telemetry twice + FLCC's own page; monthly records | High | the monthly records contain placeholder days, so the two methods share inputs |
| Bent County out of severe drought; OK/TX worse | USDM API; DSCI recomputed for all seven states | High | the map predates the Sep 30 storm |
| Today's price: $275 G/P, range $210–340 | 8 anchor variants by script ($271–308, median $277 after the OK Oct 2 report; $262–308 / $273.5 before it); script reproduces Sep 19 numbers exactly; reviewer reproduced the spread | Low (label), but bounded — see §3 | no local print; geography split; one Oklahoma lot moves the main anchor $28 |
| Winter: $280 Nov–Jan | base rate (+6%, 5 of 6 seasons) cut by judgment for the rain | Low | weather has no measured skill (L10); our state-row judgment beat persistence this cycle only because of Colorado; Kansas went the other way |
| Futures, Kansas late-September prices, US–China truce | delayed quotes / trade press | Medium (labeled secondary) | not primary; not used in the estimate |
3. Why the price range is honest — three separate checks
The ±$65 range is our calibration floor, set from past misses. This cycle we tested whether it is too wide or too narrow by three methods that don't depend on it:
- Spread between sales. The 19 Good-or-better large-square sales of 100+ tons since Sep 10 have a tonnage-weighted standard deviation of $32. Two standard deviations is ±$64. So even if we knew the regional average exactly, a single real sale could land anywhere in about ±$65 of it.
- Spread between methods. Eight defensible ways of averaging those sales run $271–308 (after Oklahoma's Oct 2 report; $262–308 before). All sit inside $210–340.
- Our own record. Price-row errors so far: median $9.50, 80th percentile $32, worst $65 (n=8). Intervals have contained the outcome 9 of 19 times overall; for the six state-price rows, 5 contained it and 1 was NEAR (Kansas July, $126 vs $130–155).
Put together: we'd put roughly even odds on a real comparable Southeast sale landing within $255–295, and about 9 in 10 on $210–340. These are judgment probabilities, not measured coverage; the measured local coverage doesn't exist yet (0 comparable local outcomes graded). That is exactly why the label stays LOW (L9).
4. Where we weren't confident, and what we did about it
| Weak point | What we did | Result |
|---|---|---|
| Is the comp-set code right? | Rebuilt it (comp_set.py) and required it to reproduce the Sep 19 numbers before trusting it | exact match ($269.8 / 1,840 t; $268.7 / 4,050 t) |
| Did we cherry-pick the estimate? | Computed all 8 reasonable variants; took the median; reviewer independently got the same spread | $271–308, median $277 → $275 (morning, before the OK Oct 2 report: $262–308, $273.5) |
| How sensitive is it to rules? | Varied size cutoff (100/200/500 t), dropped Oklahoma, dropped the $170 lot, added rounds/delivered | $271–308 on the same product (morning: $262–308); rounds or Fair grades pull it to $245–249, a different product |
| Wyoming 1,600-t repeat | diffed the two reports; reviewer checked the volume line | repeats verbatim AND AMS counts it twice. Unknown if one deal; anchors shown both ways |
| North vs south: which market is Southeast Colorado in? | Split by region; then checked 65 same-week SE vs NE Colorado report pairs back to 2020 | North $283, KS/OK $239. SE CO has run a median $16 above NE CO in the same week (mixed lot types, so heterogeneous). No evidence SE CO prices at KS/OK levels |
| Old-crop and weather discounts | Searched the ledger and all current reports for same-report pairs | one old-crop pair (−37%), one bleached pair (−11%, different cuttings). Not enough to size a discount |
| Missing Kansas (Sep 29) and Oklahoma/Texas (Oct 2) reports | tried 5 methods for Kansas; Oklahoma/Texas posted later Oct 2 and are folded in (median $273.5 → $277, still rounds to $275) | Kansas still missing. Kansas Good-or-better squares are 1,800 of 8,790 t in the rolling set; a ±$20 move in them shifts the estimate ~±$4 |
| Local asking prices | web search for Arkansas Valley hay listings | none published (Front Range asks $340–375) |
| Our own draft | adversarial outside review before publishing | 17 findings, 3 HIGH, all fixed; review published unedited |
5. What would raise confidence (and nothing else will)
- A real Southeast Colorado Good-or-better large-square sale of 100+ tons. Base rate ~2.5% per report. Until one prints, the estimate stays a regional average.
- Graded local outcomes. L9 needs 20 comparable local outcomes and 70% coverage before the label can rise. We have zero: no Southeast Good-or-better large-square sale of 100+ tons has printed since grading started. (The three Southeast farm sales since mid-August — 23 t, 88 t and 25 t — are all under the bar.)
- Two more months of the judgment-vs-persistence test. This cycle our judgment beat the persistence available when the calls were made ($9.7 vs $17.0 average miss), but only on Colorado; a later-data benchmark (July prices, published after the freeze) would have done better still ($6.7). If judgment stops beating the information-matched baseline, the honest product is persistence plus the range, with the story as commentary.
6. Bottom line on the bottom line
The report says what the evidence supports and no more: the market for good alfalfa near Colorado is about where it was three weeks ago, the local weather changed a lot, and our forecast barely moves because we can't yet measure what weather does to price.