Montevideo forecast¶

This record fixes the causal evaluation boundary and compares one node-local forecast with three matched directed-diffusion forecasts.

The later seasonal-floor study finds a stronger train-only temporal control. The learned comparison below predates that floor.

Decision¶

At tinygrad revision dd16d5a, all four learned models lose to persistence on three CPU seeds. Coordinate and road distance do not improve unit diffusion under the frozen promotion rule. Tinymesh retains the loader, sparse operator, recurrent witness, protocol, and negative result; it promotes no geometry API.

The comparison loads one raw lag, assigns examples by target hour, derives normalization from training rows only, and evaluates predictions in raw passenger-count units.

raw hourly signal[743,675,1]
          |
          +--> train targets       0:520
          +--> validation targets 520:594
          +--> test targets       594:743
          |
          v
24-hour causal windows

Validation and test windows may read observations before their target boundary. They never send a later target into training, normalization, checkpoint selection, or optimizer state.

Why one loader lag¶

The source feature and target are the same hourly passenger series. With lags=1, signal row t pairs observation t with target t+1. A 24-step recurrent window then has one owner for each hour:

X[t:t+24] -> y[t+24]

Loading four lags and then unrolling a recurrent history would send overlapping lag vectors through time and obscure that contract.

Split¶

The pinned signal has 743 next-hour examples after one loader lag:

Splitting the target axis first preserves all validation and test targets. Only inputs cross a boundary, and every crossed input predates its target.

Normalization¶

Input and target mean and population standard deviation are fitted separately for every node from training rows:

x_norm = (x - mean_x_train) / scale_x_train
y_norm = (y - mean_y_train) / scale_y_train

A constant training node receives scale one, preserving its observed mean without division by zero. Perturbation tests change every validation and test value while leaving both training standardizers unchanged.

Learned models optimize normalized targets. Metrics restore the target standardizer first; MAE and RMSE therefore use passenger-count units.

Baselines¶

Three raw-unit controls are fixed before model training:

zero          predict 0
persistence   predict the latest observed hour
train mean    predict each node's mean training target

The zero baseline is necessary because about 80% of target cells are zero. A small aggregate error without this control would be easy to misread.

At tinygrad revision dd16d5a, CPU returns:

Three nodes are constant in the training input and target rows. Persistence is the strongest baseline on both validation and test. A learned model must beat it before any graph comparison is useful.

Directed recurrence¶

The three geometry comparisons use the same recurrent class:

S = [X, H]
B(S) = [S, forward(S), reverse(S)]
Z, R = sigmoid(Linear(B(S)))
C = [X, R * H]
H_next = Z * H + (1 - Z) * tanh(Linear(B(C)))

One gate projection and one candidate projection require four sparse sums per step. A hand-written two-step edge loop matches the cell, gradients reach its candidate parameters through time, and UOp inspection finds no [N,N] or [N,E] carrier. The node-local LSTM is contextual only because its cell and parameter count differ:

Resetting the tinygrad seed before constructing each diffusion model gives the three variants identical learned tensors. They differ only in fixed affinity:

unit        1
coordinate  1 / Euclidean edge distance
road        1 / observed road distance

Each affinity is source-normalized independently in the forward and reverse graphs. The graph, reverse graph, and normalized edge fields are realized once per variant and reused across every recurrent step.

Identifiability¶

Most nodes have at most one relevant edge. Source normalization cancels any positive scalar affinity on those rows, so geometry can change only branching rows:

Seven nodes have no outgoing edge and nine have no incoming edge. This narrow identifiability bound makes the later near-equality expected rather than surprising.

Controlled run¶

Every run uses a 24-hour history, hidden width 4, batch size 32, normalized MSE, Adam at 0.01 with fused=False, and at most ten epochs. Checkpoints are evaluated at epochs 0, 5, and 10; the earliest minimum validation RMSE is restored before one test evaluation. All runs selected epoch 10.

Values below are raw-unit MAE / RMSE. Fit time includes initial validation, TinyJit compilation, training, and checkpoint validation; it excludes data loading and final test evaluation.

The twelve fits took 33.6 minutes in aggregate on an Apple M4 MacBook Air. Mean plus sample standard deviation across seeds is:

Promotion decision¶

A geometry variant must beat unit diffusion on mean validation MAE and RMSE, with the paired direction agreeing in at least two seeds. Negative deltas are better:

Neither variant reaches the test-confirmation gate. For completeness, their test MAE is also worse in every seed; small mean RMSE changes (-0.000105 coordinate and -0.000386 road) improve in two seeds, while MAE improves in zero. The rule requires both metrics.

The learned models also remain well behind persistence. This experiment proves that the complete sparse causal path trains and compares fixed geometry without leakage. It does not show that topology or geometry improves this forecast.

Limits¶

This result covers one-hour-ahead prediction, one fixed graph, one 24-hour history, three seeds, regular hourly rows, and per-node standardization. Roughly 80% of targets are zero, and scalar geometry can distinguish only a small fraction of edges after normalization. There is no calendar, weather, missingness mask, multi-horizon decoder, architecture search, learned affinity, generic trainer, public model factory, or production claim. Forecast construction and evaluation policy remain experimental; DirectedDiffusion and DiffusionGRU now live in tinymesh.nn.

Reproduce¶

uv run --locked python -m experiments.run montevideo_forecast DEV=CPU
uv run --locked python -m experiments.run montevideo_forecast DEV=CPU MODEL=all SEED=-1 EPOCHS=10 HISTORY=24 BS=32 HIDDEN=4 LR=0.01 CHECKPOINT_EVERY=5
DEV=METAL uv run --locked python -m unittest tests.test_montevideo_forecast tests.test_directed_gru