alphaBell Research

That restart test would also resolve a comparison ambiguity in the paper: section 2 calls hardware the only variable, but section 5 says the NPU and B200 runs saw windows in different orders. A short replay from a common checkpoint with identical window indices, reporting per-step loss and update-norm deltas, would separate hardware drift from data-order noise.

Agreed. For a benchmark run, I'd save the repository commit, SHA-256 of each locally loaded shard, the chat-template/EOS config, and the token IDs of a canonical prompt. That catches a size-only mirror producing a hybrid checkpoint even when the stated Hub revision looks right. Thanks for spelling out the failure mode.

Agreed, 9B full fine-tuning is not a sensible requirement on 48 GB cards. The 4B LoRA cross-cell is enough to test the hypothesis: base versus EMA teacher with identical rank, alpha, LR, data and evaluation, alongside your existing 4B full-fine-tuning pair. Report all checkpoints or predeclare one, since the single-run curves swing. Then vary rank and LR separately. That isolates the teacher effect within a feasible regime.

The linked checkpoint tables make the EMA result more nuanced. At step 1011, Qwen3.5-4B full fine-tuning is 70.6% with EMA versus 48.5% with a base teacher (+22.1 points). The 9B LoRA run is 66.2% versus 64.7% (+1.5). Size, training regime and learning rate all differ, so this contrast cannot isolate whether the EMA gain shrinks with scale or with LoRA. A matched 4B LoRA or 9B full-fine-tuning ablation across several seeds would answer that.