# Nawah-Math-Reasoning — training code Everything that produced the model in this repo: two dataset pipelines, the SFT trainer, the eval harness, a `pass@k` diagnostic, and a GRPO implementation. Run top to bottom and you get the release; run any single stage and it resumes from what is already on disk. Paths below assume this directory is the working directory and `$P` is a Python with `torch`, `transformers>=5.15`, `pyarrow` and `huggingface_hub`. ## 0. Two environments, and why | env | for | pins | |---|---|---| | training / eval / Hub | `train_reasoning.py`, `eval_reasoning.py`, everything else | torch 2.9 + cu126, **transformers 5.15** | | generation | `translate_gsm.py`, `synth_generate.py` | **vLLM 0.8.5.post1**, torch 2.6 + cu124, **transformers 4.51.3** | They are not interchangeable. The model configs are transformers-v5 format (`rope_parameters`, `dtype`, `tokenizer_class: TokenizersBackend`) and transformers 4.x cannot read them — but vLLM 0.8.5 breaks on transformers 5.x (`TokenizersBackend has no attribute all_special_tokens_extended`), so the generation env must stay at 4.51.3. Do not `pip install -U` in it. vLLM 0.8.5 is itself a pin: every release from 0.20.2 up requires torch 2.11, which is a CUDA 13 build needing driver ≥ 580. On a CUDA 12.4 host that is a hard stop, and cu12 wheels do not rescue it (`ImportError: libcudart.so.13`). 0.8.5 is the last cu124 release. In transformers v5 `TrainingArguments` it is `eval_strategy` (not `evaluation_strategy`) and `warmup_steps` (there is no `warmup_ratio`). ## 1. `gsm8k-reasoning-ar` — 142,969 machine-translated rows ```bash $V translate_gsm.py 150000 # vLLM env; ~95 min on 1x A6000; resumable $P build_dataset.py # -> out_gsm/*.parquet + rejects.jsonl $P push_dataset.py --repo /gsm8k-reasoning-ar ``` `Ajhesh7/gsm8k-reasoning-SFT-datas` translated with `ByteDance-Seed/Seed-X-PPO-7B`. The trailing language tag in the prompt is **mandatory**: `Translate the following English sentence into Arabic:\n{text} `. Only 150k of the source's 600,000 rows are translated, and the sample is stratified by question pattern with a floor of 13 rows/pattern: the corpus expands from just **2,814** question patterns, so translating all 600k is ~99% redundant. **The reason `build_dataset.py` has a validator at all:** Seed-X silently corrupts arithmetic in ~0.6% of segments — `$13751` → `13571`, `4 × 44 = 176` → `4 × 46 = 176`. These read as fluent Arabic and pass any fluency check. So every row is numeral-audited, asymmetrically: **reasoning chains strictly** (every numeral must survive exactly), **questions leniently** (a value ≤ 12 may be verbalised — `6 friends` → `أصدقائها الستة` — but any number the English never contained is rejected). Naive exact-multiset matching rejects ~6% of *correct* translations; do not "simplify" it back to that. `translate_gsm.py` appends every chunk to `out_gsm/translations.jsonl` and skips cached segments on restart, so an interruption costs at most one 20k chunk. Do not pass `enable_prefix_caching=True` — it hangs at startup on this vLLM/V1 engine. ## 2. `arabic-math-reasoning-synth` — 120,462 verified rows ```bash # general pool: 100,323 rows, ~22 h GEN_MODEL=/gemma-3-12b-it BACKEND=vllm OUT_DIR=out_synth TARGET=100000 \ BATCH=256 MAX_NEW=1200 MAX_LEN=3072 GPU_UTIL=0.90 \ nohup $V -u synth_generate.py > synth_run.log 2>&1 & # relational pool: 20,139 rows, ~3.5 h. Disjoint task-id range, on purpose. GEN_MODEL=/gemma-3-12b-it BACKEND=vllm OUT_DIR=out_synth_rel POOL=relational \ START_TASK=1000000 TARGET=20000 BATCH=256 MAX_NEW=1200 MAX_LEN=3072 GPU_UTIL=0.90 SEED=1234 \ nohup $V -u synth_generate.py > synth_rel.log 2>&1 & ./finish_merge_push.sh # merge primary cache + any node shards -> out_merged_v6/ $P split_synth_v6.py # -> data_synth_v6_sft/{train,eval,eval_rel}.jsonl $P push_synth_dataset.py --repo /arabic-math-reasoning-synth ``` **Resumability is the design, not a feature.** A task is one generation call asking for 4 problems, and task *N*'s prompt is a pure function of *N* (`synth_common.build_task`) — nothing about the plan is persisted, so a restart redraws identical prompts. Finished tasks are appended to `generations.jsonl` and fsynced; a torn final line is dropped with a warning. The stop condition is **accepted rows, not tasks**. **Raw completions are stored, never just the parsed rows.** Every validator change can be re-scored over the whole cache with no GPU — which is how the accept rate went 48.4% → 64.9% without regenerating anything. **The arithmetic audit** (`synth_common.validate`) re-evaluates every `a op b = c` in the reasoning and one wrong equation rejects the row. Two things it must keep doing, both of which were bugs that rejected *correct* rows: 1. **Equation chains.** Models write `75 + 15 × 5 = 75 + 75 = 150`. Reading only to the first `=` compares 150 against 75 and rejects a correct chain. Split the whole chain, require every segment to agree. 2. **Rounding.** `3200 / 60 = 53.33` is arithmetic as people write it. `_close()` forgives rounding *at the precision the model displayed* (and floor/ceil for integers), so a genuinely wrong number still fails. Also rejected: `noop_step` (`63 + 0 = 63`, padding to hit a step count), `meta_commentary`, Latin residue, missing conclusion marker, and answers that disagree with the last computed value. **The relational pool is a separate pool** (`RELATIONAL_OPS`), deliberately not appended to `OPERATIONS`. Appending would change `rng.choice()` for every task id and silently break reproducibility of the first 100,323 rows. `build_task(task_id, seed, pool)` takes `pool="default"` or `"relational"`; keep it that way. Check `unique_templates` in `build_stats.json` rather than assuming the variation grid worked. ## 3. SFT ```bash $P prepare_data.py # Arabic_Reasoning_Dataset -> data/{train,eval}.jsonl $P prepare_gsm_sft.py # -> data_gsm_sft/{train,eval}.jsonl $P prepare_v6_sft.py # three-way mix -> data_v6_sft/{train,eval}.jsonl BASE_MODEL= OUTPUT_DIR=./Nawah-Math-Reasoning \ TRAIN_FILE=data_v6_sft/train.jsonl EVAL_FILE=data_v6_sft/eval.jsonl \ MAX_LENGTH=768 EPOCHS=5 BATCH_SIZE=64 GRAD_ACCUM=1 WARMUP_STEPS=200 EVAL_STEPS=1000 LOAD_BEST=0 \ $P -u train_reasoning.py 2>&1 | tee train.log # 21,535 steps, ~85 min on 1x A6000 ``` **`LOAD_BEST=0` is load-bearing.** Eval loss selects the *worse* checkpoint on this ladder, and that was measured rather than assumed: on a corpus with no repeated rows, the minimum-loss checkpoint scored 30.9% where the final scored 35.6%. It held for four consecutive runs. Ship the final checkpoint. **Answer styles are not normalised.** GSM8K rows end in a bare numeral, the other two corpora in an `إذن، …` sentence. Rewriting them into one style deletes what the mix adds, so a `source` tag rides on every row and eval scores each half on its own terms. The eval splits are **pinned across versions** — the same 400 `Arabic_Reasoning` rows and the same 600 GSM8K rows since the first model, and `split_synth_v6.py` copies the previous synth eval rows through verbatim rather than re-shuffling. Re-drawing would have moved 1,955 of 2,000 held-out items into train. ## 4. Eval ```bash EVAL_FILE=data_v6_sft/eval.jsonl $P eval_reasoning.py ./Nawah-Math-Reasoning 1800 EVAL_FILE=data_synth_sft/eval.jsonl $P eval_reasoning.py ./Nawah-Math-Reasoning 1000 ``` Greedy. Reports well-formedness, **number agreement**, exact match and reasoning length, broken down per `source` when the rows carry the tag. `EVAL_FILE` comes from the env — pointing it at the wrong split silently scores against the wrong data. Score on **number agreement**, not exact match: with two answer styles in the mix, exact match measures style compliance, not arithmetic. ## 5. RL — run the diagnostic first ```bash $P -u passk_diag.py ./Nawah-Math-Reasoning # 240 problems x k=8 @ T=1.0, ~25 min ``` RLVR reweights samples the model already produces. A group where every sample is wrong scores all-zero, the advantage is zero, and there is no gradient — so exploitable headroom is bounded by `pass@k − pass@1`. On the previous version that was **+27.1 points**, with 40.4% of problems never solved in 8 tries. Run this before spending a GPU-hour. That diagnostic is also what produced this release: the dead 40% turned out to be a *data* gap (relational comparisons were 1.34% of the corpus), so the fix was §2's relational pool through SFT, not RL. **Data first, then RL** — running RL first leaves the dead tail untouched and caps the gain. ```bash MODEL=./Nawah-Math-Reasoning OUTPUT_DIR=./Nawah-Math-Reasoning-grpo \ GROUP=8 PROMPTS_PER=8 STEPS=500 LR=1e-6 BETA=0.02 \ $P -u grpo_train.py 2>&1 | tee grpo.log ``` `grpo_train.py` is hand-rolled — TRL is not installable against these pins. Group-normalised advantage with no value network, binary final-answer reward (not partial credit — the failure being fixed is fluent reasoning landing on a wrong number), KL to a frozen reference via Schulman's k3 estimator, and no importance ratio or PPO clipping because sampling is on-policy with one step per batch. Zero-spread groups are skipped **and counted**; the dead-group percentage in the log is the number to watch. ## 6. Release ```bash $P build_release_code.py # stage this directory $P push_release.py --dry-run # render the card only $P push_release.py ``` Every number in the model card is read from an eval JSON on disk. Nothing is typed by hand, so the card cannot drift from the measurements. ## Demo `space/` is the Gradio demo, deployable as-is to a Space. Two things that break it if "cleaned up": the streamer must use `skip_special_tokens=False` (``/`` are real special tokens and stripping them destroys the reasoning/answer split), and `render_prompt()` must stay byte-identical to the trainer's rendering — a 52M model is very sensitive to format drift. On ZeroGPU, `import spaces` must come **before** torch, and `model.to("cuda")` at startup must stay unguarded: no GPU is attached at startup and the call is replayed inside the forked GPU process.