Flow Reasoning Models — lightweight release draft
One selected checkpoint per task, with portable benchmark/calibration assets and measured results. This repository stays private during testing.
Setup
GitHub installation only; nothing is published to PyPI.
python -m venv .venv
source .venv/bin/activate
pip install "git+https://github.com/helblazer811/Flow-Reasoning-Models.git"
Download and evaluate
Use standard Hugging Face downloads:
from huggingface_hub import snapshot_download
snapshot_download("helblazer811/FlowReasoningModel", allow_patterns=["sudoku/*"], local_dir="models")
python -m frm.evaluate --model-dir models/sudoku
Replace sudoku with maze_unique or zebra for the other models. Evaluation
checks the downloaded files and uses calibrated sequential restarts. Results go
under outputs/. Add --max-samples 8 for a functional check, not a benchmark score.
The evaluator computes fixed-depth comparisons too; reported policy NFE excludes
that additional comparison work. For manual download, obtain all files in the
corresponding model folder from this repository.
Selected models and measured results
| Bundle | Training seed | Benchmark | Sequential restart accuracy | Mean NFE |
|---|---|---|---|---|
| sudoku | 0 | EqR Sudoku-Lite, 2,048 puzzles | 99.7070% (2042/2048) | 899.75 |
| maze_unique | 0 | Official EqR Maze-Unique, 1,000 puzzles | 99.9% (999/1000) | 15.708 |
| zebra | 0 | 5x5 Zebra, 1,024-puzzle seed0 subset | 99.6094% (1020/1024) | 6.2402 |
Sudoku costs 0.8062 mean TFLOPs/puzzle; Maze-Unique costs 0.1781. Zebra FLOPs are unprofiled for this preprocessing, so only NFE is reported. Zebra seed0 scores99.8047% at257NFE with the fixed-depth held-t trajectory; its stopping/restart policy scores99.6094% at6.2402meanNFE. Its earlier SDE result is99.6094%. Every test puzzle accepts on the first attempt. FLOPs for this 5x5 setup are unprofiled; NFE is reported instead.
All three hosted checkpoints now use training seed0. Maze-Unique restart seeds0/1/2: 99.9%,98.5%,99.8%. Zebra SDE seeds0/1/2:99.6094%,99.8047%,99.5117%. Only one new Sudoku training seed was run. Zebra seed0 was preferred for a consistent release after its measured default score99.6094% exceeded seed1's 99.4141%. Source checkpoint/step/seed details remain in each manifest.
Scope and reproducibility
- Sudoku is the independently tested fixed-time FPF training variant, evaluated on Sudoku-Lite; this does not establish the paper's full-test headline. Its independent development calibration excludes known training and test puzzles.
- Maze-Unique uses the official pinned EqR1k/1k dataset and standard two-stage FPF. Its256-puzzle calibration is sampled from the training split as in the published policy.
- Zebra currently trains/tests5x5 with repacked answers. It does not reproduce the paper's mixed3x3–6x6 pinned-answer dataset. Calibration uses training puzzles explicitly disjoint from test; no test labels select stopping/restart candidates.
Each folder contains model.pt, training.yaml, threshold.json, test.npz,
calibration.npz, manifest.json, provenance.json, and expected_results.json.
The manifest records the original checkpoint hash and the compact inference-checkpoint
hash. Optimizer state is omitted; raw weights and EMA are retained. These are compact PyTorch checkpoints from the recorded training runs.
For training from scratch, download training.yaml and run
python -m frm.train --config training.yaml on the matching data.
Training and evaluation code is available in Flow-Reasoning-Models. This lightweight release supports the models and evaluations above, not every paper baseline, ablation, table or figure.