Prospector

Prospector is a compact LLM policy post-trained to answer questions over a scientific knowledge graph. The policy calls a small set of tools to traverse the graph turn by turn, building its context from what each call returns, and answers with a chain of edges that is checked against what its tools retrieved. Prospector is Qwen3.5 at 0.8B, 2B and 4B parameters with LoRA adapters, trained on one graph built from the semiconductor-manufacturing literature. With frozen weights, the same adapters transfer to graphs from three other fields.

This repository holds the weights, the training rows and the stored evaluation runs of the paper, laid out as the code repository expects them. The code that serves, evaluates and trains them is at github.com/lamm-mit/Prospector. Its README covers the evaluation harness, serving a policy, training one on your own graph, and the command behind every number in the paper.

Get started

The code repository's quick start, on a Mac with Apple silicon or a Linux machine with an NVIDIA GPU:

# install uv, unless you have it
curl -LsSf https://astral.sh/uv/install.sh | sh
export PATH="$HOME/.local/bin:$PATH"
git clone https://github.com/lamm-mit/Prospector && cd Prospector
# download this repository's adapters and data into the checkout, about 9 GB
uvx --from huggingface_hub hf download lamm-mit/Prospector --local-dir . \
    --include "models/*" --include "data/*"
# build the environment and evaluate Prospector-4B on six questions
./quickstart.sh

The first run also downloads the base model from the Hub and merges the adapters into it. The semiconductor graph is then under data/raw/, and the code repository's data/raw/README.md downloads the three other graphs. The code repository's README, under "Reproducing the paper", gives for each number in the paper the command that recomputes it and the stored run the number is computed from.

Results

The gate accepts a cited chain when every cited edge was returned by a tool and the answer names a node of the chain. Gated accuracy is the fraction of questions whose chain passes the gate and whose answer names a gold label. The table gives means over the seeds of the runs stored here, ± one standard deviation.

policy, graph	gated accuracy
Prospector-4B, semiconductor, eight seeds (flash20cap1024-s0 to -s7)	0.839 ± 0.015
Prospector-2B, semiconductor, eight seeds (q2bd2eval-ck120-s0 to -s7)	0.755 ± 0.016
Prospector-0.8B, semiconductor, eight seeds (q0.8bd2eval-ck120-s0 to -s7)	0.694 ± 0.021
Prospector-4B, bio-inspired materials, three seeds (flash20stackbio-s0 to -s2)	0.803 ± 0.017
Prospector-4B, PFAS, three seeds (stack1kpfas-ctx32k-s0 to -s2)	0.766 ± 0.022
Prospector-4B, materials properties, three seeds (stack1kmatprop-s0 to -s2)	0.809 ± 0.011

The served policies

The paper's recipe trains each size in four stages: a supervised stage on a frontier teacher's demonstrations, a GRPO round over whole episodes, a second supervised stage on the demonstrations blended with the policy's own kept episodes, and two decision rounds of GRPO that resume stored mid-episode states. A verifier is then trained on the served policy's own samples. Each size serves the second supervised stage's adapter with the checkpoint that each decision round's evaluation grid picked applied on top, in this load order, over Qwen/Qwen3.5-<size>. The policies run with the plain_chatml chat template at a budget of 20 tool calls and 1,024 generated tokens a turn.

size	policy (--adapter, in load order)	verifier
4B	models/traj_sft_4b_flash20g2/final, models/traj_grpo_4b_flash20d1/checkpoint-120, models/traj_grpo_4b_flash20d2/checkpoint-120	models/verifier_sft_4b_flash20v/final
2B	models/traj_sft_2b_flash20g2/final, models/traj_grpo_2b_flash20d1/checkpoint-100, models/traj_grpo_2b_flash20d2/checkpoint-120	models/verifier_sft_2b_flash20v/final
0.8B	models/traj_sft_0.8b_flash20g2/final, models/traj_grpo_0.8b_flash20d1/checkpoint-100, models/traj_grpo_0.8b_flash20d2/checkpoint-120	models/verifier_sft_0.8b_flash20v/final

The adapters are PEFT LoRA adapters in transformers' fused naming for Qwen3.5 (in_proj_qkv), which the code repository's pinned transformers and peft load. Each run.json records the chat template its adapter was trained under, and each GRPO round's peak.txt records the checkpoint its grid picked.

What is here

Downloaded into a checkout's root, every file lands where the code expects it. MANIFEST lists every file by pattern with its reason.

data/raw/        the semiconductor graph, the paper's own; the code repository's
                 data/raw/README.md downloads the three other graphs from their sources
models/          the recipe's six training runs at each size, each adapter with the
                 run.json that records what it was trained from, and
                 recipe_backfill_v1.json, which records how those run.json files were
                 written and each later change made by hand to a stored file here
data/traj/       the training rows of each stage, each with a .provenance.json sidecar
                 naming what it was built from, where one exists
data/runs/       the stored evaluation runs: every run the paper reads or the code
                 repository's README names, the timing runs behind the cost figure,
                 and the latent-state probe's derived files under _probe/
data/analysis/   inference_speed.json, which prices the timing runs at the posted
                 rates it dates
MANIFEST         the list of what is here and why, pattern by pattern

The set is the smallest that serves the three policies, lets prospector run re-evaluate a size over the released adapters without training anything, and recomputes every figure in the paper from stored runs, with the probe's hidden states (about 1.7 GB a run) regenerated by its states stage from the stored transcripts.

The training rows. data/traj/ holds the rows each stage trained from, named as the code derives them. Stage 1's demonstrations (tsmc_sft_mix_flash20.jsonl) are shared by the three sizes. The second supervised stage's mix, the decision rounds' stored states and the verifier's rows exist per size. The teacher runs the demonstrations came from (flash20-teach-4x-g0 to -g7 and -p2) are under data/runs/. The 0.8B's and 2B's mixes and stored states were rebuilt by the code's builders from the stored runs they were sampled from, and the same builders reproduce the 4B's files byte for byte. With these adapters in place, prospector run trains nothing. At 0.8B and 2B the command reuses the stored evaluations here, which carry the names it derives, and evaluates what it does not find. At 4B, whose stored runs carry older names, the command evaluates afresh under its own names.

The stored runs. A run is a directory data/runs/<name>--<benchmark>/ in the shape the code repository's README describes. A closed-book or retrieved-subgraph run, which has no episodes, holds one reply per question and, where it was scored, the scorer's output under the same gate.

License and citation

The adapters, the rows, the runs and the semiconductor graph are released under the Apache License 2.0.

@article{yang2026prospector,
  title   = {Prospector: learning a compact graph traversal policy for scientific reasoning},
  author  = {Yang, Haiqian and Buehler, Markus J.},
  journal = {arXiv preprint},
  year    = {2026}
}