AtomicChat/Ornith-1.5-122B-GGUF

How to Run Ornith 1.5 122B Locally

Built from ornith-ai's original weights with our own importance matrix. The calibration corpora behind our builds are public.

• You can now run Ornith in Atomic Chat.
• See our quantization analysis below for measurements and instructions.

---

Pick a file

Every number below is measured, not estimated. How we measured it is at the bottom, and the raw logs are in the metrics repo so you can check any of it yourself.

KL divergence is how far the quantized model's predictions drift from the original weights. Lower is better, and zero means identical. top-1 is how often it picks the same next word the original would have picked.

File	Size	KL divergence	top-1
Q8_0	132.6 GB	0.00962	96.28%
AD-Q6_K	104.5 GB	0.01086	96.02%
AD-Q6_K-Q5_K	93.3 GB	0.01326	95.53%
AD-Q5_K-Q4_K	78.5 GB	0.02222	94.19%
AD-Q5_K-IQ4_XS	76.0 GB	0.02323	94.05%
AD-Q4_K-IQ4_XS	71.2 GB	0.02739	93.52%
AD-IQ4_XS-IQ3_S	62.2 GB	0.05311	90.96%
AD-IQ3_S-IQ3_XXS	54.3 GB	0.08212	88.84%
AD-IQ3_XXS-IQ2_S	47.6 GB	0.13030	86.05%
AD-IQ2_S-IQ2_XS	42.8 GB	0.18881	83.33%
AD-IQ2_XS-IQ2_XXS	39.2 GB	0.23616	81.23%
AD-IQ2_XXS-IQ1_M	35.0 GB	0.35296	77.18%

AD-IQ2_XXS-IQ1_M is a research file: it is the smallest that runs, but it picks the original's next word only 77% of the time. Take AD-IQ2_XS-IQ2_XXS or larger if it fits.

AD- marks an Atomic Dynamic layout. The name says what the two largest tensor groups got: AD-<ffn_down>-<ffn_up>, collapsed to one name when both match. Nothing is named after a type it does not contain.

Which one fits your card

The file has to fit, and so does the context. This model keeps 24 KB of attention cache per token, which is 0.2 GB at 8k context and 0.8 GB at 32k. Budget for both. Memory below is in GiB (1 GiB = 1.074 GB), which is how cards and Macs report it; the file sizes above are GB on disk.

Memory for the model (GPU or unified)	File	Leaves room for
48 GiB	AD-IQ3_XXS-IQ2_S	around 112k context
64 GiB	AD-IQ4_XS-IQ3_S	around 216k context
80 GiB	AD-Q5_K-Q4_K	around 248k context
96 GiB	AD-Q6_K-Q5_K	around 344k context
128 GiB	Q8_0	around 144k context
192 GiB and up	Q8_0	full context

If you are choosing between two neighbouring files, take the larger one. The steps between them cost one or two gigabytes and buy noticeably fewer wrong words.

On a Mac, macOS lets the GPU use only part of the unified memory by default and needs some for itself. On a 128 GB Mac the practical top is AD-Q6_K (97 GiB): raise the limit first with sudo sysctl iogpu.wired_limit_mb=118000 (resets on reboot). On a 64 GB Mac take AD-IQ3_XXS-IQ2_S.

Running it

llama-server -m Ornith-1.5-122B-AD-Q4_K-IQ4_XS-00001-of-00002.gguf -ngl 99 -c 8192

The chat template is inside every file, llama.cpp applies it on its own.

Thinking is on by default. To turn it off for a request, pass the template flag through the server:

curl http://localhost:8080/v1/chat/completions -d '{
  "messages": [{"role": "user", "content": "Hello"}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

The authors' recommended sampling is temperature 0.6, top-k 20, top-p 0.95 (--temp 0.6 --top-k 20 --top-p 0.95).

The model ships a multi token prediction (MTP) head. It is inside every file here and needs no extra download. Turn it on for faster generation:

llama-server -m Ornith-1.5-122B-AD-Q4_K-IQ4_XS-00001-of-00002.gguf -ngl 99 -c 8192 --spec-type draft-mtp

On Ornith 1.5 35B-A3B, the same family with the same head, this raised generation speed by 18 to 35% in our tests, with 59 to 62% of drafted tokens accepted. We have not measured it on the 122B yet. The head is stored at q8_0 on the largest files, q5_k in the middle and q4_k on the smallest; on the 35B q5_k was accepted as often as q8_0. Use a recent llama.cpp: builds without MTP support for qwen35moe may refuse to load files that carry the head.

Images and video

This model reads images, not only text. That needs one extra file, the vision projector, which is separate from the quant and shared by all of them:

• mmproj-Ornith-1.5-122B-F16.gguf for most people
• mmproj-Ornith-1.5-122B-BF16.gguf if your setup prefers bfloat16

Download it once alongside whichever quant you picked:

llama-mtmd-cli \
  -m Ornith-1.5-122B-AD-Q4_K-IQ4_XS-00001-of-00002.gguf \
  --mmproj mmproj-Ornith-1.5-122B-F16.gguf \
  --image your-photo.jpg -ngl 99 -c 8192 \
  -p "What is in this image?"

The projector is not quantized and is the same for every file. The numbers in this card are for text: the calibration corpus is text only, so the image path is not part of the KL measurements.

What we found while building these

• Only the experts move. As on Ornith 1.5 35B-A3B, every file is Q8_0 with the routed experts overridden, the same type in all 48 blocks, ffn_down_exps one step above gate and up. Attention, the linear-attention blocks, the shared expert, embeddings and head stay q8_0. On the 35B, bands of higher precision in chosen blocks bought nothing over this flat layout.
• The floor is higher than on smaller models of this family. Q8_0 already sits at 0.0096 KL and 96.3% top-1, and every file is about twice what we measured on the same architecture at 35B. The ratio is the same at every size, so it comes from this checkpoint, not from one tensor or the layout. Read the table against the Q8_0 row, not against zero.
• The MTP head never sees the importance matrix, because llama-imatrix does not run it. So it takes k-quants only, which need none.
• The importance matrix covers 9,413 chunks of 512 tokens, collected on the BF16 weights.

The calibration data

Our importance matrix comes from a corpus we built for this model and published: AtomicChat/calib-corpora, recipe ornith-1.5-moe. It is 4,819,828 tokens across 2,903 documents:

Part	Share
agentic and tool use	22.8%
multilingual	18.7%
code	16.6%
reasoning	12.5%
long context	10.5%
vocabulary sweep	10.3%
structured data	4.4%
graphics	4.2%

Every conversation is rendered through this model's own chat template, so the chat markup appears as the single tokens the model actually sees.

If you build your own importance matrix from this corpus, pass --parse-special to llama-imatrix. Without it the chat markup is read as ordinary punctuation.

How we measured

Reference: the original BF16 weights, converted to GGUF and run unquantized. Perplexity on the held-out set is 5.7625 plus or minus 0.03582.

Held-out text: eval_neutral from our calibration dataset, never used for calibration. 87 chunks at 4096 context.

Metric: per-token KL divergence of each file's predictions against the reference, plus top-1 agreement.

Hardware: 4x H100 80GB HBM3.

The raw logits of the reference are published in the metrics repo, split into parts because of the file size limit. With them you can measure your own build against exactly the same point we did:

cat base-neutral.kld.*.part > base-neutral.kld

llama-perplexity -m your-quant.gguf -f eval_neutral.txt \
  --kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99

Reproducing a file

The importance matrix was collected on the BF16 weights rather than on a quantized copy, in one run:

llama-imatrix -m Ornith-1.5-122B-BF16.gguf -f calib_train.txt -o imatrix.gguf \
  -ngl 99 -c 512 -b 8192 -ub 8192 --parse-special

Then, for AD-Q4_K-IQ4_XS:

llama-quantize \
  --imatrix imatrix.gguf \
  --tensor-type '^blk\.48\.=q5_k' \
  --tensor-type '^token_embd\.weight$=q8_0' \
  --tensor-type '^output\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.ffn_down_exps\.weight$=q4_k' \
  --tensor-type '^blk\.\d+\.ffn_(gate|up)_exps\.weight$=iq4_xs' \
  --tensor-type '^blk\.\d+\.ffn_(gate|down|up)_shexp\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_(q|k|v|qkv|gate|output)\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.ssm_(out|alpha|beta)\.weight$=q8_0' \
  --override-kv general.file_type=int:30 \
  Ornith-1.5-122B-BF16.gguf Ornith-1.5-122B-AD-Q4_K-IQ4_XS.gguf Q8_0

The rules for every other file are in ladder/ in the metrics repo, next to its quantization log. The first rule that matches a tensor wins.

Model details

Ornith 1.5 122B is built on Qwen3.5-122B-A10B by Qwen, with Ornith's continued pretraining, mid-training and post-training: the architecture and tokenizer are Qwen3.5's, the weights are Ornith's further training of Qwen's.

48 layers, hidden size 3072, vocabulary 248,320, context 262,144. One multi token prediction head.

Needs a recent llama.cpp with qwen35moe architecture support. Built and tested with llama.cpp b11736 (33d6135cb).

License and credits

Ornith releases Ornith 1.5 122B under MIT. It is built on Qwen3.5-122B-A10B, which Qwen releases under Apache 2.0; that license is included here as LICENSE-Qwen3.5. Base model by Ornith, on top of Qwen3.5 by Qwen. Quantization and evaluation by AtomicChat.