Raptor 0.6

Raptor 0.5 shipped at the end of August as our first resident agent model for the 8–16 GB Macs most people actually own. Within a week we had screen recordings of it looping: the same tool call re-issued until the harness gave up, a paragraph repeated word for word, replies that were nothing but exclamation marks. Raptor 0.6 is what came out of taking those recordings seriously — a different base model, a tune we rebuilt from scratch against the Osaurus tool surface, and a set of harness fixes that shipped alongside it.

Raptor 0.6 is a 4.1B dense model built on Spark-X2.5-4B, lightly fine-tuned for Osaurus, and quantized to 3.7 GB. It decodes at about 105 tokens/s on an M5 Max, reasons by default, and calls tools in a dialect Osaurus parses natively. It is the same kind of model 0.5 was meant to be — read the folder, open the spreadsheet, check the report against the data, draft the email, back the file up before touching it — on a base we could measure our way to trusting.

What it is

Raptor 0.6 is built on Spark-X2.5-4B (Apache-2.0), a 4.1-billion-parameter dense model with a hybrid attention stack: for every full-attention layer there are three sliding-window layers that look back 512 tokens, 36 layers in all. The base ships with a native 1M-token context, 200+ languages, and a post-training recipe aimed squarely at agentic work. Its published agent numbers are unusual for a 4B model — BFCL-V4 65.1, τ²-bench 75.1, MCP-Atlas 54.6 — and they are the vendor's numbers, not ours; we cite them because they are why we looked at it, not because we re-ran them.

Two properties of the architecture matter more to us than the benchmarks.

It is a plain transformer. Raptor 0.5 was a hybrid with recurrent linear-attention layers; the first 0.6 preview was a looped transformer that runs its 22 layers twice. Both are clever, and both gave us a week of debugging a serving stack that had to restore recurrent state exactly or address a 44-slot cache correctly. Spark-X2.5 has none of that. Its cache is ordinary key/value attention, the kind every runtime on Apple Silicon already handles, and a wrong answer from it is a model decision rather than a corrupted state.

Memory still does not grow much with the conversation. Only the nine full-attention layers keep a cache proportional to context — about 36 KB per token in fp16. The 27 sliding-window layers never hold more than 512 positions, a fixed 57 MB in total. A 32K-token agent session costs roughly 1.2 GB of cache — and in Osaurus that cache does not have to live in RAM. Macs ship with fast SSDs, fast enough that paging a session cache through them is cheaper than holding it in memory a model also needs, so Osaurus streams the cache to disk as it is produced: a small hot set in memory, the rest on SSD. The disk cache survives an app restart, so a long session picks up where it left off instead of re-prefilling.

At 4B parameters, dense is not a compromise on a small Mac. The 6-bit bundle is 3.7 GB, decode is bandwidth-bound and fast, and there is no expert routing to keep in full precision.

What happened to Raptor 0.5

Raptor 0.5 was a rank-4 LoRA on Ling-3.0-tiny, an 8B mixture-of-experts with ~1B active parameters per token. The architecture argument was sound and still is. The problem was everything that happened after the argument.

The reports came in two shapes, and it took days of one-variable controls to find that they had two different causes — neither of them where we first looked.

Replies made of exclamation marks, and tool names with a stray ! in them. This one was ours, not the model's. The Ling 3.0 architecture is a close cousin of Ling 2.6, and the serving stack Osaurus uses had a silent fallback: if the config markers that distinguish the two were not read the way it expected, it routed a Ling 3 bundle through the Ling 2.6 code path. That path produces non-finite logits for a Ling 3 model, the sampler then picks token id 0, and token id 0 in Ling's vocabulary is !. Once we could reproduce it, the fix was to make the dispatch fail loudly instead of guessing, and to pin Osaurus to that runtime. The quantized weights were checked and exonerated along the way — row 0 of the embedding and output head were pristine, and the same prompts replayed clean on the correct runtime.

The same tool call re-issued fifteen, twenty, thirty times, with the same reasoning block every time. This one was the model, and it was the harder finding. Ling's chat template keeps the model's own reasoning in the conversation history, and once one identical successful turn sat in that history, Raptor 0.5 copied its previous reasoning and its previous call nearly verbatim on the next turn — 80 to 100 percent of the time from the second identical turn on, in a ladder we measured across two runtimes. The onset was a one-in-five draw at the shipped temperature. The unquantized weights did it at the same rate, the prefix cache was shown bit-exact against a fresh prefill, and the untuned Ling base did it too, one turn later. Our fine-tune had made an existing tendency earlier and more verbatim. The only lever that helped was on the app side — omitting the historical reasoning when rendering the prompt cut a typical inspect task from 21 tool calls to 9 and the worst identical streak from 15 to 3 — but it did not eliminate it.

What it was not. Our first suspects were the harness and the cache, and we ruled both out with controls. The prefix cache restored bit-exact against a fresh prefill. We found two real harness defects along the way — a tool whose validator suggested a scope its executor then refused, and a message typed mid-run that was persisted between a tool call and its result — and fixed both, and the loop kept reproducing with both fixed, with the cache off, and on the untuned Ling base with no fine-tune at all. They were fuel, not the fire. The fixes shipped in Osaurus 0.25.0 because they were wrong regardless of Raptor.

We tried to train the copying out and could not. Six corpus cycles at rank 4 taught us that this base learns coarse rules readily and exceptions almost never: "a config request means apply" transferred from any dose; "unless it is a delete" and "unless the lookup shows it is already gone" did not, across every corpus and ratio we tried. Every behavioral nudge measured worse than simply fixing the sampling profile. And the pattern was not specific to our stack: other serving stacks have public reports of the same family looping at long context. Ling-3.0-tiny is a strong model. It was the wrong base for a model whose whole job is to make the same tool call only once.

Choosing a new base

The first 0.6 preview, published on September 9, was built on Nanbeige 4.2-3B, a looped transformer. We froze it the same day. Its adapters never beat the raw base on our selection gates, the tuned candidates kept a reasoning-off closing-tag leak and an unclosed-thread degeneration the base did not have, and the loop-through-shared-weights design needs a loader that understands it — a 22-slot cache does not crash, it emits fluent, confident, wrong tokens. That preview stays on Hugging Face as a record; it is superseded by this release.

Spark-X2.5-4B was measured raw before a single training row was written. On our named-tool catalog of 182 prompts across 91 Osaurus tool schemas, both reasoning modes, the untuned base ended every response cleanly with a native stop token, produced no whole-output incoherence, and matched the target schema on 148 of 182. That is not a tool-dispatch score — it is a syntax diagnostic — but it is the floor we needed: a base that already speaks the harness contract, so the tune could be about judgment rather than grammar.

The tune

Raptor 0.6 was trained in twelve rounds in about a day on three DGX Spark boxes, each round from the raw base rather than continuing from the previous candidate. The recipe never changed: a rank-4 LoRA on the attention projections only — the fused query/key/value projection and the output projection of all 36 layers, 72 matrices, 2.2M trainable parameters — eight optimizer updates at a constant learning rate of 2e-5, one pass, no packing, no repetition penalties or stop-token repairs anywhere in the loop. The MLP, the tied embedding, every norm and the per-head attention gate are byte-identical to the base. The whole tune moves each affected weight by at most one bf16 ULP.

What changed between rounds was the corpus, and every addition was a response to a measured failure in the previous round's generated outputs. The final corpus is 668 rows, 43,084 supervised tokens, grown from 286 rows and 28,907 tokens in round one. Its families, in the order they were added:

• Source-grounded trajectories. Every example renders through the model's own chat template against real Osaurus tool schemas, with the tool observations shaped exactly as the host returns them, checked against the host's source. No evaluation prompt, identifier or answer was copied into training.
• Exact reasoning twins. Every root exists in a reasoning-on and a reasoning-off variant with identical non-reasoning content, so the tune teaches actions and finals in both modes without touching how the model thinks. This is the lesson from the 0.5 cycles: masking or templating the reasoning collapsed it; leaving it alone worked.
• Delegation boundaries. Missing task, missing agent, missing both — paired with known-target counterparts so the model learns to clarify when the target is absent and to delegate cleanly when it is not, rather than learning to always ask.
• Mixed-history mode switches. Reasoning toggled between turns before a dependent read, so a mode change mid-conversation does not leak a closing tag or lose the chain.
• Long first actions. A first tool call after up to 12K tokens of background, with the dependent path taken only from the observed result and the final answer only from the second observation.
• Resolution. The last family, and the one that named the release checkpoint. Eight fresh scenarios that separate shorthand the conversation already resolves from genuine ambiguity: give the names the user asked for, ask one concrete question when two catalogue entries share a name, ask for the destination instead of inventing an address, and never claim a shipment happened. This targeted a specific measured failure — an earlier candidate that found the right answer to a 64K ledger question, then reopened the same resolved choice over and over until it ran out of output budget without ever answering.

Every round carried the same general and numerical retention rows, so a model that got better at the harness did not get worse at arithmetic. And every round was scored against the raw base on the same frozen suites — the 182-prompt catalog, 38 short scenarios, 54 supplementary multi-turn conversations, and long-input tests at 16K, 32K and 64K tokens — with the rule that a candidate is rejected for a new severe failure class, not for ordinary 4B-model mistakes.

The selected checkpoint is deliberately modest. On the named-tool catalog it is within a few cases of the raw base (143 versus 148 of 182 strict schema passes); that suite measures syntax the base already had. Where the tune shows is in the generated conversations: observation-conditioned chains that finish, clarifications that ask one question, corrections that are taken, and the 64K ledger prompt that a prior round could not close now answered correctly in both reasoning modes on a clarified retry.

Built for 8–16 GB

The 3.7 GB bundle is a JANG_6M quantization: attention and the tied embedding at 8-bit, the MLP carrying the 6-bit budget, group size 64, every scale stored in bfloat16 so the fused bf16 kernels stay in play. The per-head attention gate and every norm are kept in full precision.

Calibration was re-captured on the tuned weights, not reused from the base. The tune moved attention, so the activation statistics that drive the fit moved with it. One 1.5M-token capture across code, agentic transcripts, tool calls, general chat, Chinese, long-context and science prompts feeds all three of the calibration methods we use: activation-aware scaling folded into 72 of 72 norm sites and 36 of 36 attention gates, an error-corrected weight fit on 180 of 181 tensors, and per-channel importance weighting. The single holdout is the tied embedding, a lookup with no input activation to condition on.

Measured on 15,839 held-out positions against the model's own bf16 weights, with calibration and evaluation prompts drawn from disjoint corpora:

bundle	size	median KL	mean KL	top-1 agreement	decode (M5 Max)
Raptor 0.6 JANG_6M (ships)	3.7 GB	0.0054	0.0132	97.7%	104.8 tok/s
base Spark-X2.5-4B JANG_6M	3.7 GB	0.0048	0.0143	97.7%	104.7 tok/s
base Spark-X2.5-4B JANG_8M	4.4 GB	0.0017	—	98.7%	91.6 tok/s

The 8-bit build is closer to the reference and 0.7 GB larger; both are well inside the range where a quantization is invisible in use, and the 6-bit build is 14% faster. On a model whose job is round trips, we took the speed. There is no 4-bit tier. On the two previous Raptor bases a 4-bit build landed 5× to 18× further from the reference than the 6-bit one for a saving of under 1 GB, which is the wrong trade for a model whose job is precise tool calls, so we did not build one here.

There is no stock-MLX comparison row because none can exist yet — mlx-lm has no spark2_5 architecture, on any release or on main. The runtime ships with Osaurus.

For llama.cpp users there is a GGUF sibling — Q6_K at 3.4 GB (mean KLD 0.0047 against bf16) and Q4_K_M at 2.6 GB (0.040), both fitted with an importance matrix. It needs the vendor's llama.cpp fork, which carries the architecture. Osaurus runs the MLX bundle, not the GGUF.

How it performs

The tune was selected under an explicit tolerance for ordinary 4B-model mistakes — a stale tool name, a schema slip, an arithmetic error — and an explicit intolerance for the failures that defined 0.5: sustained non-functional repetition, leaked reasoning boundaries, broken termination, and the inability to produce a useful final answer.

Every generated output of the selected checkpoint was read in full rather than scored by a grader alone: 182 catalog prompts across 91 Osaurus tool schemas in both reasoning modes, 38 short scenarios, 54 multi-turn supplement conversations, and long-input tests at 16K, 32K and 64K tokens. Across all of them the model ended with a native stop in every case but one, which hit the output cap and finished cleanly when given a larger budget. There were no reasoning-off closing-tag leaks and no whole-output incoherence. Observation-conditioned tool chains completed, corrections were taken, denied actions were handled, and the 64K ledger prompt that an earlier round could not close was answered correctly in both reasoning modes on a clarified retry.

Because the tune moves attention by at most one bf16 ULP and the quantization tracks the tuned weights at a median KL of 0.005, the base model's published capability numbers are a fair proxy for what ships. Thinking mode, against the two same-size models people would otherwise reach for on this hardware:

benchmark	Spark-X2.5-4B (Raptor 0.6 base)	Qwen3.5-4B	Gemma4-E4B
BFCL-V4	65.1	50.3	36.9
τ²-bench	75.1	79.9	42.2
τ³-bench	30.4	6.7	10.1
MCP-Atlas	54.6	40.8	15.0
SWE-Bench Pro	44.4	29.4	4.0
IFEval	93.0	89.8	45.3

These are as compiled in the Spark-X2.5 model card, which mixes its own runs with figures from the other models' cards. We did not re-run them, and they measure the base rather than the harness behaviour we tuned for; treat them as a floor on general capability, not a claim about agent sessions in Osaurus.

Other things to know, carried over from the checkpoint's own handoff notes:

• Reasoning traces can be long. Hard prompts can exceed 8,192 output tokens of reasoning. Budget for real input plus output rather than silently truncating history. Reasoning off is a supported, tested lane.
• Long-context recall is not guaranteed. A 64K retrieval prompt was answered correctly on a clarified retry in both modes; an earlier attempt copied and re-examined the source text instead of answering. Allow clarification and retry.
• Identity is not enforced. The model will often describe itself as Spark. We preferred quality over persona training, and the 0.5 cycles showed persona nudges cost more than they returned.
• Outside the harness, keep a system prompt. Osaurus always sends one that describes the available tools, and reasoning is on by default. If you serve the model elsewhere with reasoning off and no system prompt, one line telling it to call available tools keeps mid-conversation tool offers reliable.

Raptor 0.6 ships with the base model's sampling profile — temperature 1.0, top_p 0.95, no top_k truncation, no repetition penalty — stamped in generation_config.json. That is a change from 0.5, which needed temperature 0.7 and a repetition penalty to stay on the rails. This base does not, and we did not add one.

What changed in Osaurus

Debugging 0.5 turned up several harness defects that were not the cause of the loops but were wrong regardless, and fixing them benefits every model Osaurus runs. Osaurus 0.25.0 carries all of it, and is the minimum version for Raptor 0.6 because it is the first release whose engine includes the Spark-X2.5 runtime.

• The model dispatcher fails loudly on an architecture it cannot identify instead of routing to a lookalike code path.
• A tool whose validator names a scope now reads that scope instead of refusing it, and an unchanged deterministic error is not re-executed.
• A message typed while the agent is running lands before the pending assistant turn, never between a tool call and its result.
• The repetition-penalty window is 64 tokens when a penalty is set, and a model that has run out of retries ends in one tool-free status step rather than an empty reply.
• Repeated tool_not_found errors surface an advisory naming the tools that actually exist.
• Disk-cache rows are published atomically and validated on read, so a partial row can never restore as zeros.

A further lever from the 0.5 investigation — letting a bundle declare that its historical reasoning should be omitted from the prompt — is staged but not needed here. Ling's template kept the model's own reasoning in history unconditionally; Spark's renders it only when the app sends it, so the harness is already in control of what Raptor 0.6 sees of its own past thinking.

What it's for

Raptor 0.6 is a resident model for machines where residency is expensive. Good fits: agent loops and tool calling, file and folder work, document-versus-spreadsheet checks, drafting emails and summaries as reviewable files, shell one-liners in the sandbox, and long multi-turn sessions where a person is waiting on each step. It is bilingual in English and Chinese and reasons by default.

Less good, by design: it is not a coder — route real code-writing to a big model; Raptor 0.6 runs the workflow around the code and will happily execute code it did not write. It is not a research synthesizer; it reads your files, not twenty sources. And it is a 4B model: expect ordinary small-model slips, and keep your harness's approval step in front of anything that mutates state.

Downloads & usage

The model runs in Osaurus 0.25.0 or newer with tools included and no setup — download it from the model list. The bundle declares its own reasoning parser, tool dialect and sampling profile, so the harness wires itself with no per-model configuration. Outside Osaurus it needs an MLX runtime that knows the spark2_5 architecture; a bare mlx_lm.load will not resolve it on its own.

• Raptor 0.6 JANG_6M — 3.7 GB — Hugging Face
• Raptor 0.6 GGUF — Q6_K 3.4 GB, Q4_K_M 2.6 GB, with importance matrix — Hugging Face

Base model XHToken/Spark-X2.5-4B is Apache-2.0 licensed; Raptor 0.6 inherits it.

Notes

• The harness is the contract. The model is trained against Osaurus's real tool surface; that is where its edge is, and keeping it current as the surface evolves is a maintenance cost we accept rather than hide.
• Measure the untuned base first. Every 0.5 loop report and every gate failure on this release was replayed on the untuned base, on the unquantized weights and with the cache off before anyone called it a tune defect or quantization damage. One of the two 0.5 loop mechanisms was not in the weights at all, and the other was in the base.
• Numbers are reproducible. Quantization KL is measured on a held-out corpus disjoint from calibration; throughput figures are medians of repeated live-runtime probes at a fixed prompt shape with the first probe discarded; base-model benchmarks are the vendor's and are labelled as such.

@misc{osaurus2026raptor06,
  title  = {Raptor 0.6: a 4B on-device agent model for small Macs},
  author = {Osaurus},
  year   = {2026},
  url    = {https://osaurus.ai/models/raptor-06}
}