Allosaurus v0.1

Should you download it?

Allosaurus is for Macs with 48 GB of memory or more, and for work that benefits from a large model: coding, agent tasks that use tools, security work, and reasoning through problems step by step.

You need Osaurus 0.25.18 or newer, and 0.25.20 or newer for the full speeds below. The download is about 95 GB. That is much larger than the 35 GiB it uses in memory, because one big part of the model stays on your SSD and is read from there as needed (explained below). Make sure you have the disk space.

On a smaller Mac, Raptor 0.6 runs on 8 GB, and Which model fits your Mac? shows what runs well on yours.

What we changed

In short: Same model, same knowledge. All of our work went into how it is compressed, so that it fits in less memory and loses less along the way.

Allosaurus is Qwen's Qwen3.8-Flash-Next, quantized and calibrated specifically for Osaurus. Every weight comes from Qwen's release, and we didn't fine-tune it. The work went into what decides how a quantized model behaves: how each weight is stored, which parts stay in memory, how many bits each part gets, and what data the rounding was tuned on.

That data was chosen for how the model gets used in Osaurus: tool calling, coding, security work and reasoning that has to finish. Every calibration sample went through the model's own chat template, the same one Osaurus uses to build a request. Tool calls are in the XML format Osaurus parses for this model. We then checked the results on the decisions an Osaurus agent depends on: whether to call a tool or answer, and when to stop reasoning.

The goal was narrow. Our smallest earlier release of this model, JANG_1L, needed about 41 GiB of memory. We wanted something that fits more comfortably on a 48 GB Mac and stays closer to the original model. Allosaurus needs about 35 GiB. Measured against the unquantized model, its median KL is about 3x lower than JANG_1L's and its top-1 agreement is about 9 points higher. Both terms are explained further down.

The model underneath

In short: The model is huge, but only a small slice of it works on any given word: about 6.7 billion of its 125 billion parameters. That is what makes it practical to run on a Mac.

Qwen3.8-Flash-Next is a 125-billion-parameter mixture-of-experts model. A few things about it shape everything else:

• Only a small slice runs per token. Each of its 48 layers has 512 "routed experts", small feed-forward networks. A router picks 10 of them for each token, and one shared expert always runs. About 6.7B parameters are active per token.
• Two kinds of attention. 36 layers use linear attention, which keeps a fixed-size running state instead of a cache that grows with the conversation. The other 12 use sparse attention: a small indexer picks which earlier tokens are worth looking at.
• A very large lookup table. The model has an n-gram embedding table with 51 billion entries. For every token it looks up 16 rows based on the last few tokens and adds them to the model's input. In full precision this table is bigger than the rest of the model put together.
• It also has one multi-token prediction (MTP) layer for speculative decoding, and a vision tower for images and video.

What sits in memory and what doesn't

In short: One very large part of the model, a 54 GiB lookup table, stays on your Mac's SSD, and Osaurus reads only a few kilobytes of it for each word. Everything else, about 35 GiB, sits in memory. That split is why the model fits a 48 GB Mac at all.

The lookup table is the reason this model fits a 48 GB Mac at all. It is 54 GiB even after quantization, but each token reads only 16 rows of 180 bytes from it. So Osaurus leaves the table on the SSD and reads just those rows as each token needs them. It deliberately keeps the table out of the operating system's file cache: random reads from a 54 GiB file would only push more useful data out of memory. A few kilobytes per token is a trivial load for an Apple SSD.

Everything else is meant to stay in memory, about 35 GiB in total:

For one generated token, Osaurus:

1. Looks up the token's embedding in memory and its 16 table rows on the SSD.
2. Runs the 48 layers. In each, the attention step either updates its running state or looks back over the selected earlier tokens. Then the router picks 10 experts, those experts run, and the shared expert is added.
3. Turns the final output into next-token probabilities.

Each token reads about 3.4 GB of weights, and only about 0.6 GB of that is routed experts. The weights read on every token (attention, shared experts, output layer) set the speed far more than the experts do. So we used the room we saved on the experts to keep quality, not to squeeze them even smaller.

JANGH: rotate, then quantize to a codebook

In short: The experts take up most of the model's memory, so we built a new way to store them using about 2 bits per weight, where full-precision models typically use 16. Before compressing, we blend each small group of numbers so that one unusually large value can't distort its neighbors. The result is smaller than the standard 2-bit format and more accurate than the standard 3-bit one.

The routed experts are about 80% of what sits in memory, so their format matters most.

Standard 2-bit quantization splits each row of weights into groups of 64 and stores a scale and an offset for each group. With that overhead included it costs about 2.5 bits per weight. It also handles outliers badly: one large weight in a group sets the scale, and the other 63 weights get rounded coarsely.

JANGH, our expert format, does three things differently:

It rotates first. Every block of 32 weights is multiplied by a fixed Hadamard matrix, a simple ±1 transform that mixes the 32 values evenly. An outlier stops being one big value: its size gets spread across the whole block. After rotation the blocks look very similar to each other, so one fixed set of levels fits all of them. The rotation undoes itself when applied a second time, so at runtime Osaurus rotates the input going into each expert and uses the stored weights exactly as they are on disk.

It uses a codebook instead of an evenly spaced grid. At 2 bits each weight is one of four values, placed by a small formula that is tuned to the shape of the rotated weights. The kernel calculates the values directly, with no lookup table.

It stores one scale per row. There are no offsets and no per-group scales, which brings the cost down to 2.006 bits per weight. The experts come to 28.1 GiB in JANGH, against 35.6 GiB in standard 2-bit.

On held-out data, 2-bit JANGH experts reproduce the original expert outputs more accurately than standard 3-bit quantization does.

Calibration

In short: Compressing a model always loses some detail. Calibration decides which detail to protect, by showing the compressor examples of the work the model will actually do. We used coding, tool use, security and reasoning, formatted exactly the way Osaurus talks to the model.

Quantizing is mostly about deciding where the rounding error goes, and that depends on what the model will actually be asked to do. Calibration is how we tell the quantizer.

The corpus

In short: The examples lean toward real Osaurus work. The tools we test on later were kept out of them, so the test results aren't inflated.

We built a 607,000-token calibration set weighted toward the work this model does in Osaurus (code, security, tool use and reasoning), rather than general web text:

Every sample is rendered through the model's own chat template, the same way Osaurus formats a real request.

The tool-calling conversations are synthetic. They use 13 everyday agent tools: running shell commands, reading, writing and searching files, HTTP requests, SQL, web search, calendar, email and a calculator. Each conversation offers between two and six tools. Some requests need one call, some need two in a row, and some need no tool at all, so the model also learns to answer directly when no tool applies. Some conversations include reasoning and some don't. All calls are in the model's native XML format. We kept five more tools out of calibration entirely and used them only for testing, so the tool results below are measured on tools the model never saw during calibration.

Reasoning calibration

In short: Heavily compressed models can lose the knack of finishing their thinking and moving on to an answer. We tested that directly: Allosaurus stops thinking where the original does almost exactly as reliably.

Qwen3.8-Flash-Next reasons before it answers, at three effort levels (low, medium, xhigh). The reasoning traces in the corpus are rendered at a randomly chosen effort, so the quantizer sees all three as they really appear.

The decision we care about most is when the model stops thinking. On an earlier model we saw 2-bit quantization make it less willing to end its reasoning. It didn't loop; it simply never closed. So for Allosaurus we tested that decision directly. We took traces where a 6-bit build reasoned for 1,500 to 5,800 tokens and then stopped, and measured how likely each build was to stop at the same point:

• the unquantized model: 0.99994 (median)
• Allosaurus: 0.99983 (median)
• JANG_1L: as low as 0.55 on its worst trace

GPTQ on every expert

In short: Each of the model's 24,576 expert pieces was compressed using examples of what that piece actually handles, rather than one set of examples for all of them.

For the routed experts we used GPTQ. The quantizer rounds a matrix one column at a time, and after each column it adjusts the columns not yet rounded to absorb the error. It needs to know which input directions matter, which it gets from the inputs that actually reached that expert during calibration. We collected those statistics separately for each expert, which makes 24,576 expert matrices each calibrated on its own data. In most layers, more than 450 of the 512 experts saw at least 1,000 calibration tokens.

We also tried a variant that adjusts GPTQ's scales upward, because it fixed a reasoning problem on another model. On this model it was worse on every test set, so Allosaurus ships plain GPTQ.

The rest of the model

In short: The parts that read images and that speed up generation were tuned for the first time in this release, and both came out closer to the original for it.

The parts kept in memory use importance-weighted quantization: activation statistics from the same corpus pick the scales that protect the weights that matter most. They also use AWQ, which rescales inputs so the most important channels lose less precision.

Two parts had never been calibrated in our earlier releases of this model:

• The vision tower. We calibrated it on 985 images and video clips: documents, screenshots, charts, invoices, terminal captures, and real and screen-recorded video. Calibrated beat uncalibrated at every bit width we tested.
• The MTP layer. It got its own calibration pass. Its first guess now matches the next token 70.5% of the time, against 70.7% for the unquantized model.

Deciding where the bits go

In short: Memory is a budget. We measured which parts of the model lose the least quality when compressed further, shrank those, and left the sensitive parts alone.

We didn't follow a fixed rule like "attention always gets 8 bits". Starting from a version where everything outside the experts was 8-bit, we lowered one group at a time. For each group we measured how much quality dropped per GiB saved, then took the cheapest savings until the in-memory part reached about 34 GiB:

At that budget the experts stay at 2 bits throughout. There was no room left to give any expert layer 3 bits.

How close is it to the original?

In short: We ran Allosaurus and the full-size model side by side on more than 500,000 tokens of text it was never tuned on, and compared their predictions. Even a near-perfect copy can't agree 100% of the time, so each test has a realistic ceiling. On general text Allosaurus lands 13 points below that ceiling; our previous build landed 22 below.

All the numbers below come from running Allosaurus and the unquantized model on the same text and comparing their predictions position by position. That covers 514,498 positions from four sets we never used for calibration:

• general text: code, security, science, systems, chat
• reasoning at all three efforts
• tool-calling conversations
• office, finance and math tasks

We report two numbers:

• Top-1 agreement is how often both models pick the same most likely next token.
• KL divergence measures how much the whole spread of next-token probabilities changed. Osaurus samples from that whole spread by default, not just the top pick, so KL catches changes that top-1 misses. Lower is better.

One thing to know first: 100% top-1 isn't achievable. Many next tokens are near-ties, and any tiny change to the weights flips some of them. We measured this by adding 0.1% noise to a single weight matrix of the unquantized model. On general text, that version agrees with the original only 94.5% of the time. That is the realistic ceiling.

How to read the table: the ceiling is the best realistic agreement, higher percentages are better, and lower KL is better.

On general text, Allosaurus lands 13 points below the ceiling and JANG_1L lands 22 below, so Allosaurus loses about 40% less while using 6 GiB less memory.

Specific decisions:

• Calling a tool versus answering directly: across 108 conversations on tools the model never saw in calibration, Allosaurus made a different choice from the original model 11 times, compared with 34 for JANG_1L.
• Ending its reasoning at the same point as the original: 120 of 120 reasoning traces and 297 of 300 office tasks.
• xhigh effort with no length limit: Allosaurus finished reasoning on its own 90.5% of the time. JANG_1L finished 82.1% of the time and our 6-bit release 97.6%. It doesn't loop, but it sometimes reasons longer than the original, so give xhigh a generous token limit or use medium.

Where it's weaker: on long documents, JANG_1L stays closer to the original model by these measures. Allosaurus still did better on our long-context recall tests (19/20, 18/20 and 17/20 at 16k, 32k and 64k tokens).

Single-request checks in Osaurus also passed: reading every line item off an invoice image and totalling it, answering questions about short videos, 35 of 40 networking tasks and 48 of 60 office and finance tasks with exact answers.

Speed

In short: A small helper guesses the next few words, and the full model checks all of the guesses in one go. The answer still comes from the full model; it just arrives faster. It helps most on code and predictable text, and least on open-ended writing.

Allosaurus ships with its multi-token prediction (MTP) layer, and Osaurus uses it by default. On each step the MTP layer guesses the next few tokens. The full model then checks all the guesses in a single pass and keeps every token up to the first wrong guess. Osaurus adapts how far ahead it guesses based on how many guesses are being accepted. The output is still the full model's output; MTP only changes how many tokens each pass produces.

How much it helps depends on how predictable the text is. Below are measurements on an M5 Max in Osaurus with MTP off and on (Adaptive), taken in four fresh runs in the order off, on, on, off. Each run used the same ten prompts with Qwen's default sampling and thinking off, timed over 300-token windows:

Open-ended prose gains little, because there are many plausible next words and fewer guesses land. Code and simple, predictable text gain the most.

In a real Osaurus conversation it holds up with longer context too. One follow-up turn in a tool-using chat had a 9,230-token prompt. Allosaurus started answering in about 1 second and generated at 101 tok/s. The Osaurus app itself used 11–13 GB of memory while doing so, because the expert weights are memory-mapped from disk and paged in by macOS rather than copied into the app.

Using it

• Requires: Osaurus 0.25.18 or newer (0.25.20 or newer for the MTP speeds above).
• Reasoning effort: low, medium or xhigh (xhigh is the default). Reasoning can also be turned off.
• Inputs: text, images and video.
• Tools: calls tools in Qwen's native XML format, which Osaurus parses.
• Speculative decoding: MTP is on (Adaptive) by default. Turn it off in the model settings if you want plain one-token-at-a-time decoding.
• Defaults: temperature 1.0, top-p 0.95, top-k 20, as Qwen recommends.

Allosaurus was quantized and evaluated by Jinho Jang. The base model is by the Qwen team.

Glossary

• Parameter (weight): one of the numbers a model learned during training. Allosaurus has 125 billion of them.
• Token: the unit a model reads and writes, roughly three-quarters of an English word on average. Speeds are given in tokens per second (tok/s).
• Quantization: storing each weight with fewer bits so the model takes less memory. Like saving a photo at a lower quality setting: a much smaller file, with some detail lost.
• Bits per weight: how much storage each weight gets. Full-precision models typically use 16; Allosaurus's experts use about 2.
• Unquantized model: the original, uncompressed model from Qwen. Every comparison on this page is against it.
• JANG_1L: our earlier, smallest 2-bit build of this model, used as the comparison throughout.
• JANGH: our new 2-bit storage format for the experts, introduced with Allosaurus.
• Mixture of experts (MoE): a model split into many small specialist networks, called experts. For each token a router picks a few to run, so most of the model sits idle at any moment. A shared expert runs on every token.
• Attention: how the model looks back at earlier parts of the conversation. Linear attention keeps a fixed-size running summary; sparse attention rereads only the earlier tokens an indexer picks as relevant.
• N-gram table: a very large lookup table keyed on the last few tokens, which adds extra context to each token's input.
• Memory-mapped: the file stays on disk and macOS loads the pieces that are needed into memory, instead of the app copying the whole file in.
• Chat template: the exact format a conversation is written in before the model reads it.
• Calibration: showing the quantizer example inputs so it knows which weights matter most. The set of examples is the calibration corpus.
• GPTQ: a quantization method that rounds weights one column at a time and adjusts the remaining ones to make up for the error.
• AWQ: a method that rescales inputs so the most important channels keep more precision.
• Hadamard rotation: a simple, fixed transform that mixes a block of numbers evenly, spreading out any unusually large value.
• Codebook: a small fixed set of allowed values that each weight is rounded to.
• Top-1 agreement: how often two models pick the same most likely next token.
• KL divergence: how much the full set of next-token probabilities changed between two models. Zero means identical; lower is better.
• Reasoning effort: how long the model thinks before it answers: low, medium or xhigh.
• MTP and speculative decoding: a small extra layer guesses several tokens ahead, and the full model verifies all of them in one pass, producing more tokens per step.
• Vision tower: the part of the model that reads images and video.
• GB and GiB: Mac memory is sold in GB; model sizes on this page are in GiB, which is about 7% larger. 35 GiB is about 37.6 GB, and a 48 GB Mac has about 44.7 GiB.