Thinklite-27B: Deleting the Reasoning Models Don't Need

Reasoning models are often finished before they stop. In one of our tests we gave Qwen3.8-27B a payments bug: retry code, an app log, and a provider log, and the question of which duplicate charges to refund. Fifteen percent of the way into its reasoning, it had the right four charges and the right total, $217.98. It spent the remaining 85% checking, restating and re-deriving that answer, writing the total eight times before it replied eleven minutes later.

Thinklite-27B is Qwen3.8-27B trained to stop when it is done. On the same request, it gave the same answer in 2 minutes 37 seconds, with less than a quarter of the tokens. Across six public benchmarks at reasoning_effort="xhigh", it uses 47% less reasoning than Qwen3.8-27B and writes half as many output tokens, so a user waits about half as long for each complete answer. It keeps 99.6% of the base model's average accuracy: 89.8 against 90.2. The weights are open under Apache-2.0 at Sparset/Thinklite-27B.

The payments-bug request, run side by side: both models get the same prompt and seed on the same GPU hardware.

Two other fine-tunes of the same base model target the same problem, so we measured them alongside: ThinkingCap-Qwen3.8-27B from BottleCap and Swift-1.5-Qwen3.8-27b from UkisAI. At xhigh, Thinklite-27B saves the most reasoning, 47.0% against 45.8% and 35.6%, while both other models keep more accuracy, 1.5 and 1.3 points more on average. At reasoning_effort="low" the gap in savings widens (28.0% against 16.4% and 12.7%), and so does the accuracy cost: Thinklite-27B keeps 97.5% of the base model's accuracy there, 2.2 points below. The rest of this post explains how it was trained, shows what changes in its reasoning, and sets out where it is weaker.

Why Reasoning Runs Long

Models like Qwen3.8-27B think in plain text before they answer, and the reasoning_effort setting controls how much. At xhigh the model reasons thoroughly, which pays off on hard problems, but it does not notice when a problem turns out to be easy. Much of the extra text is the model talking itself into confidence: restating the question, solving it a second time, checking a sum it has already checked.

None of that is free. A reasoning model writes its thinking at the same speed as its answer, so every unneeded token adds to the wait and to the serving cost. On our suite, Qwen3.8-27B writes 11.5 million output tokens at xhigh, and for a single user the median LiveCodeBench answer takes more than twelve minutes.

The obvious fixes change more than the length. A length penalty rewards brevity directly, including brevity that skips a step the problem needed. Training on another model's short answers teaches that model's habits along with its length. Our own first attempts, which fine-tuned the model on edited copies of its reasoning, shortened answers at low effort but barely moved the long reasoning at xhigh. We wanted the model to learn, from its own behaviour, when it is done.

How Thinklite-27B Learns to Stop

Thinklite-27B is trained in two steps. Both start from answers the model writes itself.

Learning From an Instructed Copy of Itself

Qwen3.8-27B can already reason briefly when it is told to. Asked to keep its reasoning short and to the point, it skips most of the re-checking and still answers well. The catch is that this needs the instruction, and it is not how the model behaves by default.

On-policy distillation turns the instructed behaviour into the default. The model answers a training prompt on its own, with no instruction. A frozen copy of the model then reads that same answer with the efficiency instruction added to its prompt and scores every token: where the instructed copy would not have written a token, such as the start of a redundant check, the score is low. Training moves the model's token probabilities toward the instructed copy's, using a per-token reverse KL divergence. Because the model is corrected on text it actually wrote, it changes its own habits instead of imitating someone else's style, and the instruction is no longer needed when you use it.

Two safeguards keep this from becoming brevity at any cost. Only answers checked as correct are used: math against reference answers, code against unit tests, structured output against its schema and instructions against their rules. And low-effort answers are trained toward the unchanged model, to limit drift at that setting.

Preferring the Shorter of Two Correct Answers

Distillation shortened math, knowledge, and code reasoning the most. On instruction following and function calling, the reasoning stayed comparatively long, much of it deliberation over formatting rules and tool choices the model already handles correctly. For those tasks, we added preference optimization.

The model answered each prompt several times. From its correct answers, we paired the shortest with a much longer one, so both sides of a pair are right and the only thing the pair can teach is length. The visible answer on the preferred side had to stay complete, which keeps the savings in the thinking. We trained on these pairs with SimPO, a reference-free variant of direct preference optimization (DPO) that compares length-normalized likelihoods; we compared it with DPO and kept SimPO because it held accuracy better on held-out prompts.

Preference training pushes steadily: every step makes the model a little shorter, and past some point it starts losing answers. So we saved several checkpoints and chose one with a rule fixed before training, on held-out prompts the model never trained on: no group of tasks could lose more than a few correct answers, and instruction-following reasoning had to fall by at least 15%. The released model was the only checkpoint that met every condition; later ones were shorter but missed more instructions.

The trained changes are merged into ordinary bf16 weights. Thinklite-27B keeps the architecture, tokenizer, chat template and reasoning_effort control of Qwen3.8-27B, and it generates tokens at the same speed (26.3 against 26.2 tokens per second for a single user on our GPU).

What the Benchmarks Show

We ran Thinklite-27B, Qwen3.8-27B, ThinkingCap-Qwen3.8-27B and Swift-1.5-Qwen3.8-27b on one frozen suite: GSM8K (1,319 problems), AIME 2025 (30 problems, two samples each), a 1,000-question stratified subset of MMLU-Pro, IFEval (541 prompts, prompt-level strict), LiveCodeBench v6 (131 problems, pass@1) and BFCL v4 non-live function calling (1,240 cases). Every model got identical prompts, sampling, token caps, and scoring, and all answers were scored together on one machine. In the tables, each cell gives accuracy in percent and, for the three fine-tunes, the change in reasoning tokens against Qwen3.8-27B. The appendix has the full setup.

At xhigh: Half the Reasoning, 99.6% of the Accuracy

Thinklite-27B cut reasoning on every benchmark: most on MMLU-Pro (69%) and GSM8K (63%), least on BFCL (28%), where the base model already reasons briefly. On four of the six benchmarks its accuracy difference from the base model is not distinguishable from zero in a 95% paired-bootstrap interval. It lost accuracy on MMLU-Pro (84.4 against 86.3) and IFEval (89.1 against 91.5), and both of those losses are outside the interval. AIME 2025 moved in its favour (95.0 against 93.3), but with 60 graded answers, that is within noise.

Compared with the other two models, the result is a trade-off, not a clear win (Figure 2). Thinklite-27B saves more reasoning than ThinkingCap-Qwen3.8-27B and much more than Swift-1.5-Qwen3.8-27b, but both keep more accuracy: they reach 100% on AIME 2025 and score about two points higher on MMLU-Pro. The savings also fall in different places. ThinkingCap cuts more on GSM8K, IFEval and BFCL; Thinklite-27B cuts more on AIME 2025, MMLU-Pro and LiveCodeBench, where answers are longest.

At Low Effort: More Savings, 97.5% of the Accuracy

At reasoning_effort="low", where reasoning is already shorter, Thinklite-27B still removes 28% of it, close to twice as much as the other two models. The cost is clearer here: its average accuracy is 2.2 points below the base model, mostly from AIME 2025 (81.7 against 90.0, on 60 answers) and IFEval (84.3 against 86.1). ThinkingCap and Swift stay within half a point of the base model at this setting. If you run at low effort and hard math matters most, Thinklite-27B is not the conservative choice.

Fewer Tokens, Shorter Waits

Because Thinklite-27B generates at the base model's speed, its savings turn directly into time. We measure time to final output (TTFO): how long one user waits for the complete answer, thinking included. Over all 4,291 requests at xhigh, Thinklite-27B's TTFO is 2.0 times shorter than Qwen3.8-27B's, against 1.7 times for ThinkingCap and 1.6 times for Swift; at low effort the factors are 1.3, 1.2 and 1.1. The gain is largest on long tasks. The median LiveCodeBench answer arrives after 4.1 minutes instead of 12.3, and the median AIME answer after 3.3 minutes instead of 9.2. Where the base model already answers in seconds the gain is small: 6.8 against 7.4 seconds on BFCL. The appendix has the times for every benchmark.

Image Understanding Is Unchanged

Qwen3.8-27B also reads images, and Thinklite-27B keeps that ability. Its training data was text only and its vision encoder is byte-for-byte the base model's, but the language model that interprets what the encoder sees was retrained, so we checked it. Both models answered the same 88 questions from six public image datasets, each answerable only from the picture: a chart, a scanned document, an infographic, a math figure, an exam diagram or a photo. Both ran at reasoning_effort="low" with the same seeds and sampling as the benchmarks above. Each cell gives the score in percent and, for Thinklite-27B, the change in reasoning tokens.

Both models answered the same 83 questions correctly and missed the same five. The one score difference is partial credit on an InfographicVQA answer, a town name that Thinklite-27B spelled correctly. Reasoning fell by 24% on average across the datasets. On 18 of the questions also run at xhigh, Thinklite-27B answered 17 correctly and the base model 16, with 52% less reasoning. With 12 to 16 questions per dataset this is a check that image understanding survived training, not evidence that it improved.

What Changes in the Reasoning

Averages say how much shorter the reasoning is, not what was removed. The example below is one GSM8K problem, chosen by a rule: among the problems both models solved at xhigh, the one where each model's reasoning length is closest to its own benchmark average (443 and 163 tokens). Both runs used the same prompt and seed, and both answered 1,300.

The problem, paraphrased: Elise is selling her father's collection of 250 books over three years at $20 each. In the first year she sold twice as many books as she has sold this year; this year she has sold 45, and 50 books are still unsold. How much did she earn in the second year?

Qwen3.8-27B: 435 reasoning tokens, unedited.

We need answer math problem. Need reason step by step. Let's parse carefully.

Problem: Elise selling Dad's collection of 250 books for three years. Each book sells at 20$, she sold twice as many books in first year as she has sold in current year. Currently 50 unsold books, and her sales number this year is 45. What's total amount money earned in second year?

We need infer years: three years: first year, second year, current year (third?) She has been selling for three years. Current year sales = 45. First year sales = twice current year = 90. Total collection 250. Currently 50 unsold books. So total sold so far after current year = 250 - 50 = 200. Sales first + second + current = 200. First 90 + second + 45 = 200 => second = 65. Money earned second year = 65*20 = 1300. Need box.

Check if "current year" means third year? "for three years" might current year included? Yes, sales number this year is 45. There are currently 50 unsold books. If she has been selling for three years, after three years (including current) unsold 50. If current year not complete? But says sales number this year is 45, currently 50 unsold, presumably cumulative. Fine.

Need final concise with steps. Ensure maybe "has sold in current year" present tense? Could mean to date. Good.

Double-check ambiguity: She sold twice as many books in first year as she has sold in current year. Current year sales = 45, so first =90. Collection 250, 50 unsold, total sold over three years =200. second=200-90-45=65. Earnings = 1300. Box.

Thinklite-27B: 163 reasoning tokens, unedited.

We need answer math. Need think step by step.

Problem: Elise selling collection of 250 books over three years. Each book sells at 20 dollars. She sold twice as many books in first year as she has sold in current year. Current year is third year? "for three years" and current year sales number is 45. So first year = 2*45=90. Currently 50 unsold books. Total sold =250-50=200. Second year =200 - 90 -45 =65. Amount earned in second year=65*20=1300. Boxed 1300.

Need final with reasoning concise. Ensure boxed.

Both solve the problem the same way and reach 1,300 in about the same number of steps. The difference is what comes after. Qwen3.8-27B goes back to ask whether "current year" means the third year, considers the tense of a verb, and then works the whole problem a second time under "Double-check ambiguity". Thinklite-27B settles the same question in one clause and stops. Solving once and skipping the second pass is the pattern we see most often when we read its answers to short problems.

On a Longer Task

The payments bug from the start of this post shows the same thing at a larger scale. Both models answered correctly. Qwen3.8-27B had the full refund list and the $217.98 total 15% of the way through its 13,009 reasoning tokens and spent the rest verifying it; Thinklite-27B used 2,592 reasoning tokens and finished in 2 minutes 37 seconds instead of 11 minutes 5 seconds, with the GPU serving 20 such requests at once. We ran the prompt 20 times per model with the same seeds for both. Thinklite-27B was correct in 19 runs; its one miss listed the right charges but added them up wrong. The base model was correct in all 20. The run quoted here is the largest speed-up among the 19 pairs where both were correct; the median was 2.7 times. The video at the top of this post shows this run side by side.

Where a Shorter Route Costs an Answer

Skipping a check is usually harmless and occasionally fatal. IFEval, which tests whether a model follows formatting instructions exactly, is where Thinklite-27B lost the most: 27 prompts that the base model passed and it failed, against 14 the other way. The first of them asks for a blog post for teenagers, with the entire response wrapped in double quotation marks. Thinklite-27B noted the rule in its plan, opened the quote, wrote the post and ended with a signature line, so the closing quote never came. Qwen3.8-27B spent part of its reasoning on exactly this point, where the quotes must start and end and whether quotes inside the text would break them, and passed. One example does not establish a cause, but it fits where the losses are: instruction following and knowledge questions, where a final check catches small slips.

Limits

These are single runs per model and setting, and the small benchmarks carry wide intervals: AIME 2025 has 60 graded answers and LiveCodeBench 131. TTFO is computed from each answer's length and the measured single-user speed rather than timed request by request. Check instruction-following accuracy on your own prompts before relying on the model, and prefer xhigh over low when hard math matters.

Get the Weights

Thinklite-27B is a complete model and a drop-in replacement for Qwen3.8-27B, with the same chat template, at Sparset/Thinklite-27B, under Apache-2.0, the licence of the base model.

vllm serve Sparset/Thinklite-27B --reasoning-parser qwen3 --max-model-len 135168 --kv-cache-dtype fp8

Set the effort per request through the chat template, for example chat_template_kwargs={"enable_thinking": true, "reasoning_effort": "xhigh"}. The model card has the full tables and Transformers usage.

Build a Model Around Your Workflow

Thinklite-27B is an example of the work Sparset Forge does: take a strong open model, train it on its own verified behaviour toward a concrete target, here shorter reasoning, and measure what it gains and loses before anyone relies on it. The same process can fit a model to your tasks, your latency budget and your hardware, though results on public benchmarks do not guarantee the same outcome on a different workflow.

If you want a custom model, or a model running on your own infrastructure, book a meeting.

Technical Appendix

Time to Final Output by Benchmark

At reasoning_effort="xhigh":

At reasoning_effort="low":

All values are for a single user. Each answer's TTFO is its number of output tokens divided by the single-user generation speed measured on our GPU (26.2 tokens per second), plus about 0.1 seconds of prefill. All four models share the Qwen3.8-27B architecture and use the same speed. The TTFO speed-ups in the text divide the base model's total over all 4,291 requests by the model's: 2.0 times at xhigh means 11.5 million output tokens against 5.8 million.

Evaluation Setup

Every model was served with vLLM 0.30.0 in bf16 with an FP8 KV cache and 64 concurrent requests, on one NVIDIA RTX PRO 6000 Blackwell (96 GB). Sampling followed the base model's thinking-mode recommendation (temperature 1.0, top-p 0.95, top-k 20), with a fixed seed per item and sample so every model received the same seed for the same question. Output caps were 32,768 tokens, 49,152 for AIME 2025 and MMLU-Pro and 65,536 for LiveCodeBench, the same for every model; answers that reached a cap were rerun once with a 131,072-token cap. Truncated and unparseable answers count as incorrect, and no item was dropped. Scoring used math-verify for GSM8K and AIME, the official LiveCodeBench evaluator on public and private tests, the lm-eval IFEval checker on the final answer only, and the official BFCL AST checker on the 1,240 non-live core cases. Because executed tests depend on the machine, the answers of all four models were scored again together on one machine; the tables use that scoring. Accuracy intervals are 95% paired-bootstrap intervals (10,000 resamples) from the benchmark session's scoring. Training prompts were filtered against all six benchmarks by 13-gram overlap.

Release Details

For the base model see the Qwen3.8-27B model card. For the methods see on-policy distillation (Agarwal et al., arXiv 2306.13649), DPO (Rafailov et al., arXiv 2305.18290) and SimPO (Meng et al., arXiv 2405.14734).