Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues →Paper of the Week #6 — The Check Passed. What Did It Check?
This week: a Lean proof that does not match the paper it formalizes, a decision-model table whose test sets are in its training mix, labels that override definitions, memory that wins or loses depending on how much the model reads, NCCL symmetric memory gains that depend on payload size and GPU, a 40-49% token cut that needs lowercase and thinking off, and an AI prescribing pilot whose first phase has two physicians check every prescription.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses — quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All →
Can a 32 GB Mac Run a Model Three Times Its Memory? Qwen3.8-Flash-Next on an M2 Max
Qwen3.8-Flash-Next's 4-bit weights are about 105 GB. With slotstream, which reads the model's experts from the SSD, a 32 GB MacBook Pro (M2 Max) ran it at 9.5 tokens per second for short replies and read an 8K prompt in 55 seconds. It scored 280/300 on GSM8K and 156/164 on HumanEval, and gave byte-identical answers across repeats and across two memory plans. A long job of 300 questions took 3.6 hours, and one long request was dropped for memory pressure and succeeded on retry. Against Qwen3.5-9B, which fits in memory, GSM8K couldn't be told apart (93.3% vs 91.0%) and HumanEval solved 18 more (95.1% vs 84.1%), while the 9B decoded 3.5 times faster.
A 2-Bit Model Remembers How to Be Right and Forgets How to Be Sure
Cutting decider-4b to 2-3 bits kept its accuracy (86.4% → 85.6%) but took away its decisiveness, and the error pointed toward the base model it was fine-tuned from (cosine 0.29 for the Q2_K file, against about 0.02 for a random direction). Random noise of the same size, five seeds, also made it less decisive, often more so, but had no such direction (−0.04 to 0.07) and moved accuracy between 81.6% and 89.6%. Noise only leaned that far once it was big enough to drop accuracy to between 13% and 32%. One refitted temperature brings the probabilities back.
How Much of an Uncertainty Score Is Just Answer Length? U-Space on Qwen3.5-4B
A new paper finds that answer length alone flags a language model's wrong answers almost as well as many uncertainty scores. We ran the authors' code on Qwen3.5-4B with their items: length alone reached AUROC 65.3 on three benchmarks, from 75.2 on TriviaQA to 48.6 on SuperGPQA. After controlling for length, U-Lens beat MSP (+8.4) and length alone (+4.4 before control), but its edge over mean log-loss (+1.1) was too small to call. On the three benchmarks in the main comparisons up to a third of answers hit the 16,384-token limit and were excluded; on Omni-MATH, left out of the comparisons, 92.5% did.
How Far Can You Shrink a Decision Model? decider-4b from BF16 to Q2_K on llama.cpp
We ran the open decision model decider-4b at five GGUF sizes, from BF16 (10.4 GiB of GPU memory) to Q2_K (4.4 GiB), on 500 TREC questions. Accuracy differences were too small to detect at every size. The probabilities moved: log-loss got slightly better at Q4_K_M and worse at Q8_0 (barely), Q3_K_M and Q2_K. Q2_K changed 13% of the answers and Q3_K_M cut the share of answers above 0.9 from 40% to 23%. Q4_K_M halved the memory; no accuracy difference was detected and its log-loss improved.
Can You Act on Microsoft-Decision-1's Probabilities? We Swapped the Labels on 500 Questions
We sent 500 TREC questions to Microsoft-Decision-1 and Jev 1.13 through OpenRouter, and ran two local decision models, decider-4b and laya, with normal labels and with option names swapped against their definitions. Decision-1 followed the definitions (95.2% against 96.4%, not a difference by our rule) and at a 0.9 threshold handled 80% of the swapped cases on its own, 5 of those 400 wrong. laya followed the labels and was wrong on 110 of the 115 swapped cases it would have handled. The same request sent twice gave slightly different probabilities on Decision-1.
Calling Microsoft-Decision-1 on OpenRouter: Six Things We Hit on Day One
A short field guide from our first day with Microsoft-Decision-1 through OpenRouter: it uses the System One endpoint, not chat completions; choice options must be a map; confidence is not the top probability; the model ID carries a date; requests hit rate limits; and the same question cost us 3.5 to 4.8 times as much on Jev. Working Python included.