Learn AI by Building
From your first dataset to production agents — deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues →Paper of the Week #8 — Agents Beat All 20 Science Benchmarks, and None of 135 Follow-up Papers Passed
This week's roundup. Agent4Sci reports that frontier agents beat human-verified baselines on all 20 scientific ML benchmarks it collected, mostly through engineering; Into the Unknown reports that none of 135 agent-written follow-up papers met every condition set by the scientists who posed the problems. Also: BizAgentBench, which grades goals and procedure separately on payment-company workflows, and CarryOn, which rebuilds an agent's context from what it has produced instead of its chat history.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses — quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA — fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All →
The DeepSeek-V2-Lite GGUF We Downloaded Ran Without Its Small KV Cache: Check Two Tensor Names First
DeepSeek-V2-Lite's attention should keep a 32K-token KV cache at 972 MiB in llama.cpp. The 2024 Q4_K_M GGUF we downloaded used 8,640 MiB because it lacks the attn_k_b and attn_v_b tensors. Converting the original weights ourselves brought the cache back to 972 MiB and the process from 20,162 to 11,330 MiB. Here is how to check a file in one command and convert it if needed.
Five LoRA Variants on One Table, and the Setting or Condition Each Abstract Leaves Out
LoRA, LoRA-FA, VeRA, Delta-LoRA and LoRA+ on one table: which of W, A and B each trains, where each paper says the saving is, and the setting or condition you only find past the abstract.
Pick llama-server's -c From Two Short Runs, Not a VRAM Estimate: Load Memory Within 1 MiB at Seven Checked Lengths, and the Estimate Overshot Gemma 3 at 128K by 13.7 GiB
We started llama-server at 4K and 8K on Qwen3-8B, Qwen3.5-9B and Gemma 3 4B, drew a line through the two memory readings, and picked the longest -c under a memory budget. At the seven lengths we then started, the line predicted load memory within 1 MiB. Prompt processing added 12 to 14 MiB and pushed two picks over budget, so we subtract 16 MiB more. A simple all-layers estimate overshot Gemma 3 at 128K by 13.7 GiB.
Paper of the Week #8 — Agents Beat All 20 Science Benchmarks, and None of 135 Follow-up Papers Passed
This week's roundup. Agent4Sci reports that frontier agents beat human-verified baselines on all 20 scientific ML benchmarks it collected, mostly through engineering; Into the Unknown reports that none of 135 agent-written follow-up papers met every condition set by the scientists who posed the problems. Also: BizAgentBench, which grades goals and procedure separately on payment-company workflows, and CarryOn, which rebuilds an agent's context from what it has produced instead of its chat history.
Harness² Selector Code: What It Sees and What It Never Sees
Reading google-research/harness2 (commit 1cfe254): the step that picks one of k harness candidates with no answer key sees only the request, the source material and anonymized, untruncated outputs. A failed pick records the first final by label but is left out of scoring, and when a successful pick leads the reflector to write a lesson, the pick's stated reason is stored verbatim in the library next to the adopted harness.
Harness²: A Loop That Edits the Harness Without an Answer Key
Harness² (arXiv:2610.14224) makes the harness around a model eight editable parts and recurses on it with no ground truth: 11.5 points on WorkBuddy-Bench from one improvement step (GLM-5 under OpenCode), tested on the OpenCode and Codex substrates, and a selection step, active only above k=1, that the authors admit can pick the weaker candidate. We read the paper and ran the released demo locally.