Learn AI by Building
From your first dataset to production agents โ deep-dive series, hands-on notebooks, and experiments you can rerun yourself.
Paper of the Week
All Issues โPaper of the Week #10 โ Reorganize the Agent's Workspace, and Haiku 4.5 Runs at a Quarter of the Cost
This week: EnvOpt (UC Berkeley, UChicago, Bespoke Labs) lets one agent rebuild the workspace another agent works in, without answer keys. On AutomationBench, Claude Haiku 4.5's cost fell 72.7% and its pass rate rose from 2.33% to 16.67%. The same workspace cut Opus 5's cost by 68.1%, with pass rate moving from 33.83% to 35.17%. We checked every number against the paper's tables and note where the shared figures need context. Also: SlackBench, from the same group, where the best of 33 agent setups scored 80.25% on questions about a research team's Slack, and reading the workspace as plain files beat Slack tools by 7.85 points on average.
Premium Series
Our Products
Courses and code built from what we measure here
Video Courses
13 hands-on courses โ quantization, diffusion, RAG, on-device AI, decision models. $199 lifetime bundle, first 3 lectures of every course free
LLM Quantization and Compression Hands-On
GPTQ, AWQ, GGUF, QLoRA โ fit LLMs into the memory you have. The course behind our KV-cache measurements
Free Benchmark Code
The scripts and raw results behind the measurement posts, starting with the KV cache harness. Free with an account
Premium Series
140 deep-dive posts across 21 series, bilingual KO/EN, with production-ready code and notebooks
Latest Posts
View All โ
Inkling 2 Mini on 12, 16, and 24 GB GPUs: Which Quant Fits, and Doubling Context with -ub 128
We tested Unsloth's Mainline_Support GGUFs (UD-Q2_K_XL, Q3_K_XL, Q4_K_XL) of Inkling 2 Mini on llama.cpp b12611 on an A100 against 8, 12, 16, and 24 GB card budgets. KV allocates for 48 loop visits rather than 24 physical layers (1,264 MiB at 32K), while the CUDA compute buffer takes 3,356.2 MiB at default -ub 512. Lowering -ub to 128 cuts compute memory nearly in half twice, keeping 43K on 12 GB (Q2), 41K on 16 GB (Q3), and 121K on 24 GB (Q4) within budget after a 30K-token prompt. On 4 Python coding tasks at 16K, Q2 scored 0.975 within 12 GB (11,586 MiB peak, 174 MiB margin).

MiniMax H3 Video + Audio on 8 GB VRAM? Inside FreeVideo's Adaptive Execution Engine
We ran FlashML-org/FreeVideo's Adaptive Execution Planner across 9 VRAM/RAM configs on an A100 and aggregated 146 community benchmarks. On 8 GB VRAM + 16 GB RAM, 0 of 50 transformer blocks stay in VRAM and one 10s 768p clip reads 76.1 GB from SSD plus 173 GB over PCIe; upgrading system RAM to 32 GB cuts SSD reads to 0 GB.

Running a 10-File Ops Check as an Agent Loop vs. a Generated Python Script: 270x Input Tokens and a $2,828 Sum Error
We ran the same 10-file daily transaction triage (180 records, 3 anomalies) on Claude Haiku 4.5 with max_tokens=4096 two ways. Across 3 runs, the runtime agent loop burned 102,546 input tokens and 29.9s per run and submitted a sum off by $2,828.29 every time. Asking the same model once (379 input tokens) to generate a Python script passed 30/30 runs in 33.7ms each, costing 8,637x less hourly even with 100% prompt caching.

Catching Silent Failures in Agent Runs: Comparing LLM Judges and a Python Code Gate on 20 Cases
We tested 20 hand-crafted agent-run fixtures (12 silent failures, 8 valid passes) across self-report, two Claude Haiku 4.5 judges, and a Python code gate. A summary-only judge caught 5/12 failures; a full-diff judge caught 11/12; the structured code gate caught 12/12 in 0.33 ms at zero tokens.

Mounting 8 MCP Servers in a Custom Harness Burned 47,800 Tokens: Measurements and Diet Comparison
We measured 8 MCP servers (134 tools) in a custom harness that forwards all tools on every turn, using Anthropic's count_tokens API and Claude Haiku 4.5. All 8 took 47,800 Claude tokens (Notion alone: 24,441). Across 15 single-run tasks, all three setups scored 12/15 while allowlisting cut billed input tokens by 81.8% and deferred loading cut them by 91.3%.

Swapping vLLM for SGLang: the two-card answer held, but on one card 5 of 32 users waited in line
Same Qwen3.8-27B and two A100 PCIe cards, swapping vLLM for SGLang 0.5.23. Two-card winners held (split for 1 user, copies for 64), but on one card SGLang capped concurrency at 27, leaving 5 of 32 users in line with 14.4 s TTFT p95.