LightOn's open 2B ColGrep-Agent cut Claude Code's API tokens by a third in code-search tests
LightOn released ColGrep-Agent-2B, a local code-search agent that coding agents can hand searches to. In 624 Claude Code sessions, delegating search cut API tokens by 34% versus grep, with no statistically significant change in accuracy.
Photo: Basile Morin / Wikimedia Commons, CC BY-SA 4.0. Hugging Face and LightOn logos via Wikimedia Commons (public domain).
LightOn introduced ColGrep-Agent-2B on the Hugging Face blog on October 12: an open, 2-billion-parameter code-search agent that runs locally and sits between a coding agent and a repository. Given a question, it searches with ColGrep, LightOn's multi-vector semantic code search, reads promising files, follows references and returns file paths, line ranges and the code itself.
The team tested it with Claude Code running Sonnet 5.5 on 52 code-search questions across 13 repositories created in 2026, running each question three times in four setups, for 624 sessions. Delegating search to the agent cut API tokens by 34.3% and cache-aware cost per question from 7.3 cents to 4.4 cents compared with grep. Accuracy was 80.8% versus 83.3% for grep, a difference the authors say is not statistically significant, though they note the confidence interval is wide enough that a meaningful drop can't be ruled out.
The trade-off is latency: end-to-end time per question rose from 13.0 seconds with grep to 21.6 seconds. A "Balanced" setup matched grep's observed accuracy while saving 21% of tokens. The authors stress the results cover code-search questions, not end-to-end coding tasks.
ColGrep-Agent-2B was trained from MiniCPM5-2B with a short supervised fine-tuning cold start followed by reinforcement learning with a GRPO variant. The model and GGUF weights are available now; the default install downloads a quantized model of about 2.5 GB and needs no Python environment or API key, according to the post.