Veer
A programming language for LLM programs whose compiler proves KV-cache memory, prefill cost and numeric error bounds before anything runs.
- Role
- Solo
- Stack

Problem
Agents, RAG pipelines and tree search are limited by KV-cache memory and prefill time. Serving engines guess at cache reuse, editing an agent’s history quietly re-prefills thousands of tokens, and the only way to know whether a quantized model is still accurate enough is to run evals. I wanted those costs to be visible in the program and checked by a compiler.
What I built
Contexts are linear values. Every operation (append, generate, fork, splice, speculate, drop) consumes its context, so leaks, use-after-free and ambiguous edits are compile errors.
A static planner, written as an abstract interpreter over paged KV segments, proves upper bounds on peak KV bytes, prefill and decode tokens, context length and splice re-prefill cost. @budget(kv = 2GiB, prefill = 12000) is a compile-time contract.
Precision types carry a proven error bound through quantization, matmul and activations across FP8, INT8, MXFP4 and NVFP4, so @accuracy(rel <= 0.25) is also checked at compile time.
A six-crate Rust workspace: parser and diagnostics, bit-exact quantization simulators, type checker, planner, a runtime with a simulated engine and an OpenAI-compatible HTTP engine, and the veer CLI (check, plan, run, types, explain).
Results
- v0.1.0 released with binaries for macOS (arm64, x64), Windows (x64, MSI) and Linux (x64, arm64), plus shell and PowerShell installers.
- The test suite includes soundness property tests for the planner and error-bound algebra, and every program in the docs is compiled in CI.