Abstract
AI is beginning to make substantive contributions to LLM-inference optimization. Existing AI optimization is predominantly profiling-based. Profiling-bound feedback confines the search to the capabilities and performance of an existing software stack.
AI-driven system architecting instead requires a general workload representation, a verifiable mutation space, and an implementation-independent evaluator. We present RoofLang, an embedded domain-specific language and intermediate representation that combines expert-authored compute and hardware graphs, semantics-preserving transformations, explicit placement and communication, and roofline-based discrete-event simulation.
By restricting agent actions to a normalized, verification-sound design space, RoofLang provides an analytical oracle for throughput–interactivity optimization before implementation. In our preliminary results, RoofLang reveals that DeepSeek V4-series models could achieve 3.5–39.5× higher peak decode throughput than other representative models. This gap is disproportionate to their total parameter counts and arises largely from compact KV-cache designs that support larger batches and reduce memory traffic. A persistent optimizer agent further discovered several new architectures that improved both throughput and interactivity of DeepSeek V4 Pro on NVIDIA B300 by 6.23–50.1%. These findings position RoofLang as an initial, verification-constrained substrate for AI-driven architecting rather than a high-fidelity production predictor.