Deep Dive into SGLang, Volume I: Runtime, Scheduling, Memory, and Decoding (Foundation Books)

Deep Dive into SGLang, Volume I explains SGLang’s runtime request path as a set of algorithms, data structures, and systems tradeoffs.This volume focuses on the core inference-serving loop: tokenization, admission, prefill and decode scheduling, prefix lookup, KV-cache allocation, forward execution, sampling, streaming, and cleanup. Instead of treating SGLang as a catalog of commands, it builds a working model of the runtime state that makes modern LLM serving possible.Volume I covers:The inference serving problem and request lifecycleTransformer inference cost modelsSGLang’s runtime as a distributed state machineContinuous batching and chunked prefillKV-cache memory managementRadixAttention and prefix reuseHierarchical cachingAttention backends and forward batchesModel architecture registrationSampling and logits processingStructured outputs, reasoning parsers, and tool-call parsingSpeculative decodingDiffusion language models and blockwise decodingEach chapter explains the systems problem before implementation details, then ties technical claims back to SGLang source files, tests, or benchmark artifacts. The book is written for readers who already know basic transformer and GPU-systems vocabulary and want a deeper, source-grounded understanding of how a production LLM serving runtime is organized.This is Volume I of Deep Dive into SGLang. Volume II continues into quantization, distributed serving, kernels, hardware backends, benchmarking, correctness, multimodal serving, diffusion inference, and technical extension.Independent explanatory guide. Not affiliated with or endorsed by the SGLang project. Read more