Serving large language models on our own stack.

• Qwen3-32B — a from-scratch GPU forward pass, parity-validated against a CPU reference (see the GPU LLM training stack).
• Falcon-H1-7B— a chat runtime with on-the-fly retrieval when the model's own confidence is low, built on the lab's Sadalsuud-style routing primitives.
• An 80-billion-parameter model at 66 tokens per second — served through KeiChat: a MAX-native HTTP server and tokenizer, bridged over a local FIFO to a resident, graph-captured CUDA forward-pass engine. Weights stay resident in GPU memory between turns.
• Serving primitives— MAX's own paged key-value cache, FlashAttention-2, and GGUF weight loading, fronted by a native OpenAI-compatible HTTP server, with no Python in the serving path.

A separate effort reimplements a large hybrid-attention / mixture-of-experts 80-billion-parameter architecture entirely in MAX — derived kubik-by-kubik from the math, never copied from any reference source, and gated at every step against a third-party reference engine's own token-by-token outputs as the correctness oracle. With its own CUDA-graph capture wired in, the MAX port was last measured at 59.7 tokens per second — 93% of the reference engine's measured 64 tokens per second on the same hardware, for a system built from scratch in the lab's own language.

A from-scratch MAX port of Qwen3.5-35B-A3B (a hybrid Gated-DeltaNet / gated-attention architecture with a 256-expert mixture-of-experts layer) loads its full 20 GB Q4_K_M GGUF and its hybrid-architecture scaffolding today. The forward-pass runtime that turns the loaded weights into generated tokens is still being assembled — reported here as in-progress, not yet serving.