ai infrastructure · llm inference

Serving large language models on our own stack.

Language models running on the MAX language and runtime — including a clean-room MAX reimplementation of a large hybrid architecture, gated module by module against a reference engine's own outputs and measured against it honestly.

Native on MAX

  • Qwen3-32B — a from-scratch GPU forward pass, parity-checked against a CPU reference in our tests (see the GPU LLM training stack).
  • Falcon-H1-7B (base model) — on MAX with a confidence trigger (the Sadalsuud detector) designed to fetch from Wikipedia or refuse when confidence is low.
  • Serving primitives— MAX's own paged key-value cache, FlashAttention-2, and GGUF weight loading, fronted by a native OpenAI-compatible HTTP server, with no Python in the serving path.

A clean-room MAX port of a large hybrid model

A separate effort reimplements a large hybrid-attention / mixture-of-experts 80-billion-parameter architecture entirely in MAX — derived module by module from the math, never copied from any reference source, and gated at every step against a third-party reference engine's own token-by-token outputs as the correctness oracle. In our tests the MAX port decoded at 59.7 tokens per second — 93% of the reference engine's measured 64 tokens per second on the same hardware, for a system built from scratch in the lab's own language.

Qwen3.5-35B-A3B from a real GGUF

A from-scratch MAX port of Qwen3.5-35B-A3B (a hybrid Gated-DeltaNet / gated-attention architecture with a 256-expert mixture-of-experts layer), designed to load the model's GGUF weights directly and generate tokens on the lab's own stack.

cross-references

→ MAX (the language)→ GPU LLM training stack→ MAX-OS on silicon