# Serving large language models on our own stack

Canonical page: https://keilab.io/ai/llm-inference/

ai infrastructure · llm inference

Language models running on the MAX language and runtime — including a clean-room MAX reimplementation of a large hybrid architecture, gated module by module against a reference engine's own outputs and measured against it honestly.

## Native on MAX

- Qwen3-32B — a from-scratch GPU forward pass, parity-checked against a CPU reference in our tests (see the GPU LLM training stack).
- Falcon-H1-7B (base model) — on MAX with a confidence trigger (the Sadalsuud detector) designed to fetch from Wikipedia or refuse when confidence is low.
- Serving primitives— MAX's own paged key-value cache, FlashAttention-2, and GGUF weight loading, fronted by a native OpenAI-compatible HTTP server, with no Python in the serving path.

## A clean-room MAX port of a large hybrid model

A separate effort reimplements a large hybrid-attention / mixture-of-experts 80-billion-parameter architecture entirely in MAX — derived module by module from the math, never copied from any reference source, and gated at every step against a third-party reference engine's own token-by-token outputs as the correctness oracle. In our tests the MAX port decoded at 59.7 tokens per second — 93% of the reference engine's measured 64 tokens per second on the same hardware, for a system built from scratch in the lab's own language.

## Qwen3.5-35B-A3B from a real GGUF

A from-scratch MAX port of Qwen3.5-35B-A3B (a hybrid Gated-DeltaNet / gated-attention architecture with a 256-expert mixture-of-experts layer), designed to load the model's GGUF weights directly and generate tokens on the lab's own stack.

cross-references

- [→ MAX (the language)](https://keilab.io/ai/max/)
- [→ GPU LLM training stack](https://keilab.io/ai/gpu-llm/)
- [→ MAX-OS on silicon](https://keilab.io/hardware/max-os-silicon/)

Status: In development

80B (MAX port): In our tests: 59.7 tok/s, 93% of the reference engine's 64 tok/s

Qwen3.5-35B: MAX port from a GGUF file

Serving primitives: Paged KV-cache · FlashAttention-2 · GGUF · OpenAI-compatible HTTP

Sensitivity: Engineering public

[⊸ Engineering inquiry](https://keilab.io/contact/)

---
KeiLab is operated by KeiSei LLC, a Wyoming limited liability company. +1 307 210 8538
Site index for AI agents: https://keilab.io/llms.txt