Concepts

This page maps the terms that the rest of the documentation relies on: the model key, the sealed checkpoint, the transforms, the vocabulary permutation, the trusted domain and the engine. Each term is defined once here and described in full on its own page.

The workflow at a glance

kLLM separates the life of a model into two phases.

Seal once. On a machine the model owner controls, kllm keygen creates a model key and kllm seal rewrites a local Hugging Face checkpoint under that key.
Serve anywhere. kllm serve starts stock SGLang or vLLM on the sealed checkpoint. The engine runs the transformed weights with its own model classes and kernels. Only a holder of the key can decrypt the tokenizer that turns the model’s token ids back into text.

$ kllm keygen
$ kllm seal ~/models/Qwen3-8B -o Qwen3-8B-sealed --transform perm
$ kllm serve Qwen3-8B-sealed --key

kLLM is a layer on top of the inference engine. It does not replace or fork SGLang or vLLM; plain models are served through the same kllm serve command.

Terms

Model key

A model key is 32 random bytes (256 bits). Every secret of a sealed model is derived from it with HKDF-SHA256 under a distinct label: the weight transforms, the vocabulary permutation and the key that encrypts the tokenizer. One key can seal any number of models. See Model keys.

Key file and passphrase

The key is stored in a key file, by default ~/.kllm/keys/default.key, encrypted under a passphrase with scrypt and AES-256-GCM. The passphrase protects the file at rest only, in the way an SSH passphrase protects a private key. Someone who holds sealed weights but not the key file faces a 256-bit search regardless of the strength of the passphrase.

Fingerprint

The fingerprint is a public, 16-hex-digit identifier of a key, written in four groups of four digits (for example 3f9a-0c41-d27e-88b5). It is derived from the key with HKDF and reveals nothing about it. kllm seal records the fingerprint in the sealed checkpoint, and kllm serve refuses a key whose fingerprint differs.

Sealed checkpoint

A sealed checkpoint is the output of kllm seal. It keeps the architecture name, the directory layout, the tensor names, the shapes and the dtypes of the original, so the engine loads it as an ordinary checkpoint. It differs in four respects:

the weights are rewritten under the key’s transforms;
config.json carries a kllm section with the scheme, the transform and the key fingerprint;
the tokenizer’s vocabulary files are replaced by an encrypted file, trusted.kllm;
a tied LM head is written as a separate tensor (tie_word_embeddings is false).

Without the key the tokenizer cannot be decrypted, and the sealed weights driven by the public tokenizer produce no useful output. A party that holds the public model from which the weights derive, whether an unmodified public model or the base of a fine-tune, can nevertheless recover the transforms and Π from the sealed weights without the key (Weights of a public model, or of a fine-tune of one). See Sealing and Sealed checkpoint format.

Transform: `orth` and `perm`

A transform is the static, key-derived change of basis that sealing applies to the weights, so that the network computes the same function in a secret basis. Both transforms are exact in exact arithmetic. They differ in how they treat the residual stream:

orth (default): A dense random orthogonal basis. It mixes coordinates, so outlier channels and per-channel statistics are not visible. The rewritten weights are rounded again to the checkpoint dtype.
perm: A random signed permutation. Every sealed weight is a bit-exact rearrangement of an original value; channel magnitudes and outliers survive, in a shuffled order. FP8 checkpoints are sealed under perm only.

See Transforms and Choosing a transform.

Vocabulary permutation

The vocabulary permutation, written Π, is a key-derived permutation of the regular token ids. Special (added) tokens keep their ids. Sealing applies Π to the rows of the embedding and of the LM head, and writes a tokenizer whose vocabulary speaks Π-permuted ids. The engine therefore receives and emits permuted token ids, and only the permuted tokenizer maps them to text.

Permuted tokenizer and `trusted.kllm`

The permuted tokenizer is the only part of a sealed model that maps between the secret basis and text. Sealing encrypts it into trusted.kllm, a chunked AES-256-GCM container under a key derived from the model key. At serving time it is decrypted into memory in the trusted domain and passed to the engine as its tokenizer; it never reaches the GPU.

Trusted domain

The trusted domain is where the model key and the decrypted tokenizer may exist. It is either the CPU and memory of a confidential VM (Intel TDX, AMD SEV-SNP or Hygon CSV), or, in client-keyed mode, the user’s own machine. The GPU is outside the trusted domain in every mode: it is an ordinary device whose memory an operator of the host can read.

Warning

kllm serve --key on an ordinary host decrypts the tokenizer into the host’s /dev/shm. The host is then the trusted domain, and kLLM offers no protection from whoever controls it. This is a development mode.

Engine

The engine is the inference server that runs the model: SGLang (--engine sglang, the default) or vLLM (--engine vllm). kLLM starts the engine’s own entry point with its own model classes, kernels, CUDA graphs, batching and tensor parallelism. The engine needs no kLLM code to run a sealed language model; it receives a sealed checkpoint and a tokenizer directory. See Serving.

Launcher

A launcher is whatever starts the engine process. kllm serve is one launcher; NVIDIA Dynamo, llm-d and Kubernetes manifests are others. kllm unlock decrypts a sealed model’s tokenizer and prints its path, so that any launcher can pass it to the engine. See Other launchers: kllm unlock.

Recipe

A recipe is a set of measured SGLang arguments for one model on one GPU family. kLLM 0.1.0.dev0 ships three recipes, all for GLM-5.3 on H200 or B300 GPUs. See Recipes.

Key source

A key source is where kllm serve obtains the model key: a passphrase-protected key file (--key-source passphrase, the default) or, inside a confidential VM, the Confidential Data Hub, which releases the key after attestation (--key-source cdh). The cdh key source is part of kLLM Enterprise. See Registering a key source and Confidential VM deployment.

Client-keyed mode

In client-keyed mode the cloud serves a sealed model without its key and without a confidential VM. kllm seal --client writes a separate cloud directory without key material or tokenizer files; the user keeps the key and the tokenizer and talks to the cloud in token ids through kllm client proxy. See Client-keyed mode.

Deployment profile

A deployment profile names a trust boundary and states what it claims and what it does not: Private Seal (client-keyed mode), Attested CPU (a confidential VM with the Enterprise key source) and two profiles that are not available in 0.1.0.dev0. Profiles are trust boundaries, not scores. See Deployment profiles.

What sealing does not hide

Important

Sealing does not encrypt GPU memory. The transforms are static, so an operator who reads GPU memory sees user text as a substitution cipher over tokens rather than as ciphertext, and a party that holds the public checkpoint can recover the transforms and Π of a public model, or of a fine-tune of one, from the sealed weights. Read Claims before relying on a deployment.