Glossary

This page defines the terms used in this documentation. Terms are listed alphabetically. The concepts behind them are introduced in Concepts.

attestation binding check: The check performed by python -m kllm.keyed.attest: the evidence returned by an attesting gateway names the TLS key of the connection. It does not verify the quote’s signature chain or measurements. See Verifying a deployment.
Attested CPU: The deployment profile in which kllm serve --key-source cdh --production runs inside a confidential VM and the model key is released by a key broker after attestation. The key and the decrypted tokenizer stay in confidential-VM memory, and users can check that TLS terminates in the attested VM. It does not protect GPU memory, and it has not yet been validated on TEE hardware. The key source and the gateway it needs are part of kLLM Enterprise. See Deployment profiles.
catalog: The list of public models that the kLLM team seals ahead of time and publishes under https://huggingface.co/ANetResearch, each named after its base with the suffix -kllm and sealed under its own catalog model key. Part of the upcoming 0.2 line, rolling out. See The catalog of official sealed models.
catalog manifest: The kllm-manifest.json file of a published catalog model: base and revision, key fingerprint and version, kLLM version and the SHA-256 digest of every file. It contains no key material and is not a signature.
catalog model key: The key of one catalog model, derived from an offline master key and issued, on application, to every applicant approved for that model. Every holder of a model’s key holds the same key; it is not a confidentiality boundary. See Model keys.
CDH
Confidential Data Hub: The component of Confidential Containers guest-components that, inside a confidential VM, has the VM attested to the key broker and releases a secret resource to it. --key-source cdh obtains the model key through it.
client-keyed mode: Serving in which the cloud holds neither the model key nor the tokenizer. The cloud runs kllm serve --client on the cloud part of a sealed model and exchanges token ids (LLMs) or latents (Z-Image) with the user, who runs kllm client. It protects what the cloud logs and relays, not GPU memory. See Client-keyed mode.
cloud part: The directory that kllm seal --client or kllm cloud writes for the cloud (by default OUTPUT-cloud). It contains the files the cloud needs, with the weights unlocked, and no key material, tokenizer or trusted.kllm blob.
Confidential GPU: A roadmap deployment profile for GPUs whose memory is protected by confidential computing. It is not implemented.
confidential VM: A virtual machine whose CPU state and memory the host cannot read, such as Intel TDX, AMD SEV-SNP or Hygon CSV. In kLLM deployments its GPU is an ordinary passthrough device, whose memory the host can read.
deployment profile: A named trust boundary that states what a deployment claims and what it does not claim: Official catalog, Private Seal, Attested CPU and Confidential GPU. Profiles are not scores. See Deployment profiles.
dry run: kllm serve --dry-run: prints the engine command line that kLLM would execute, with the environment it would set, and exits.
engine: The inference server that runs the model: SGLang or vLLM. kLLM starts the stock engine with its own model classes and kernels, and does not patch or fork it.
engine arguments: Arguments after -- on the kllm serve command line. They are passed to the engine unchanged, after kLLM’s arguments and the recipe’s arguments.
fingerprint: A short identifier of a model key, derived from it with HKDF and printed as four groups of four hexadecimal digits. Each sealed model records the fingerprint of the key that sealed it, and kLLM refuses to serve it under a key with a different fingerprint. The fingerprint does not reveal the key.
FP8 block: The square group of weights that shares one scale in a block-wise FP8 checkpoint. Under perm, every permutation moves whole blocks and shuffles inside them, so the FP8 values are only moved and sign-flipped.
GPU family: The class of the first visible GPU (for example H200 or B300), derived from its nvidia-smi name. Recipes are matched on it.
INT8 DiT kernels: Optional kernels for the diffusion-transformer blocks of MiniMax H3, enabled with kllm serve --int8. They are a kernel optimization separate from sealing, with measured numerical drift. See Running the H3 diffusion transformer on INT8 kernels.
key broker
KBS: A service run by the deployer, such as a Trustee key broker, that holds the model key as a resource and releases it only to confidential VMs whose measurements match the deployer’s policy.
key file: The file that stores a model key, encrypted under a passphrase with scrypt and AES-256-GCM. The default path is ~/.kllm/keys/default.key. See Model keys.
key source: Where a serving process obtains the model key: a passphrase-protected key file, the environment, or a plug-in registered under the kllm.keysource entry-point group, such as cdh. See Registering a key source.
keyed serving: Serving with kllm serve --key: the serving process obtains the model key from a key source, decrypts the permuted tokenizer into memory and starts the engine with it. The machine that runs it is the trusted domain; outside a confidential VM this is a development mode. See Serving.
kLLM Enterprise: The separately licensed package (kllm-enterprise) that provides the key sources for confidential VMs (cdh, gramine) and the attesting TLS gateway. The rest of kLLM is licensed under Apache-2.0.
launch spec: The LaunchSpec object in kllm.backends.launch that describes one engine start: model, tensor-parallel size, host, port, recipe, engine arguments, environment, engine and tokenizer path. It renders the engine command line.
locked weights: Sealed weights encrypted in place by kllm seal (the default; --no-lock skips it): every value is shifted within the finite values of its format by an AES-256-CTR key stream derived from the model key and the tensor’s name. The files load in any engine but compute nothing useful, and cannot be matched to the public base, without the key. kllm serve --key unlocks them into a cache directory. See Locked weights.
model key: The 256-bit secret under which a model is sealed. Every transform of a sealed model, the encryption of its tokenizer and the locking of its weights are derived from it. See Model keys. The keys of the official sealed models, issued on application, are catalog model keys.
Official catalog: The deployment profile in which the official sealed models of the catalog are run with their catalog model keys (Model keys). Part of the upcoming 0.2 line, rolling out with the catalog. It demonstrates the workflow and claims no confidentiality: each model key is shared by every applicant approved for the model, and a sealed copy of a public model, once unlocked to run, reveals its transforms and vocabulary permutation to any holder of the public weights.
orth: The transform that rewrites the residual stream with a dense key-derived orthogonal basis and V/O with one orthogonal basis per KV head. It also hides coordinates and outliers; the sealed model equals the original up to the rounding of the rewritten weights. See Sealing.
perm: The transform that uses key-derived signed permutations in place of orthogonal bases. Every sealed weight is a bit-exact rearrangement of an original one. It is required for FP8 checkpoints. See Sealing.
permuted tokenizer: The tokenizer of a sealed LLM, rewritten so that it reads and emits token ids under the vocabulary permutation. It is stored only inside trusted.kllm and is decrypted into memory by kllm serve --key, kllm unlock or kllm client.
plain serving: Serving an unsealed model with kllm serve, without a key. kLLM then only selects a recipe and starts the engine.
Private Seal: The deployment profile in which the user runs kllm keygen and kllm seal and serves with --client, without a TEE; the key and the tokenizer stay on the user’s machine. The sealed checkpoint on the user’s machine is useless without the key, its tokenizer encrypted and its weights locked, and cloud logs and relayed payloads hold token ids under the vocabulary permutation (LLMs) or ciphertext (Z-Image results). It does not protect against a cloud that reads GPU memory or analyses the weights; the cloud’s copy is unlocked, and for a public model, or a fine-tune of one, that analysis recovers the transforms and the vocabulary permutation.
production mode: Enabled by kllm serve --production or KLLM_PRODUCTION. Keys from passphrases or the environment are refused; only --key-source cdh is accepted. See Settings.
provider: The kllm.keyed.provider.Provider class, which implements key handling, sealing and the preparation of keyed and client-keyed launches behind the TrustProvider interface of kllm.trust. See Architecture overview.
recipe: A set of SGLang arguments measured for one model on one GPU family, applied by kllm serve when the model and the GPUs match. See Contributing a serving recipe.
regular token ids: The ids of the tokenizer’s base vocabulary. The vocabulary permutation applies to them only; added and special tokens keep their ids.
residual stream: The hidden state that flows between the layers of a transformer. Sealing maps it into a secret basis (orth) or a signed permutation of its channels (perm).
scheme: The sealing format recorded in a sealed model’s metadata: l1 for decoder LLMs and MiniMax H3, client for client-keyed Z-Image, and d1-v1 for the keyed Z-Image pipeline used by kllm demo.
sealed checkpoint
sealed model: The output of kllm seal: a checkpoint with the original architecture, tensor names, shapes and dtypes, whose weights are transformed under the key and, by default, locked (locked weights), together with trusted.kllm and a kllm section in config.json (LLMs) or a kllm.json file (pipelines). See Sealed checkpoint format.
sealing: Rewriting a checkpoint under a model key so that the engine computes the same function in a secret basis, and encrypting the tokenizer that maps that basis to text.
signed permutation: A permutation of coordinates combined with a sign flip of each coordinate. It is the building block of perm.
substitution cipher over tokens: What user text reduces to for an operator who reads GPU memory: each token maps to a fixed, secret token id. Frequency analysis recovers part of the text, and for a public model, or a fine-tune of one, the mapping itself is recoverable from the unlocked sealed weights with the public checkpoint. See Attacks.
tensor parallelism: Splitting each layer of a model across several GPUs. kllm serve --tp sets the number of GPUs.
transform: The key-derived rewrite applied to the weights at sealing time: orth or perm. Transforms are static: the same key gives the same transform for every request.
trusted domain: The place where the key and the decrypted tokenizer live: the memory of a confidential VM, or the user’s machine in client-keyed mode. The GPU is outside it.
trusted.kllm: The encrypted blob in a sealed model directory. It holds the permuted tokenizer and, for some schemes, trusted tensors, encrypted with AES-256-GCM in chunks under a key derived from the model key.
vocabulary permutation: The key-derived permutation of the regular token ids, applied to the embedding and LM-head rows. The permuted tokenizer speaks the permuted ids.