Deployment profiles

A deployment profile names a combination of where the key lives, where the tokenizer is decrypted and which party is trusted. Profiles are trust boundaries, not scores: each one makes a specific claim against a specific adversary, and none of them is a weaker or stronger version of another. Choose a profile by asking who must not see the model and the user’s data, and what that party can access.

Summary

Profile	Implemented as	Claims	Does not claim
Official catalog (0.2 line)	Official sealed models of the catalog, each run with its model key, issued to every applicant approved for that model.	Without the model key, the published files compute nothing useful and cannot be matched to the public base. A key holder can reproduce the published weights from the public base with `kllm verify`.	Confidentiality between key holders: every holder of a model’s key holds the same key, and the kLLM team can derive it. Protection of the weights once unlocked to run: the base models are public.
Private Seal	`kllm keygen` and `kllm seal`, served with `--client` (no TEE). The key and the tokenizer stay on the user’s machine.	The sealed checkpoint on the user’s machine is useless without the key: the tokenizer is encrypted and the weights are locked. Cloud logs and relayed payloads hold token ids under the vocabulary permutation (LLMs) or ciphertext (Z-Image results).	Protection against a cloud that reads GPU memory or analyses the weights. The cloud’s copy is unlocked; for a public model, or a fine-tune of one, that analysis recovers the transforms and Π from it (Weights of a public model, or of a fine-tune of one).
Attested CPU	`kllm serve --key-source cdh --production` in a confidential VM (TDX, SEV-SNP, Hygon CSV). The key source and the attesting gateway are kLLM Enterprise.	The key and the decrypted tokenizer stay in confidential-VM memory. Users can check that TLS terminates in the attested VM.	GPU memory protection (the GPU is an ordinary passthrough device). Not yet validated on TEE hardware.
Confidential GPU (roadmap)	Not implemented.	—	—

Warning

kllm serve --key outside a confidential VM is not a profile. It decrypts the tokenizer into /dev/shm on the host (or into the default temporary directory when /dev/shm is absent) and unlocks locked weights into a cache directory on the host (~/.cache/kllm/unlocked by default). kLLM removes neither copy. This mode offers no protection from the host. It is a development mode. kllm unlock behaves the same way.

Official catalog

Note

Part of the upcoming 0.2 line. The application flow for model keys is on the main branch, and the catalog of official sealed models is rolling out on Hugging Face. See The catalog of official sealed models and Model keys.

The Official catalog profile shows the workflow on the official sealed models of the catalog without each user sealing a model first. Each catalog model is sealed under its own model key, and every applicant approved for that model receives the same key. A key that is shared provides no confidentiality: anyone holding it can decrypt the tokenizer, unlock the weights, read the token ids and run the model as its owner would. The shared key is not the only reason: the catalog models are sealed copies of public models. Their published files are locked and cannot be matched to the public base without the key, but wherever a model is unlocked to run, any holder of the public weights can recover the transforms and Π from it (Weights of a public model, or of a fine-tune of one). Treat every prompt and output in this profile as readable by any party that observes it and holds the model key, or holds the public weights together with the unlocked sealed weights. Use Private Seal or Attested CPU for any data or model that must stay confidential, and note that neither profile protects the weights or Π of a public model while it runs.

Private Seal

The user creates a key and seals the model on their own machine, uploads only the cloud part, and serves it without a key:

$ kllm keygen
$ kllm seal ~/models/Qwen3-8B -o Qwen3-8B-sealed --transform perm --client
$ kllm serve Qwen3-8B-sealed-cloud --client --host 127.0.0.1    # cloud
$ kllm client proxy Qwen3-8B-sealed --url http://127.0.0.1:8000  # user, over a tunnel

Adversary in scope: the cloud’s software as it records or relays traffic, including engine and access logs, gateways, packet captures and stored request and response bodies. With SGLang logging every request in full, a --key server’s log held 25 of 27 three-word phrases of a test prompt and reply; a --client server’s log held none.

Out of scope: a cloud that inspects deliberately. Token ids under a fixed vocabulary permutation are a substitution cipher open to frequency analysis; for a public model, or a fine-tune of one, a cloud that holds the public checkpoint recovers the transforms and Π from the sealed weights it serves, which are unlocked, and can then read the token ids in its logs as text, so for such models the log protection holds only against software and operators that do not run that analysis; Z-Image latents in GPU memory show the image; and a serving process that is attached to or modified sees everything it computes. kllm serve --client binds 0.0.0.0 by default and has no authentication; bind 127.0.0.1 or place it behind a separate gateway. Client-keyed mode documents the procedure and the measured results.

Attested CPU

The serving stack runs inside a confidential VM. The model key is released by the deployer’s key broker only after the VM has been attested, and the decrypted tokenizer is written to guest memory, which the confidential-computing hardware is designed to keep from the host:

$ kllm serve Qwen3-8B-sealed --key-source cdh --production

--production (or KLLM_PRODUCTION=1) refuses keys from passphrases or the environment, so the key can come only from the Confidential Data Hub. The cdh key source and the attesting TLS gateway are part of kLLM Enterprise. Users check that the TLS connection ends inside the attested VM with the open-source binding check python -m kllm.keyed.attest; verifying the quote’s signature chain and measurements remains the job of an existing verifier, and the binding check alone exits with status 0 even when the gateway reports "source": "none". See Confidential VM deployment and Verifying a deployment.

The GPU remains an ordinary passthrough device whose memory, and the PCIe traffic to it, the cloud can read. Against an operator who reads GPU memory, user text reduces to a substitution cipher over tokens: in measurements on Qwen3-8B, unigram frequency matching decoded 23.8% of tokens, and the nearest neighbor of a token’s layer-1 activation was the same token 99.4% of the time. The transforms and Π of a public model, or of a fine-tune of one, are recoverable from the sealed weights under both transforms by a cloud that holds the public checkpoint, and MiniMax H3’s decoded video and audio are in GPU memory in the clear.

Important

The Attested CPU profile has not yet been validated on TDX, SEV-SNP or Hygon CSV hardware. Outside a TEE the attesting gateway reports "source": "none", and KLLM_PRODUCTION=1 refuses to start.

Confidential GPU

Note

Planned — not available. kLLM does not implement or claim support for confidential-computing GPUs. See Roadmap.

Choosing a profile

For evaluating the workflow with public models and no confidential data, use a catalog model with its model key, or kllm serve --key on a machine under the evaluator’s control. This is a development mode and makes no confidentiality claim.
For keeping prompts and outputs out of a cloud’s logs and stored payloads without a confidential VM, use Private Seal.
For keeping the key and the tokenizer away from the host operating system and the hypervisor, use Attested CPU, and accept that GPU memory is not protected.

The adversary in each profile is described in detail in Threat model; the exact list of claims is in Claims.