Threat model

This page defines the parties, the adversary and the trust assumptions of each way kLLM can be deployed. What each party holds is listed in Exposure; the measured attacks are in Attacks; the resulting claims are in Claims.

Deployment profiles are trust boundaries, not scores. A profile states which component holds the model key and the decrypted tokenizer, and against which adversary that placement is meant to hold. The profiles are introduced in Deployment profiles.

Parties

Model deployer: The holder of the model key. The deployer runs kllm keygen and kllm seal on a machine it trusts, and either serves its model to its own users or rents GPUs to serve it to itself.
User: The party that sends prompts and receives replies. In client-keyed mode the user and the deployer are the same party.
Cloud: The host operating system, the hypervisor and the operators of the machine that runs the GPU. In every profile below, the cloud is the adversary.
Storage thief: A party that obtains a copy of the sealed checkpoint, but not the model key.

Any of these parties may also hold the public model from which the sealed weights derive, either because the sealed model is a public model or because it is a fine-tune of one. Such a party can recover the transforms and the vocabulary permutation Π from the sealed weights without the key once it holds them unlocked (Weights of a public model, or of a fine-tune of one). Locked weights on disk cannot be matched to the public model without the key (Locked weights), but every party that runs the model holds them unlocked; the profiles below do not protect against such a party.

Profiles and adversaries

Profile	Implemented as	Adversary considered	Outside the model
Official catalog (0.2 line)	Official sealed models of the catalog, each run with its model key, which every applicant approved for that model receives.	A party that obtains the published files but not the model key.	Confidentiality between key holders: each model key is shared by every applicant approved for the model, and the kLLM team can derive it. The weights once unlocked to run: the catalog models are public models, whose transforms and Π any holder of the public weights can recover from the unlocked weights.
Private Seal	`kllm keygen` and `kllm seal --client`, served with `kllm serve --client` (no confidential VM). The key and the tokenizer stay on the user’s machine.	Cloud software that records or relays requests and responses.	A cloud that reads GPU memory or analyses the weights it serves.
Attested CPU	`kllm serve --key-source cdh --production` in a confidential VM (Intel TDX, AMD SEV-SNP, Hygon CSV). The key source and the attesting gateway are kLLM Enterprise.	A cloud that controls the host and the hypervisor, with confidential-VM memory protected by the hardware.	GPU memory, which is an ordinary passthrough device.
Confidential GPU (roadmap)	Not implemented.	Not applicable.	Not applicable.

Note

The Official catalog profile and its model keys are part of the upcoming 0.2 line and claim no confidentiality (Model keys). Support for confidential-computing GPUs is planned and is not part of the threat model of any current release.

Attested CPU: serving in a confidential VM

The deployer serves its model to its users on a cloud machine. kLLM runs the whole serving stack inside a confidential VM (Intel TDX, AMD SEV-SNP or Hygon CSV), whose CPU state and memory the confidential-computing hardware is designed to keep from the cloud. The GPU is an ordinary passthrough device: the cloud can read its memory and the PCIe traffic to it. GPU DMA passes through swiotlb bounce buffers in shared memory, and that traffic is visible to the cloud.

The model key is held by the deployer’s key broker (a Trustee KBS) and is released to the Confidential Data Hub inside the VM only after the VM has been attested against the deployer’s policy:

$ export KLLM_PRODUCTION=1
$ kllm serve /models/Qwen3-8B-sealed --key-source cdh --host 127.0.0.1 --port 8000

--production (or KLLM_PRODUCTION=1) refuses keys from passphrases and from the environment; with it, kllm serve accepts the key only from --key-source cdh. Before the engine starts, kllm serve unlocks locked weights into its cache directory (KLLM_UNLOCKED_DIR, by default ~/.cache/kllm/unlocked) inside the VM. The unlocked copy is ordinary files; place that directory on storage that the cloud cannot read, such as an encrypted volume or guest memory, or the cloud reads the unlocked weights from it. Both the deployer and the users attest the VM before trusting it. The full procedure is in Confidential VM deployment.

Users check that TLS terminates inside the attested VM in two steps. The open-source binding check python -m kllm.keyed.attest (see Verifying a deployment) confirms only that the evidence returned by the gateway names the key of the TLS connection. It does not verify the quote’s signature chain or measurements; that is the job of an existing verifier (the Trustee attestation service, Intel DCAP or AMD snpguest). Without --cafile it disables TLS certificate checking, and when the gateway reports "source": "none" (no TEE) it prints a warning but still exits with status 0. The binding check on its own therefore does not establish that the VM is genuine and does not stop an active intermediary.

Assumptions of this profile:

The confidential-VM hardware and firmware protect guest memory as specified.
The guest image is measured and reproducible, so that the deployer and the users can check its measurements against published reference values.
The serving process inside the VM is the one that was measured. A process that is attached to or modified sees everything it computes.

Important

The cdh and gramine key sources and the attesting TLS gateway are kLLM Enterprise components. With open-source kLLM alone this profile cannot be deployed end to end. The Attested CPU profile has not yet been validated on Intel TDX, AMD SEV-SNP or Hygon CSV hardware.

Private Seal: client-keyed serving without a confidential VM

The user seals the model on its own machine and uploads only the weights and configuration files. The cloud serves them without a key and without a confidential VM; the key, the permuted tokenizer and, for Z-Image, the VAE never leave the user. The uploaded weights are unlocked on the user’s side (kllm seal --client or kllm cloud --key), because the keyless server cannot unlock them. See Client-keyed mode for the procedure.

The security goal is narrower than in the Attested CPU profile: no plaintext in what the cloud’s software records or relays, that is, engine logs, access logs, gateways, packet captures and stored request and response bodies. A cloud that deliberately inspects GPU memory or the weights is out of scope, and this profile gives no protection against it. For a public model, or a fine-tune of one, inspecting the weights is enough: a cloud that holds the public checkpoint recovers Π from the sealed weights it serves (Weights of a public model, or of a fine-tune of one) and can then read the token ids in its logs as text. For such models the claim holds only against software and operators that do not run that analysis; the sealed checkpoint itself is useless without the key only to a party that has no public relative of the weights.

kllm serve --client refuses --key, --key-source cdh and production mode, and warns when user-side files (the encrypted tokenizer, tokenizer files or the VAE) are present in the directory it serves.

Warning

kllm serve binds 0.0.0.0 by default, also with --client, and the keyless servers have no authentication. The kllm client commands send no credentials, so an engine started with an API key cannot be used through them. The Z-Image latent server does not bound steps or size. Unless the deployment adds its own controls, the cloud endpoint is an unauthenticated GPU endpoint that any party able to reach it can use or exhaust. Bind it to 127.0.0.1 and reach it through an SSH tunnel, or place it behind a gateway or firewall that restricts who can connect:

$ kllm serve Qwen3-8B-sealed-cloud --client --host 127.0.0.1 --port 8000

Transport is the deployment’s responsibility. The --url scheme decides whether kllm client uses TLS. Z-Image replies are encrypted to a one-off X25519 key of the requester, but the server has no identity: an active intermediary on a plain HTTP path can substitute its own reply key and re-encrypt the result undetected. Use TLS or an SSH tunnel between the user and the cloud.

kllm client proxy holds the decrypted permuted tokenizer and decodes any token ids it is given. It binds 127.0.0.1:8001 by default and has no authentication; do not expose it on other interfaces.

Development mode: `kllm serve --key` outside a confidential VM

kllm serve MODEL --key [KEYFILE] unlocks the key file with its passphrase (from the terminal or KLLM_PASSPHRASE), decrypts the permuted tokenizer into a 0700 directory under /dev/shm and, for a locked model, unlocks the weights into a 0700 cache directory, ~/.cache/kllm/unlocked by default. Inside a confidential VM the /dev/shm directory is guest memory. On an ordinary host both are the host’s.

Important

kllm serve --key outside a confidential VM decrypts the tokenizer into host /dev/shm. It then offers no protection from the host: the host can read the tokenizer and therefore the text of every request and reply, and the unlocked weights in the cache directory. It is a development mode, not a Private Seal deployment. The tokenizer directory is not removed when the engine exits, so the decrypted tokenizer remains until the host is rebooted; the unlocked weights remain until they are removed. kllm unlock behaves the same way.

kllm run (upcoming 0.2 line) starts kllm serve --key for a sealed model in this mode. It reads the passphrase itself and passes it to the background server only through the KLLM_PASSPHRASE variable of the server’s environment, never on a command line or in a file. Processes running under the same user account, and the administrator of the host, can read a process’s environment while it runs, and so can read the passphrase.

Storage thief

A party that copies the sealed checkpoint from the deployer’s storage obtains the sealed weights, locked unless the model was sealed with --no-lock, and the encrypted tokenizer, trusted.kllm. The tokenizer is AES-256-GCM encrypted under a key derived from the model key, and kllm serve refuses a key whose fingerprint differs from the one recorded at sealing. Locked weights cannot be matched to the public base without the key. A copy of a cache of unlocked weights, or of a cloud’s copy for client-keyed serving, is not a locked checkpoint. What such a party can do is described in A stolen checkpoint, or a wrong key.

Outside the model in every profile

A serving process that is attached to or modified. It sees everything it computes.
GPU memory protection. No current profile protects GPU memory; see the roadmap in Roadmap.
Recovery of the transforms and Π of a public model, or of a fine-tune of one, from the unlocked sealed weights by a party that holds the public checkpoint, which the static transforms do not prevent (see Weights of a public model, or of a fine-tune of one). Locking protects the files on disk only; once the model is unlocked to run, a sealed copy of a public model keeps neither the model nor Π secret from such a party, whatever key seals it.