Client-keyed mode

Client-keyed mode serves a sealed model on rented GPUs without a confidential VM and without the key ever reaching the cloud. The user seals the model on their own machine, uploads only the weights and configuration files, and keeps the key, the permuted tokenizer and, for Z-Image, the VAE. The cloud runs the sealed weights on token ids and returns token ids (LLMs) or encrypted latents (Z-Image). This mode is the basis of the Private Seal profile described in Deployment profiles. This page describes sealing for the cloud, serving, the local proxy for LLMs, the Z-Image variant, the measured results and the scope of protection.

Important

The security goal of this mode is narrow: no plaintext in what the cloud’s software records or relays (engine logs, access logs, gateways, packet captures, stored request and response bodies). It gives no protection against a cloud that deliberately reads GPU memory, analyses the weights it serves, or attaches to the serving process. Read What this mode protects before relying on it.

Overview

A client-keyed deployment has two sides:

Side	Holds	Runs
User’s machine	Key file, sealed directory with `trusted.kllm`, tokenizer files, chat template, Z-Image VAE	`kllm seal --client` or `kllm cloud`, `kllm client proxy` (LLMs), `kllm client generate` (Z-Image)
Cloud	The `-cloud` directory: sealed weights, unlocked, and configuration files only	`kllm serve --client`: SGLang or vLLM with `--skip-tokenizer-init` (LLMs), or kLLM’s keyless latent server (Z-Image)

Client-keyed mode supports the sealed LLM architectures listed in Supported models and Z-Image pipelines. MiniMax H3 pipelines cannot be served with --client.

On the user’s machine, install the client extra for LLMs or the image extra for Z-Image. In the cloud, install seal together with serve (SGLang) or vllm for LLMs, or image for Z-Image. See Installation guide.

Sealing for the cloud

kllm seal --client seals the checkpoint as usual and then writes the cloud’s part next to it:

$ kllm seal ~/models/Qwen3-8B -o Qwen3-8B-sealed --transform perm --client

The command creates two directories:

Qwen3-8B-sealed: The user’s copy: the complete sealed model, including trusted.kllm, with its weights locked (Locked weights). Keep it on the user’s machine.
Qwen3-8B-sealed-cloud: The cloud’s part, with its weights unlocked: the keyless server cannot unlock them. --cloud DIR selects another location; the default is OUTPUT-cloud. Both directories must not exist beforehand.

The export leaves out every file that stays with the user:

trusted.kllm, the encrypted permuted tokenizer;
every tokenizer file, including the public ones: tokenizer.json, vocab.json, merges.txt, tokenizer_config.json, special_tokens_map.json, added_tokens.json, tokenizer.model, spiece.model;
every file whose name starts with chat_template;
the tokenizer/, processor/ and vae/ subdirectories of a pipeline;
any .cache directory.

Weight shards are written unlocked with the key the model was sealed under; the key itself is not copied. All other files are hard-linked into the cloud directory, or copied when the two directories are on different file systems. With --no-lock, the weight shards are hard-linked as well.

Note

Because the export uses hard links, editing a hard-linked file of the cloud directory in place also edits the user’s sealed copy. The cloud directory keeps the kllm metadata, which includes the key fingerprint. The fingerprint reveals nothing about the key, but it lets the cloud recognize models sealed under the same key.

Upload only the -cloud directory. kllm serve --client prints a warning when the directory it serves still contains files that belong on the user’s side, and refuses a directory whose weights are locked.

For a model that is already sealed, such as a catalog model (The catalog of official sealed models), kllm cloud writes the cloud’s part on its own:

$ kllm cloud Qwen3-8B-kllm -o Qwen3-8B-kllm-cloud --key Qwen3-8B-kllm.key

The output is the same as the -cloud directory of kllm seal --client. --key is needed only when the weights are locked; see kllm cloud.

Serving in the cloud

$ kllm serve Qwen3-8B-sealed-cloud --client
$ kllm serve Qwen3-8B-sealed-cloud --client --engine vllm

For a sealed LLM, kllm serve --client starts the engine with --skip-tokenizer-init prepended to the engine arguments: the engine accepts and returns token ids and never loads a tokenizer. For a client-keyed Z-Image pipeline, it starts python -m kllm.keyed.client.server instead of an engine; see Z-Image.

--client refuses --key, --key-source cdh and production mode (--production or KLLM_PRODUCTION=1): the key stays with the user. Arguments after -- are passed to the engine or the latent server unchanged.

Warning

kllm serve binds 0.0.0.0 by default, and neither the engine endpoint nor the Z-Image latent server performs any authentication. kllm client sends no credentials and does not enforce TLS; the URL scheme decides. Unless a firewall, gateway or tunnel is in front of it, the server is an open GPU endpoint over plain HTTP that any third party can use or exhaust. The latent server does not bound steps or size.

Bind the server to the loopback interface and reach it through an SSH tunnel, or place it behind a separate authenticating TLS gateway:

$ kllm serve Qwen3-8B-sealed-cloud --client --host 127.0.0.1 --port 8000
$ ssh -N -L 8000:127.0.0.1:8000 user@CLOUD        # on the user's machine

An engine started with its own --api-key option cannot be used through kllm client, which has no way to send the key.

The local proxy (LLMs)

kllm client proxy runs on the user’s machine. It unlocks the key, decrypts the permuted tokenizer into a private directory under /dev/shm, loads it into process memory and removes the directory, applies the chat template locally and talks token ids to the cloud.

$ kllm client proxy Qwen3-8B-sealed --url http://127.0.0.1:8000
$ curl localhost:8001/v1/chat/completions \
    -d '{"messages": [{"role": "user", "content": "hi"}]}'

`kllm client proxy` options
Option	Default	Meaning
`MODEL`	(required)	The user’s sealed directory, not the `-cloud` copy.
`--url URL`	(required)	The cloud server, for example `http://host:8000`.
`--key KEYFILE`	`~/.kllm/keys/default.key`	Key file; the passphrase is read from the terminal or `KLLM_PASSPHRASE`.
`--engine`	`sglang`	`sglang` or `vllm`; must match the engine in the cloud.
`--host`	`127.0.0.1`	Local bind address.
`--port`	`8001`	Local port.
`--name`	directory name	Model name reported by `/v1/models`.

The proxy serves the following endpoints:

Endpoint	Behavior
`GET /health`	Status and model name.
`GET /v1/models`	One model, named by `--name`.
`POST /v1/chat/completions`	OpenAI-compatible chat, streaming or not, with stop strings and `chat_template_kwargs`. `n` must be 1.
`POST /v1/completions`	`prompt` is one string, a one-element list of strings, or a list of integer token ids, which are forwarded as already protected ids. `logprobs`, `prompt_logprobs` and `echo` require `--engine vllm`.
`POST /generate`	SGLang’s native API (`--engine sglang` only), non-streaming; `text` is encoded locally and the decoded `text` is added to the response. `sampling_params` and every other request field are forwarded to the cloud unchanged; do not place text, such as stop strings, in them.
`POST /flush_cache`	Forwarded to the engine.

Behavior to be aware of:

Log-probabilities returned through /v1/completions refer to protected token ids and are passed back unchanged; echo has no textual effect.
Non-streaming requests are processed one at a time; use streaming for concurrent clients.
A non-streaming completion whose last token ends inside a multi-byte character can return empty text. Streaming requests are not affected.

Warning

The proxy holds the decrypted tokenizer and decodes any token ids it receives, so it acts as a decoding oracle for the vocabulary permutation. It has no authentication. Keep the default --host 127.0.0.1.

Z-Image

A Z-Image pipeline sealed with --client uses its own client-keyed scheme: the text encoder is sealed under the LLM transforms, and the DiT reads and writes each 2x2 latent patch in a key-derived orthogonal basis. The cloud runs the sampler in that basis; the user maps the result back and decodes it with the public VAE.

$ kllm seal ~/models/Z-Image -o Z-Image-sealed --client                 # user
$ kllm serve Z-Image-sealed-cloud --client --host 127.0.0.1 --port 30010   # cloud
$ kllm client generate Z-Image-sealed "A red fox in fresh snow at dawn" \
    --url http://127.0.0.1:30010 -o fox.png                              # user

kllm client generate encodes the prompt locally, truncated to 512 token ids, and sends the ids together with a fresh X25519 public key generated for that request. The server samples, encrypts the final latents to that key with its own one-off X25519 key (ECDH, HKDF-SHA256, AES-256-GCM) and discards its half. The client decrypts the latents, maps them back to canonical coordinates and decodes them with the VAE on --device. The server never loads a tokenizer or a VAE, writes nothing to disk and runs without request logging.

`kllm client generate` options
Option	Default	Meaning
`MODEL PROMPT`	(required)	The user’s sealed directory and the prompt.
`--url URL`	(required)	The cloud’s latent server.
`--key KEYFILE`	`~/.kllm/keys/default.key`	Key file.
`-o`, `--output`	`kllm.png`	Output image.
`--negative-prompt`	empty prompt	Encoded locally; sent when the guidance scale is above 0.
`--size`	`1024x1024`	Width by height.
`--steps`	`28`	Denoising steps.
`--guidance-scale`	`4.0`	Classifier-free guidance scale.
`--seed`	`0`	Seed of the cloud’s noise generator.
`--device`	`cuda`, `mps` or `cpu`	Device for the VAE, chosen in that order when not given.

The reply encryption has no server identity. On a plain-HTTP path an active intermediary can substitute its own public key and read the latents, so use TLS or an SSH tunnel. The response carries the latent tensor shape and the generation time in the clear. See Generating images in client-keyed mode for a complete procedure.

Measured results

Qwen3-1.7B sealed with perm on one NVIDIA A40, keyed serving (--key) against client-keyed serving through the proxy, one after the other on the same GPU, 1024 input and 128 output tokens, single runs (benchmarks/client.sh):

	SGLang `--key`	SGLang `--client`	vLLM `--key`	vLLM `--client`
Median TTFT, concurrency 1 (ms)	66.7	71.1	75.4	73.9
Median TPOT, concurrency 1 (ms)	6.81	6.87	7.23	7.26
Output throughput, concurrency 16 (tok/s)	1039	1042	1036	963
Median TPOT, concurrency 16 (ms)	12.38	12.63	12.61	12.96
Perplexity	5.4975	5.4975	5.4943	5.4943 to 5.5110

The token ids reaching the engine were the same as under --key, and so were the log-probabilities, to the bit, on every SGLang run. Under vLLM, two of four launches of the keyless engine matched --key to the bit and two differed by 0.05 in mean |Δ log-prob| (perplexity 5.5017 and 5.5110). The proxy added about 0.1 to 0.3 ms to TPOT. At concurrency 16 no throughput loss was measured under SGLang (1039 against 1042 tok/s); under vLLM throughput was 7% lower. These benchmark clients send streaming requests.

With SGLang logging every request in full (--log-requests --log-requests-level 3), the --key server’s log held 25 of the 27 three-word phrases of a test prompt and its reply; the --client server’s log held none, only token ids.

For Z-Image sealed with orth (1024x1024, 28 steps, one NVIDIA A40), client-keyed generation took 48.1 s end to end against 48.4 s for the plain diffusers pipeline, with a latent relative error of 11.5% against a noise floor of 7.2%; every image pair showed the same scene, subject and composition. The full table, the decode times on the user’s side and the conditions are in Measured results.

What this mode protects

Protected: what the cloud’s software records or passes along. Engine and access logs, gateways, packet captures and stored payloads hold token ids under the key’s vocabulary permutation, or ciphertext for Z-Image results. Request metadata is not protected: sampling and generation parameters, token counts, timing and the Z-Image latent shape travel in the clear.

Not protected, and not claimed:

Token ids are a substitution cipher. The permutation is fixed per key, so collected ids can be attacked by frequency analysis (23.8% of tokens decoded from unigram frequencies alone, measured on Qwen3-8B), and a cloud holding the public base model can recover the permutation from the sealed weights it serves, which are unlocked; locking protects the user’s copy, not the cloud’s. That recovery applies to a sealed copy of an unmodified public model as much as to a fine-tune of one (98.5% of Π on Qwen3-8B, the remainder being unused tokens; see Weights of a public model, or of a fine-tune of one). For such models a cloud that holds the public checkpoint can read the token ids in its logs as text once it runs that analysis; the log protection then holds only against software and operators that do not.
Z-Image latents in GPU memory show the image. A principal component analysis of the cloud’s patch vectors, computed without the key, recovers a clear thumbnail. Only five DiT tensors are re-based; the DiT’s activations are not hidden.
The serving process sees everything it computes. Attaching to it or modifying its code reveals what the GPU sees.

The user's image, a plain latent channel, the cloud's latent channel, one coordinate of the cloud's patch vectors, and a keyless PCA thumbnail. — From left: the user’s image; a channel of the plain latents; a channel of what the cloud holds; one coordinate of the cloud’s patch vectors; the top three principal components of the cloud’s patch vectors as RGB, computed without the key.

For the full threat model see Threat model and Claims.