Attacks

This page reports the attacks in tools/attacks/ and how to reproduce them. Unless stated otherwise, every number was measured on Qwen3-8B. The attacks need no key: they use only what the cloud holds (see Exposure) and public material.

The scripts import kllm and run from a checkout of the repository with the seal extra installed. tools/attacks/weights.py reads a sharded safetensors checkpoint (with model.safetensors.index.json) and runs on a GPU by default (--device cuda).

Weights of a public model, or of a fine-tune of one

tools/attacks/weights.py seals a simulated fine-tune (base + eps·std·noise per tensor) with a fresh random key and attacks it with only the public base and the sealed GPU weights. At eps = 0 the sealed model is an unmodified copy of the public model. The GPU holds the weights unlocked, so the attack applies to every model that runs, whether or not its files on disk are locked; what locking changes for the files is described in Locked weights.

Under `orth`

Second moments cancel the vocabulary permutation: for the LM head, W’ᵀW’ = Qᵀ(AᵀA)Q, so matching eigenvectors gives Q up to signs, which third moments fix. Nearest-neighbor matching and orthogonal Procrustes then refine Q and Π. With Q recovered, the other transforms reduce to nearest-neighbor matching.

Weight recovery under `orth`
eps	Q relative error	Π recovered	MLP permutation recovered	Error of recovered weights	Error of using the base
0.003	0.044	98.5%	100%	0.013	0.004
0.01	0.032	98.6%	100%	0.014	0.014
0.03	0.013	98.5%	99.99%	0.013	0.042
0.05	1.40	0.1%	0%	(failed)	0.070

Under `perm`

The attack is easier. Each column of the LM head keeps its values, only reordered over the vocabulary, so the sorted magnitudes of a column identify it, and third moments give its sign.

Weight recovery under `perm`
eps	Q relative error	Π recovered	MLP permutation recovered	Error of recovered weights	Error of using the base
0	0.0000	98.5%	100%	0.0000	0
0.01	0.0028	98.6%	100%	0.0028	0.0100
0.03	0.0083	98.5%	100%	0.0083	0.0300
0.05	0.0139	98.5%	100%	0.0138	0.0499
0.1	0.0277	98.5%	100%	0.0277	0.0995

Interpretation

The 1.5% of Π not recovered are near-duplicate rows (unused tokens). Real fine-tunes change weights by well under 3%. The static transforms therefore do not protect a fine-tune of a public model from a cloud that reads GPU memory, under either transform; under perm, not even a large fine-tune. At eps = 0.05 the spectral start for orth fails; a better start would likely succeed.

The perm row at eps = 0 is a sealed copy of the unmodified public model: Q is recovered exactly, the MLP permutation completely and Π to 98.5%, the remainder being the unused near-duplicate tokens. Under orth the smallest measured change, 0.3% of the weight standard deviation, gives the same Π recovery; an unmodified model was not run separately under orth. A sealed copy of a public model therefore keeps neither its transforms nor Π secret from any party that holds the public weights and the unlocked sealed weights, and such a party can map the sealed model’s token ids to text without the key. The key itself is not recovered, since it only seeds the transforms through HKDF, and trusted.kllm stays encrypted; what the blob protects, Π, is read from the weights instead. This is the position of every sealed copy of a public checkpoint once its weights are unlocked to run, including the catalog models (The catalog of official sealed models), whatever key seals them.

For weights with no public relative there is no reference to match against. Under orth, recovering Q then amounts to blind source separation; no attack is known for that case, and no proof of security exists either. Under perm, per-channel statistics survive, which leaves more structure to work with.

The same holds for MiniMax H3: the FL2VA weights are public, so a sealed copy does not protect the weights, and Π of its text encoder is expected to fall to the same attack (not measured on H3); what the seal still withholds from a reader of logs and relayed payloads is the user’s text. It also holds for FP8 checkpoints, which are sealed under perm only, and for mixture-of-experts models sealed under perm: the column-profile attack on the LM head applies unchanged. Experts are expected to be matched to those of the base by the same kind of statistics as MLP channels, while the block structure of FP8 permutations leaves fewer orders to search. Neither expectation has been measured on these models.

Note

Scope of the measurement: the script attacks the LM head (for Q and Π) and the gate_proj of one layer (--layer, default 10, for the MLP permutation). It uses the transform code of kllm seal, not the command itself. The statement that the other transforms reduce to nearest-neighbor matching once Q is known is not measured for attention heads, RoPE turns or the V/O transforms.

To reproduce the two tables:

$ python tools/attacks/weights.py --base ~/models/Qwen3-8B --eps 0.003 0.01 0.03 0.05
$ python tools/attacks/weights.py --base ~/models/Qwen3-8B --transform perm \
      --eps 0 0.01 0.03 0.05 0.1

Further options: --rounds (refinement rounds, default 6), --sample (rows used per round, default 32768) and --device (default cuda).

User tokens

tools/attacks/tokens.py used a 150k-token corpus written by Qwen3-8B on 24 topics: half served as the attacker’s reference, half as the victim’s traffic.

Token ids. The GPU sees a substitution cipher over the vocabulary. Plain unigram frequency matching decodes 23.8% of victim tokens; n-gram or language-model attacks perform better.
Activations. Q is an isometry, so the GPU can cluster activations. The nearest neighbor of a token’s layer-1 input is the same token 99.4% of the time (chance 0.6%), and the rate stays above 50% at every layer of the model. A signed permutation is an isometry as well, so the same holds under perm. Under either transform the GPU can build a token codebook and is back to the substitution cipher.
Logits. The argmax index of every step is Π⁻¹(next token), another substitution cipher, on output tokens.

Same-token nearest-neighbor rate by layer input (Qwen3-8B)
Layer input	1	2	4	8	12	16	20	24	28	32	36 (final)
Same-token 1-NN rate	99.4%	97.1%	95.1%	87.7%	75.9%	69.1%	63.2%	58.8%	55.2%	51.9%	53.5%

Note

Scope of the measurement: the substitution cipher is simulated by applying Π⁻¹ from a fresh random key to the victim’s token ids. The nearest-neighbor rates are measured on activations of the plain model; they carry over to the sealed model because both transforms are isometries of the residual stream.

To reproduce, write a corpus with a plain model server that exposes /v1/chat/completions, then run the attack. --all-layers adds the per-layer nearest-neighbor measurement and runs the full model on a GPU:

$ kllm serve ~/models/Qwen3-8B --port 30001
$ python tools/attacks/tokens.py --generate http://127.0.0.1:30001 --corpus corpus.txt
$ python tools/attacks/tokens.py --base ~/models/Qwen3-8B --corpus corpus.txt --all-layers

Corpus options: --texts (default 400) and --max-tokens (default 400). Measurement options: --seq (default 512) and --nn-tokens (default 8192).

MiniMax H3 outputs

The H3 VAE decoders run on the GPU, so the generated video and audio are in GPU memory in the clear. The transforms hide the prompt (permuted token ids, text-encoder activations in the secret basis) and the latents during denoising (a fixed permutation of channels, which statistics would undo).

A stolen checkpoint, or a wrong key

The checkpoint on disk is all that a thief of the deployer’s storage obtains. Without the key the tokenizer cannot be decrypted (trusted.kllm is AES-256-GCM encrypted under a key derived from the model key), and kllm serve refuses a key with the wrong fingerprint. Run with the public tokenizer instead, models sealed without locking produce nothing useful: Qwen3 emits unrelated tokens in random scripts, and H3, given the same prompt and seed, generates an unrelated video (frame PSNR 10.5 dB against plain, audio spectrogram SNR −11.4 dB). Locked weights compute nothing useful with any tokenizer.

Locked weights

By default the weights on disk are locked: encrypted in place under a key stream derived from the model key (Locked weights). Without the key, each locked tensor is, up to the security of AES-256, uniformly random codes in the range of finite values up to its largest power of two. A thief who also holds the public model from which the sealed weights derive therefore has nothing to match them against, and the attack in Weights of a public model, or of a fine-tune of one does not apply to the locked files. What stays in the clear is the layout (tensor names, shapes and dtypes, and the configuration) and each tensor’s largest binary exponent.

This holds for the files only. Wherever the model is unlocked to run, on a server with the key, in its cache of unlocked weights or in the cloud’s copy for client-keyed serving, the weights are in the clear, and a party that holds them and the public base is in the case of Weights of a public model, or of a fine-tune of one. A thief of a checkpoint sealed with --no-lock, or by a kLLM version without locking, is in that case as well: that thief recovers the transforms and Π from the sealed weights and needs neither the key nor the encrypted tokenizer.

Client-keyed Z-Image latents

In client-keyed mode the Z-Image latents in the server’s GPU memory show the image; a principal component analysis without the key recovers a clear thumbnail. The figure and the reasoning are in Not protected in client-keyed mode.