Release notes

This page lists the changes in each version of kLLM, newest first. It is maintained alongside CHANGELOG.md in the repository.

Note

No version of kLLM has been published on PyPI yet. The source tree carries the development version 0.1.0.dev0. pip install kllm becomes available with the first tagged release; until then, install from a source checkout as described in Installation guide.

Upcoming 0.2 line (main branch)

The following changes are on the main branch and are not part of a tagged release. The source tree still carries the version 0.1.0.dev0. Options and output may change before 0.2.0.

New features

  • Locked weights. kllm seal now also encrypts every weight in place, by AES-256-CTR within the finite values of its format; --no-lock skips it. The files keep their layout and load in any engine, but compute nothing useful and cannot be matched to the public base without the key. kllm serve --key unlocks them into a 0700 cache (~/.cache/kllm/unlocked, KLLM_UNLOCKED_DIR). Locking protects the files, not a running model: whoever runs the model holds the unlocked weights, and the matching attack on public bases applies to them again. See Locked weights.

  • The catalog of official sealed models. Public models sealed ahead of time, each under its own key derived from an offline master key, named after their base (for example Qwen3-8B-kllm) and published with a kllm-manifest.json. The catalog is rolling out in the ANetResearch organization on Hugging Face. MiniMax H3 is held for license reasons. See The catalog of official sealed models.

  • Model keys. The key of each catalog model is issued on application, as a passphrase-protected key file; one application can cover several catalog models. Every applicant approved for a model receives the same key, so model keys provide no confidentiality. See Model keys.

  • kllm verify MODEL --base REPO seals the public base again under the key and compares the result with a sealed model tensor by tensor: a reproducibility check of published models. See kllm verify.

  • kllm cloud MODEL -o DIR writes the keyless cloud part of a sealed model for client-keyed serving, unlocking locked weights on the user’s side. See kllm cloud.

  • A pull and run workflow on production engines: kllm pull, kllm run, kllm list (alias ls), kllm ps, kllm stop, kllm rm, kllm show, kllm logs and kllm search, with model aliases such as qwen3:8b resolved through a curated model library, a local store under KLLM_HOME and a store record per model. Catalog models have aliases such as qwen3:8b-sealed. See Quickstart and Model library and naming.

  • A named key ring: kllm key new, kllm key list, kllm key import and kllm key rm. Sealed models are matched to keys by fingerprint.

  • kllm create seals a pulled model under a private key into the store, using kllm seal unchanged; --no-lock is passed through.

  • kllm check checks a local model against its store record and, for a catalog model, its published manifest, or runs the attestation binding check for an https:// address.

  • kllm doctor, kllm completion and kllm request.

  • A public model wishlist through the Model request issue form. See Requesting model support.

  • The kLLM website and the key-application service with its administration console, which delivers the key files of the granted catalog models. See The kLLM website and The key-application service.

Changes

  • kllm seal locks the weights by default. Engines that load a sealed directory directly need the unlocked copy: kllm unlock writes it and reports its path on standard error (Other launchers: kllm unlock). kllm serve --client refuses a directory with locked weights; kllm seal --client and kllm cloud write the cloud’s part unlocked.

  • The 0.2-line manifest check is named kllm check. Earlier builds of the 0.2 line named it kllm verify, which is now the core reproducibility check.

  • The shared community key is replaced by model keys, one per catalog model. The key-application service takes ACCESS_KEY_DIR instead of ACCESS_KEY_FILE and ACCESS_KEY_FILENAME.

  • The store record of a local model moved out of the model directory, to .DIRECTORY.kllm-store.json next to it, so that a catalog model’s own kllm-manifest.json stays as published.

Known issues

  • kllm run unlocks a sealed model on the local machine as kllm serve --key does, so the decrypted tokenizer is written to /dev/shm and the unlocked weights to the cache directory, and neither is removed. It protects nothing from the machine it runs on.

  • kllm run passes the key passphrase to the background server through the server’s environment, which other processes of the same user can read.

  • No catalog model has been confirmed as published yet.

Unreleased (0.1.0.dev0)

Highlights

  • A second transform, perm, alongside orth.

  • Sealing for Qwen2/2.5, Llama 3.x, Qwen3-MoE and FP8 checkpoints, in addition to Qwen3.

  • Serving on vLLM as well as SGLang.

  • Client-keyed mode: serving without a confidential VM while the key and the tokenizer stay on the user’s machine.

  • Sealing and serving of the MiniMax H3 FL2VA pipeline.

Measured on one NVIDIA A40 48 GB (SGLang 0.5.21 or vLLM 0.31.0, bf16, 1024 input and 128 output tokens, single runs on a GPU shared with another tenant’s light job), sealed output throughput matched plain serving in every measured keyed configuration, and was never more than 0.3% lower at concurrency 16. Client-keyed mode under vLLM cost 7% throughput at concurrency 16 (1036 to 963 tokens/s). See Benchmarking plain and sealed serving.

New features

  • kllm seal --transform perm seals with signed permutations of the residual stream. Every sealed weight is a bit-exact rearrangement of an original one. orth remains the default. See Choosing a transform.

  • MiniMax H3 (FL2VA): the text encoder, the diffusion transformer (DiT), the latent interfaces and the tokenizers are sealed together. kllm serve runs the pipeline with sglang serve --model-variant fl2va. MiniMax H3 is served with SGLang only. See Sealing and serving MiniMax H3.

  • New model families:

    • Qwen2/2.5 and Llama 3.x, with attention biases, without Q/K norms and with RoPE scaling;

    • Qwen3-MoE, with an expert permutation and per-expert MLP transforms;

    • FP8 checkpoints of the supported families with block-wise or per-tensor scales, under perm only.

    See Supported models and Sealing FP8 and mixture-of-experts checkpoints.

  • kllm serve --engine vllm serves sealed models with vLLM. See Serving with vLLM.

  • kllm unlock MODEL decrypts a sealed model’s tokenizer into memory and prints its path, for launchers such as NVIDIA Dynamo and llm-d.

  • kllm serve --int8 [attention|linear|all] runs the MiniMax H3 DiT on INT8 kernels: INT8 attention (Triton) and W8A8 linear layers whose weights are re-quantized at each call. On an A40, the time per denoising step fell by 25% (24.56 s to 18.38 s). This is a kernel optimization independent of sealing, with measured numerical drift. Only the attention part is verified to commute with perm. See Running the H3 diffusion transformer on INT8 kernels.

  • Client-keyed mode, without a confidential VM:

    • kllm seal --client also writes a -cloud directory (or the directory given with --cloud) that holds the weights only, without tokenizer or key material;

    • kllm serve --client serves that directory without a key: SGLang or vLLM started with --skip-tokenizer-init, or a latent server for Z-Image;

    • kllm client proxy and kllm client generate tokenize, detokenize and decode on the user’s side;

    • Z-Image results are encrypted to a one-off X25519 key of each request.

    benchmarks/client.sh and benchmarks/zimage_client.py measure the mode. Scope and limits are described in Client-keyed mode and Claims.

  • benchmarks/ provides one command per table of the README. examples/ provides a quickstart script and NVIDIA Dynamo and Kubernetes setups. A Dockerfile builds an image on top of an SGLang or vLLM image.

Changes

  • The keyed code, previously distributed as kllm-trust, is now part of the open-source package as kllm.keyed, installed with the seal extra.

  • Key release inside confidential VMs (the cdh and gramine key sources) and the attesting TLS gateway moved to kLLM Enterprise. The user-side binding check, python -m kllm.keyed.attest, remains open source. See License.

  • Sealed LLMs and MiniMax H3 pipelines use the L1 scheme: static transforms of the weights plus a permuted vocabulary. Keyed Z-Image pipelines keep their own scheme (d1-v1), and client-keyed Z-Image pipelines are tagged client. See Versioning and format stability.

Known issues

  • kllm demo up fails to start its services. The demo is under repair.

  • kllm serve --key outside a confidential VM decrypts the tokenizer into host /dev/shm and offers no protection from the host. It is a development mode. The decrypted tokenizer is not removed when the server exits.

  • kllm serve --client binds 0.0.0.0 by default and has no authentication. Bind 127.0.0.1 or place it behind a separate gateway.

  • Because --key takes an optional value, write kllm serve MODEL --key [KEYFILE] with the model first; kllm serve --key MODEL reads MODEL as the key file.

  • The NVIDIA Dynamo, llm-d and Kubernetes examples have not been run end to end.

  • The base images in the Dockerfile are not pinned to a version.

  • The Attested CPU profile has not yet been validated on TDX, SEV-SNP or Hygon CSV hardware.

0.1.0

First version, not published on PyPI:

  • serving recipes for SGLang;

  • kllm seal and kllm serve --key for Qwen3 under orth;

  • the Z-Image demo.