Settings

kLLM has no configuration file. Its behavior is set by command-line options (see Command-line reference), by the environment variables on this page and by a small number of fixed paths. This page lists every environment variable that kLLM 0.1.0.dev0 reads or sets, the process that reads it and the mode in which it matters, together with the variables added by the commands of the upcoming 0.2 line (Model, key and runtime commands (0.2 line)), which are marked as such.

Overview

Variable	Read by	Mode
KLLM_PASSPHRASE	kllm commands that unlock a key file	Development and unattended starts with --key-source passphrase
KLLM_PRODUCTION	kllm serve; the key-source and serving code in kllm.keyed	Production (Attested CPU profile)
KLLM_KEY_SOURCE	kllm.keyed.tee.keysource.load_key()	Library use and key-source plug-ins
KLLM_MODEL_KEY	kllm.keyed.d1.server; load_key() with the env source	Demo and development only; refused in production
KLLM_MODEL_DIR	Set by kLLM for the demo image server	Demo only
KLLM_TRUSTED_FROM	kllm.keyed.d1.server	Demo only
KLLM_INT8	SGLang processes that load the INT8 site hook	MiniMax H3 with kllm serve --int8
KLLM_UNLOCKED_DIR	kllm serve --key, --key-source cdh and kllm unlock	All modes that serve a model with locked weights
KLLM_CDH_URL, KLLM_CDH_RESOURCE	The cdh key source (kLLM Enterprise)	Confidential VM
KLLM_AA_URL	The attesting TLS gateway (kLLM Enterprise)	Confidential VM
KLLM_HOME	The commands of the 0.2 line	All modes (0.2 line)
KLLM_LIBRARY	The commands of the 0.2 line that resolve aliases	All modes (0.2 line)
HF_TOKEN, HF_ENDPOINT	huggingface_hub, used by kllm pull and kllm verify	Downloads
CUDA_VISIBLE_DEVICES	kllm serve, kllm demo up	All modes
PYTHONPATH	Extended by kllm serve --int8	MiniMax H3 with INT8 kernels

kLLM variables

KLLM_PASSPHRASE

The passphrase of the key file. When it is set and non-empty, kllm keygen, kllm seal, kllm serve with --key, kllm unlock, kllm client proxy, kllm client generate and kllm demo up use it instead of prompting. kllm keygen then does not ask for confirmation. Without it, and without a terminal, these commands exit with no terminal to read the key passphrase; set KLLM_PASSPHRASE.

In the 0.2 line, kllm key new, kllm create and kllm run read it as well. kllm run reads the passphrase in its own process and passes it to the background server only through that server’s environment, as KLLM_PASSPHRASE; it never appears on a command line or in a file.

The passphrase protects the key file at rest only. A new passphrase must be at least 10 characters long. The benchmark scripts in benchmarks/ and the example in examples/quickstart.sh set a default passphrase when the variable is unset; do not reuse those defaults for keys that matter.

Warning

The environment of a process is readable by its user and by root, for example through /proc/<pid>/environ. Use KLLM_PASSPHRASE for development and unattended test setups. In production, the key comes from a key broker (--key-source cdh) and no passphrase is involved.

KLLM_UNLOCKED_DIR

The directory into which kllm serve --key, kllm serve --key-source cdh and kllm unlock write the unlocked copy of a model with locked weights. The default is ~/.cache/kllm/unlocked. The directory is created with mode 0700, and each copy is a subdirectory named after the sealed directory and a digest of the key fingerprint and the weight files. kLLM does not remove the copies. The unlocked copy holds the sealed weights in the clear; place the directory on storage that only the parties allowed to run the model can read. See Locked weights.

KLLM_HOME

The directory in which the commands of the 0.2 line keep their state. The default is ~/.kllm. It holds models/ (the local store), keys/ (the key ring), run/ (records of background servers) and logs/ (server logs), each created with mode 0700. Part of the upcoming 0.2 line.

KLLM_LIBRARY

Replaces the bundled model library with another catalog in the same format, given as a file path or an https:// address. Use it to test a catalog or to point at a private one. See Model library and naming. Part of the upcoming 0.2 line.

KLLM_PRODUCTION

Production mode. Any value other than empty or 0 enables it; kllm serve --production enables it as well and sets KLLM_PRODUCTION=1 in its own environment, so the engine inherits it. In production mode:

• kllm serve starts only with --key-source cdh. Every other invocation, for plain models as well, exits with --production takes the model key from the CDH only (--key-source cdh).

• kllm serve --client is refused.

• Keyed serving refuses any key that was not released by the cdh key source.

• load_key() refuses keys from the environment (KLLM_PRODUCTION=1 refuses keys from the environment; use cdh).

See Confidential VM deployment.

KLLM_KEY_SOURCE

The name of the key source used by kllm.keyed.tee.keysource.load_key(). The default is env, which reads KLLM_MODEL_KEY. Any other value names a key source registered under the kllm.keysource entry-point group; an unregistered name raises an error that refers to kLLM Enterprise.

The kllm commands of 0.1.0.dev0 do not read this variable. kllm serve selects its key source with --key-source. See Registering a key source.

KLLM_MODEL_KEY

The raw 256-bit model key, as 64 hexadecimal digits (load_key() also accepts 32 raw bytes). It is the key itself, not a passphrase.

• kLLM sets it in the environment of the keyed Z-Image server that kllm demo up starts (python -m kllm.keyed.d1.server). That server removes it from its environment when it starts and exits if it is missing.

• load_key() reads it when the key source is env. It is refused in production mode.

• kllm serve --key does not pass the key to SGLang or vLLM. The engine receives only the path of the decrypted tokenizer.

• kllm serve --dry-run prints the variable as <secret>.

Warning

Removing the variable from Python’s view does not remove it from the initial environment block of the process, which Linux keeps readable through /proc/<pid>/environ for the life of the process. Never set KLLM_MODEL_KEY on a host you do not trust.

KLLM_MODEL_DIR

The absolute path of the sealed model directory. kLLM sets it, together with KLLM_MODEL_KEY, in the environment of the demo’s keyed Z-Image server. No component of the open-source package 0.1.0.dev0 reads it.

KLLM_TRUSTED_FROM

Demo only. The path of the public base pipeline. When it is set, the keyed Z-Image server takes its trusted tensors from that public checkpoint instead of the encrypted trusted.kllm blob, which is how an attacker holding the sealed files would run the model. kllm demo up sets it for the wrong-key image server. Do not set it otherwise.

KLLM_INT8

Selects the INT8 kernels for the 50 DiT blocks of MiniMax H3: attention, linear, a comma-separated list of both, or all (1 is accepted as a synonym of all). Empty or 0 disables the kernels; any other value is an error.

kllm serve --int8 [PARTS] sets it, and prepends the kernel site directory to PYTHONPATH, for a sealed H3 pipeline. The sitecustomize.py in that directory reads the variable in every SGLang process and installs the selected kernels. To use the kernels with sglang serve directly:

$ PYTHONPATH=$(python -m kllm.kernels.h3) KLLM_INT8=all sglang serve --model-path ...

See Running the H3 diffusion transformer on INT8 kernels.

kLLM Enterprise variables

The cdh key source and the attesting TLS gateway are part of kLLM Enterprise, which is licensed separately. The open-source package does not read these variables.

Variable	Meaning
KLLM_CDH_URL	Overrides the address of the Confidential Data Hub REST API in the confidential VM. The CDH REST API listens on http://127.0.0.1:8006 by default.
KLLM_CDH_RESOURCE	Overrides the resource path of the model key in the deployer’s key broker (default default/kllm/model-key). The resource holds 32 raw bytes or 64 hexadecimal digits.
KLLM_AA_URL	Address of the Attestation Agent from which the gateway obtains evidence, for example http://127.0.0.1:8006. The gateway can also obtain evidence from the kernel’s configfs-tsm interface.

See Confidential VM deployment and Verifying a deployment.

Other variables

CUDA_VISIBLE_DEVICES

kllm serve counts the GPUs that nvidia-smi reports and that a numeric CUDA_VISIBLE_DEVICES selects; the count sets the default tensor-parallel size, and the first GPU’s name sets the GPU family used to match recipes. Only numeric indices are resolved: a value that contains a GPU UUID leaves every GPU counted. kllm demo up takes its default --gpus from this variable and sets it for each demo server.

PYTHONPATH

kllm serve --int8 prepends the INT8 kernel site directory to the existing value in the engine’s environment.

HOME

The default key path ~/.kllm/keys/default.key and the demo state directory ~/.kllm/demo are resolved from the home directory of the user.

HF_TOKEN, HF_ENDPOINT and the other huggingface_hub variables

Read by huggingface_hub when kllm pull or kllm verify downloads from Hugging Face: HF_TOKEN for gated repositories, HF_ENDPOINT for a mirror.

The engine reads its own environment variables as usual; kLLM passes its environment to the engine unchanged, together with the variables of the selected recipe and those listed above.

Tool and benchmark variables

The scripts in tools/ and benchmarks/ are not part of the installed package. The benchmark shell scripts set KLLM_PASSPHRASE to kllm-bench when it is unset; the Python tools that unlock a key file, such as tools/zimage_fidelity.py, require the variable and do not prompt. They also read the following variables of their own:

Variable	Meaning
KLLM_DUMP_LATENTS	tools/h3_same_noise.py hook: directory in which every H3 server process saves its final latents before VAE decoding.
KLLM_NOISE_HOOK_STRICT	tools/h3_same_noise.py hook: raise instead of ignoring a failure to install the hook.
GPU, PORT, ENGINE, TRANSFORMS, KEY, OUT and others	Settings of the benchmark scripts; see benchmarks/README.md and Benchmarking plain and sealed serving.

Files and paths

Path	Purpose
~/.kllm/keys/default.key	Default key file of every --key option. Written with mode 0600 in a directory created with mode 0700.
/dev/shm/kllm-tok-*	Directories (mode 0700) into which kllm serve --key and kllm unlock decrypt the permuted tokenizer. They are not removed when the engine exits. If /dev/shm does not exist, the system temporary directory is used.
~/.cache/kllm/unlocked/NAME-DIGEST	Unlocked copies of models with locked weights (mode 0700), written by kllm serve --key and kllm unlock; KLLM_UNLOCKED_DIR moves them. They are not removed.
~/.kllm/demo/state.json, ~/.kllm/demo/*.log	State and logs of kllm demo up.
KLLM_HOME/models/	The local model store of the 0.2 line, with a store record .DIRECTORY.kllm-store.json next to each model directory (Local store).
KLLM_HOME/keys/NAME.key	The key ring of the 0.2 line (kllm key). Files have mode 0600.
KLLM_HOME/run/servers.json	Records of the background servers started by kllm run: process ID, process group, process start time, port, engine and model.
KLLM_HOME/logs/	Logs of the background servers, printed by kllm logs.