Requesting model support

kLLM seals a model only when its architecture has a validated implementation: a sealing provider for that architecture whose transforms are checked by a float64 exactness test. kLLM does not seal unknown architectures on the fly. Models that are not yet supported are collected on a public wishlist in the GitHub repository, and the kLLM team adds them over time.

The wishlist

The wishlist is the set of open issues created with the Model request issue form of the kLLM repository. Each request names one model. Before opening a request, search the open issues for the model and add your support to an existing request instead of opening a duplicate.

The issue form (.github/ISSUE_TEMPLATE/model-request.yml) asks for:

  • the model identifier, for example its Hugging Face repository (required);

  • a link to the upstream model (required);

  • the modality: dense language model, mixture-of-experts language model, image generation (diffusion), video generation, or other (required);

  • the architecture, if known (the architectures field of its config.json);

  • the preferred engine;

  • the reason for its priority, such as the intended use, the quantization format or the hardware;

  • a confirmation that no duplicate request exists and that the model’s license permits redistribution of derived weights, or a note otherwise.

Requests are prioritized by demand and by the work each architecture needs. A request is not a commitment to support the model or to a date.

Opening a request

From the command line, kllm request prints the address of a new request with the model identifier filled in:

$ kllm request mistralai/Mistral-7B-Instruct-v0.3

The printed address opens the form with the title, the model identifier and, for a Hugging Face identifier, the upstream link filled in. Open the address in a browser, complete the form and submit it. The command itself sends nothing. kllm request is part of the upcoming 0.2 line (kllm request). Without it, open the issue form directly from the repository’s New issue page.

When an alias given to kllm pull or kllm run is not in the library, the error message names kllm search. When kllm search finds no match, it prints the matching kllm request command.

Contributing support

To add an architecture yourself, follow Adding a model family. Every new family needs a float64 exactness test. Mention the request issue in the pull request.

Customers of kLLM Enterprise can obtain support for newly released models from the day of their release, under a customer key; see kLLM Enterprise.

See also