calcuis/gguf

Distribute and run LLMs with a single file.

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU.

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and

Find the local LLM that actually runs and performs best on your hardware. Ranked by real,