Local LLM Index / local
← All tools
The Local LLM Index / Quantization & Formats / #24

Michael-A-Kuykendall/shimmy

by Michael-A-Kuykendall · Quantization & Formats · updated 8d ago

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

73
momentum
5,995
stars
586
forks
#24
rank
api-servercommand-line-tooldeveloper-toolsggufhuggingfacehuggingface-modelshuggingface-transformersinference-serverllamallamacppllm-inferencelocal-ai
View on GitHub →

More in Quantization & Formats

Quantization & Formats#1
unsloth
Local UI to run and train LLMs and diffusion models. Supports GGUF, MLX, Qwen3.8, DeepSeek
Quantization & Formats#4
project-maya
Project Maya - run GLM-5.3-Flash (321B MoE), DeepSeek-V4.1-Flash-1017 (748B MoE) and MiniM
Quantization & Formats#5
llamafile
Distribute and run LLMs with a single file.
Quantization & Formats#11
Soup
Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.