Local LLM Index / local
← All tools
The Local LLM Index / Quantization & Formats / #136

SharpAI/SwiftLM

by SharpAI · Quantization & Formats · updated 2mo ago

⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.

52
momentum
727
stars
46
forks
#136
rank
apple-siliinferenceiosllmmetalmlxmoeon-device-aiopenai-apiswift
View on GitHub →

More in Quantization & Formats

Quantization & Formats#3
llamafile
Distribute and run LLMs with a single file.
Quantization & Formats#13
shimmy
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU.
Quantization & Formats#14
cactus
Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and
Quantization & Formats#26
whichllm
Find the local LLM that actually runs and performs best on your hardware. Ranked by real,