The Local LLM Index / Quantization & Formats / #85

Helldez/BigMoeOnEdge

by Helldez · Quantization & Formats · updated today

Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.

65
momentum
676
stars
77
forks
#85
rank
androidcppedge-aigemmaggufinferencellama-cppllmmixture-of-expertsmoeon-device-aiqwen
View on GitHub →