The Local LLM Index / Quantization & Formats / #85
Helldez/BigMoeOnEdge
by Helldez · Quantization & Formats · updated today
Run MoE models bigger than your RAM. Frontier-size MoE on a phone, laptop, edge. Lossless, on stock llama.cpp. An adaptive streaming optimizer. Any hardware, any OS, any model, any quantization, behind a local OpenAI-compatible server.
65
momentum
676
stars
77
forks
#85
rank
androidcppedge-aigemmaggufinferencellama-cppllmmixture-of-expertsmoeon-device-aiqwen
View on GitHub →