The Local LLM Index / Quantization & Formats / #71
StartLuxLabs/StartLux-Prism
by StartLuxLabs · Quantization & Formats · updated 2d ago
Fast, exact inference for StartLux-Decision models on NVIDIA GPUs: the released GGUF files on int8 and 4-bit tensor cores, behind the /v1/systemone API, with up to 5.3× lower latency than vLLM in FP8 and bit-identical answers.
68
momentum
164
stars
4
forks
#71
rank
calibrationcudacuda-graphsdecision-makingggufh200inference-engineint8llm-inferencenvidiaquantizationstartlux
View on GitHub →