The Local LLM Index / Quantization & Formats / #31
intel/neural-compressor
by intel · Quantization & Formats · updated today
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
71
momentum
2,688
stars
318
forks
#31
rank
auto-tuningawqfp4gptqint4int8knowledge-distillationlarge-language-modelslow-precisionmxformatpost-training-quantizationpruning
View on GitHub →