The Local LLM Index / Quantization & Formats / #32
intel/neural-compressor
by intel · Quantization & Formats · updated 1d ago
SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime
70
momentum
2,654
stars
307
forks
#32
rank
auto-tuningawqfp4gptqint4int8knowledge-distillationlarge-language-modelslow-precisionmxformatpost-training-quantizationpruning
View on GitHub →