The Local LLM Index / Inference Engines / #209
xaskasdf/ntransformer
by xaskasdf · Inference Engines · updated 8mo ago
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.
30
momentum
467
stars
20
forks
#209
rank
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.