The Local LLM Index / Inference Engines / #185
xaskasdf/ntransformer
by xaskasdf · Inference Engines · updated 5mo ago
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.
34
momentum
464
stars
19
forks
#185
rank
High-efficiency LLM inference engine in C++/CUDA. Run Llama 70B on RTX 3090.