opencv/dldt
Deep Learning Deployment Toolkit
Language: C++
#deep_learning #inference #inference_engine #performance
Stars: 97 Issues: 2 Forks: 13
https://github.com/opencv/dldt

Tencent/Forward
a library for high performance deep learning inference on NVIDIA GPUs.
Language: C++
#cuda #deep_learning #forward #gpu #inference #inference_engine #keras #neural_network #pytorch #tensorflow #tensorrt
Stars: 102 Issues: 0 Forks: 8
https://github.com/Tencent/Forward

chengzeyi/stable-fast
An ultra lightweight inference performance optimization library for HuggingFace Diffusers on NVIDIA GPUs.
Language: Python
#cuda #deep_learning #deeplearning #diffusers #inference #inference_engine #performance_optimization #pytorch #stable_diffusion #triton
Stars: 134 Issues: 3 Forks: 5
https://github.com/chengzeyi/stable-fast

zhihu/ZhiLight
A highly optimized inference acceleration engine for Llama and its variants.
Language: C++
#cpm #cuda #gpt #inference_engine #llama #llm #llm_serving #minicpm #pytorch #qwen
Stars: 192 Issues: 1 Forks: 16
https://github.com/zhihu/ZhiLight

FareedKhan-dev/kimi-k3-in-c
A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.
Language: C
#avx2 #c99 #cpu_inference #deep_learning #from_scratch #inference_engine #kimi_k3 #linear_attention #llm #llm_inference #machine_learning #memory_efficient #mixture_of_experts #moe #mxfp4 #quantization #simd #systems_programming #transformer #zero_dependencies
Stars: 1398 Issues: 5 Forks: 220
https://github.com/FareedKhan-dev/kimi-k3-in-c