opencv/dldt
Deep Learning Deployment Toolkit
Language: C++
#deep_learning #inference #inference_engine #performance
Stars: 97 Issues: 2 Forks: 13
https://github.com/opencv/dldt

BMW-InnovationLab/BMW-TensorFlow-Training-GUI
This repository allows you to get started with a gui based training a State-of-the-art Deep Learning model with little to no configuration needed! Training with TensorFlow has never been so easy.
Language: Python
#computer_vision #computervision #deep_learning #deep_neural_networks #deeplearning #detection_api #docker #gui #inference_api #neural_network #nvidia_docker #object_detection #objectdetection #resnet #rest_api #ssd #tensorboard #tensorflow #tensorflow_gui #tensorflow_training
Stars: 176 Issues: 0 Forks: 16
https://github.com/BMW-InnovationLab/BMW-TensorFlow-Training-GUI

BMW-InnovationLab/BMW-TensorFlow-Inference-API-GPU
This is a repository for an object detection inference API using the Tensorflow framework.
Language: Python
#api #computer_vision #deep_learning #deep_neural_networks #detection_inference_api #docker #dockerfile #gpu #inference #neural_network #nvidia #object_detection #rest_api #tensorflow #tensorflow_framework #tensorflow_models
Stars: 150 Issues: 2 Forks: 50
https://github.com/BMW-InnovationLab/BMW-TensorFlow-Inference-API-GPU

vriad/zod
Typescript-first schema validation with static type inference
Language: TypeScript
#inference #schema_definitions #schema_validation #type_inference #typescript
Stars: 295 Issues: 8 Forks: 2
https://github.com/vriad/zod

Tencent/Forward
a library for high performance deep learning inference on NVIDIA GPUs.
Language: C++
#cuda #deep_learning #forward #gpu #inference #inference_engine #keras #neural_network #pytorch #tensorflow #tensorrt
Stars: 102 Issues: 0 Forks: 8
https://github.com/Tencent/Forward

tairov/llama2.mojo
Inference Llama 2 in one file of pure 🔥
#inference #llama #llama2 #modular #mojo #parallelize #performance #simd #tensor #vectorization
Stars: 200 Issues: 0 Forks: 7
https://github.com/tairov/llama2.mojo

chengzeyi/stable-fast
An ultra lightweight inference performance optimization library for HuggingFace Diffusers on NVIDIA GPUs.
Language: Python
#cuda #deep_learning #deeplearning #diffusers #inference #inference_engine #performance_optimization #pytorch #stable_diffusion #triton
Stars: 134 Issues: 3 Forks: 5
https://github.com/chengzeyi/stable-fast

Fuzzy-Search/realtime-bakllava
llama.cpp with BakLLaVA model describes what does it see
Language: Python
#bakllavva #cpp #demo_application #inference #llama #llamacpp #llm
Stars: 141 Issues: 1 Forks: 15
https://github.com/Fuzzy-Search/realtime-bakllava

hpcaitech/SwiftInfer
Efficient AI Inference & Serving
Language: Python
#artificial_intelligence #deep_learning #gpt #inference #llama #llama2 #llm_inference #llm_serving
Stars: 299 Issues: 3 Forks: 14
https://github.com/hpcaitech/SwiftInfer

arc53/llm-price-compass
LLM provider price comparison, gpu benchmarks to price per token calculation, gpu benchmark table
Language: TypeScript
#benchmark #gpu #inference_comparison #llm #llm_comparison #llm_inference #llm_price
Stars: 138 Issues: 1 Forks: 5
https://github.com/arc53/llm-price-compass

zml/zml
High performance AI inference stack. Built for production. @ziglang / @openxla / MLIR / @bazelbuild
Language: Zig
#ai #bazel #hpc #inference #xla #zig
Stars: 691 Issues: 1 Forks: 19
https://github.com/zml/zml

AgibotTech/agibot_x1_infer
The inference module for AgiBot X1.
Language: C++
#inference #open_source #robotics
Stars: 455 Issues: 2 Forks: 152
https://github.com/AgibotTech/agibot_x1_infer

thu-ml/SageAttention
Quantized Attention that achieves speedups of 2.1x and 2.7x compared to FlashAttention2 and xformers, respectively, without lossing end-to-end metrics across various models.
Language: Python
#attention #inference_acceleration #llm #quantization
Stars: 145 Issues: 6 Forks: 3
https://github.com/thu-ml/SageAttention

zhihu/ZhiLight
A highly optimized inference acceleration engine for Llama and its variants.
Language: C++
#cpm #cuda #gpt #inference_engine #llama #llm #llm_serving #minicpm #pytorch #qwen
Stars: 192 Issues: 1 Forks: 16
https://github.com/zhihu/ZhiLight