SJTU-IPADS/PowerInfer
High-speed Large Language Model Serving on PCs with Consumer-grade GPUs
Language: C
#falcon #large_language_models #llama #llm #llm_inference #local_inference
Stars: 792 Issues: 8 Forks: 32
https://github.com/SJTU-IPADS/PowerInfer

ollaya-dev/ollaya
Run open decision models locally: pull and serve Laya, decider, NLI and GLiClass behind a TypeSafe-compatible API. Ollama for decision models.
Language: Rust
#calibration #classification #decision_models #gliclass #jev #laya #llm_routing #local_inference #nli #ollama #onnx #onnxruntime #rust #typesafe #zero_shot_classification
Stars: 741 Issues: 3 Forks: 36
https://github.com/ollaya-dev/ollaya