StayLameBro/backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Language: Python
#apple_silicon #ios #iphone #llama_cpp #llm_inference #local_llm #macos #metal #qwen #sme2 #speculative_decoding
Stars: 693 Issues: 4 Forks: 66
https://github.com/StayLameBro/backburner
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Language: Python
#apple_silicon #ios #iphone #llama_cpp #llm_inference #local_llm #macos #metal #qwen #sme2 #speculative_decoding
Stars: 693 Issues: 4 Forks: 66
https://github.com/StayLameBro/backburner
GitHub
GitHub - StayLameBro/backburner: Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB…
Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable - StayLameBro/backburner
👍3❤2
eagle-hq/EAGLE-4
Official Inference Code of EAGLE-4
Language: Python
#eagle #gpu #llm_inference #sglang #speculative_decoding #tokenspeed #vllm
Stars: 889 Issues: 5 Forks: 28
https://github.com/eagle-hq/EAGLE-4
Official Inference Code of EAGLE-4
Language: Python
#eagle #gpu #llm_inference #sglang #speculative_decoding #tokenspeed #vllm
Stars: 889 Issues: 5 Forks: 28
https://github.com/eagle-hq/EAGLE-4
GitHub
GitHub - eagle-hq/EAGLE-4: Official Inference Code of EAGLE-4
Official Inference Code of EAGLE-4. Contribute to eagle-hq/EAGLE-4 development by creating an account on GitHub.
👍3
sybil-solutions/inference-optimisation-skill
Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization
Language: Python
#agent_skill #benchmarking #claude_code #inference #llm #speculative_decoding
Stars: 986 Issues: 3 Forks: 85
https://github.com/sybil-solutions/inference-optimisation-skill
Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization
Language: Python
#agent_skill #benchmarking #claude_code #inference #llm #speculative_decoding
Stars: 986 Issues: 3 Forks: 85
https://github.com/sybil-solutions/inference-optimisation-skill
GitHub
GitHub - sybil-solutions/inference-optimisation-skill: Agent skill for hill-climbing local LLM serving speed (decode, speculative…
Agent skill for hill-climbing local LLM serving speed (decode, speculative decoding, prefill, KV, startup, TTFT) on any hardware and quantization - sybil-solutions/inference-optimisation-skill
👍1