✨LK Losses: Direct Acceptance Rate Optimization for Speculative Decoding

📝 Summary:
LK losses directly optimize speculative decoding acceptance rate, outperforming standard KL divergence training. This boosts speedup, showing consistent gains of up to 10% in average acceptance length across various models and domains with no extra overhead.

🔹 Publication Date: Published on Feb 27

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.23881
• PDF: https://arxiv.org/pdf/2602.23881

🔹 Models citing this paper:
• https://huggingface.co/nebius/MTP-DeepSeek-V3-0324
• https://huggingface.co/nebius/MEDUSA-Llama-3.1-8B-Instruct
• https://huggingface.co/nebius/MLP-Speculator-Llama-3.1-8B-Instruct

✨ Datasets citing this paper:
• https://huggingface.co/datasets/nebius/Llama-3.1-8B-Instruct-Infinity-Instruct-0625
• https://huggingface.co/datasets/nebius/gpt-oss-20b-Infinity-Instruct-0625
• https://huggingface.co/datasets/nebius/DeepSeek-V3-Infinity-Instruct-0625

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeculativeDecoding #LLMs #LLMOptimization #DeepLearning #AIResearch

✨ConFu: Contemplate the Future for Better Speculative Sampling

📝 Summary:
ConFu is a novel speculative decoding framework that enhances draft models by enabling future-oriented generation prediction. It uses contemplate tokens and soft prompts to anticipate future steps, reducing error accumulation. This significantly improves token acceptance rates and inference speed...

🔹 Publication Date: Published on Mar 9

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.08899
• PDF: https://arxiv.org/pdf/2603.08899

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeculativeDecoding #LLMs #GenerativeAI #AIResearch #InferenceSpeed

✨TAPS: Task Aware Proposal Distributions for Speculative Sampling

📝 Summary:
Speculative decoding quality depends on matching draft model training data to the downstream task. Task-specific training yields specialized drafters that are best combined at inference time using confidence-based routing, outperforming averaging. Confidence is a more effective routing signal tha...

🔹 Publication Date: Published on Mar 27

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.27027
• PDF: https://arxiv.org/pdf/2603.27027
• Github: https://github.com/Moe-Zbeeb/TAPS

🔹 Models citing this paper:
• https://huggingface.co/zbeeb/Hass-MathInstruct_20epochs
• https://huggingface.co/zbeeb/Hass-ShareGPT_20epochs
• https://huggingface.co/zbeeb/Hass-Sharegpt-Mathinstruct-20epochs

✨ Datasets citing this paper:
• https://huggingface.co/datasets/zbeeb/TAPS-Datasets

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeculativeDecoding #LLM #MachineLearning #AIResearch #NLP

✨SPEED-Bench: A Unified and Diverse Benchmark for Speculative Decoding

📝 Summary:
SPEED-Bench is introduced as a new benchmark for Speculative Decoding SD evaluation. It provides diverse semantic domains and realistic serving regimes to address limitations of existing benchmarks. This enables accurate measurement of SD performance in production environments, setting a unified ...

🔹 Publication Date: Published on Feb 10

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.09557
• PDF: https://arxiv.org/pdf/2604.09557
• Project Page: https://huggingface.co/blog/nvidia/speed-bench
• Github: https://github.com/NVIDIA/Model-Optimizer

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeculativeDecoding #AIBenchmarks #LLMs #DeepLearning #ModelOptimization

✨Accelerating Speculative Decoding with Block Diffusion Draft Trees

📝 Summary:
DDTree enhances speculative decoding by constructing draft trees from block diffusion drafter distributions. It efficiently verifies multiple trajectories in parallel in a single target model pass, improving performance.

🔹 Publication Date: Published on Apr 14

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.12989
• PDF: https://arxiv.org/pdf/2604.12989
• Project Page: https://liranringel.github.io/ddtree
• Github: https://github.com/liranringel/ddtree

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeculativeDecoding #BlockDiffusion #LLMAcceleration #DeepLearning #AIResearch

🔥 DFlash: Block Diffusion for Flash Speculative Decoding

💡 The paper introduces DFlash, a speculative decoding framework designed to improve the speed of large language models while maintaining their quality. The problem with current large language models is that they require sequential decoding, which leads to high latency and poor GPU utilization. Speculative decoding has been proposed as a solution, where a fast draft model generates outputs that are then verified in parallel by the target model. However, existing speculative decoding methods still rely on sequential drafting, which limits their speedup.

To address this, the authors propose using a lightweight block diffusion model for parallel drafting. This model generates draft tokens in a single forward pass and conditions the draft model on context features extracted from the target model. The result is a framework that enables efficient drafting with high-quality outputs and higher acceptance rates.

The experiments show that DFlash achieves significant speedup over existing autoregressive methods, with over 6x lossless acceleration across a range of models and tasks. This is up to 2.5x higher speedup than the state-of-the-art speculative decoding method. The method contributes to improving the efficiency of large language models, making them more suitable for practical applications. Overall, DFlash offers a promising solution for speeding up large language models without sacrificing their performance.

📅 Published on Feb 5

🔗 Links:
• arXiv: https://arxiv.org/abs/2602.06036
• PDF: https://arxiv.org/pdf/2602.06036
• Project Page: https://z-lab.ai/projects/dflash/
• GitHub: https://github.com/z-lab/dflash ⭐ 3.1k

🤖 Models citing this paper:
• https://huggingface.co/z-lab/Qwen3.6-27B-DFlash
• https://huggingface.co/z-lab/Qwen3.6-35B-A3B-DFlash
• https://huggingface.co/z-lab/Qwen3.5-27B-DFlash

🚀 Spaces citing this paper:
• https://huggingface.co/spaces/Jackrong/qwen36-eval

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeculativeDecoding #BlockDiffusionModels #LargeLanguageModels #ParallelDecodingTechniques #FlashSpeculativeDecoding

🔥 JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

💡 The paper introduces JetSpec, a speculative decoding framework designed to improve the inference speed and acceptance rates of large language models. The problem addressed is the scaling limitation of speculative decoding, which accelerates autoregressive large language models by drafting multiple tokens and verifying them in parallel. However, increasing the draft budget only improves speed when acceptance remains high and drafting overhead stays low, creating a scaling ceiling.

The proposed JetSpec framework combines efficient forward drafting with causal conditioning to break this ceiling. It trains a causal parallel draft head over fused hidden states from the frozen target model, producing candidate trees whose scores align with the target model's autoregressive factorization. This approach enables JetSpec to convert larger draft budgets into longer accepted prefixes and higher end-to-end speedup.

The method is compared to bidirectional-head and tree-based speculative decoding baselines across various benchmarks, including math, coding, and chat tasks on dense and MoE models. The results show that JetSpec consistently outperforms these baselines, achieving significant speedup on different workloads. Specifically, JetSpec achieves up to 9.64x speedup on math tasks and 4.58x on open-ended conversational workloads, with further latency gains demonstrated through integration with virtual large language models under realistic serving loads.

Overall, the paper contributes a novel speculative decoding framework that breaks the scaling ceiling of prior methods, enabling faster and more efficient large language model inference. The code and models are made available for further research and development.

📅 Published on Jun 25

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.18394
• PDF: https://arxiv.org/pdf/2606.18394
• Project Page: https://jetspec-project.github.io/jetspec-web/

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#SpeculativeDecoding #LargeLanguageModels #AutoregressiveModeling #ParallelTreeDrafting #CausalConditioning