✨Can I Have Your Order? Monte-Carlo Tree Search for Slot Filling Ordering in Diffusion Language Models

📝 Summary:
McDiffuSE uses Monte Carlo Tree Search to optimize slot infilling order in Masked Diffusion Models, enhancing reasoning performance. It achieved significant gains, revealing non-sequential generation and larger exploration are key to overcoming model biases.

🔹 Publication Date: Published on Feb 13

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.12586
• PDF: https://arxiv.org/pdf/2602.12586

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MonteCarloTreeSearch #DiffusionModels #NLP #LanguageModels #AI

✨HLE-Verified: A Systematic Verification and Structured Revision of Humanity's Last Exam

📝 Summary:
HLE-Verified systematically validates and revises the HLE benchmark, resolving noisy items through expert review and model-based checks. This improves language model evaluation accuracy by 7-10 percentage points, especially on erroneous items, enabling more reliable measurement of model capabilit...

🔹 Publication Date: Published on Feb 15

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.13964
• PDF: https://arxiv.org/pdf/2602.13964

✨ Datasets citing this paper:
• https://huggingface.co/datasets/skylenage/HLE-Verified

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LLMEvaluation #Benchmarking #LanguageModels #AIResearch #NLP

✨Sink-Aware Pruning for Diffusion Language Models

📝 Summary:
Diffusion Language Models have high inference costs. This paper finds that their attention sinks are often unstable, unlike in autoregressive models. Sink-Aware Pruning identifies and removes these unstable sinks, improving efficiency and quality without retraining.

🔹 Publication Date: Published on Feb 19

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.17664
• PDF: https://arxiv.org/pdf/2602.17664
• Github: https://github.com/VILA-Lab/Sink-Aware-Pruning

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#DiffusionModels #LanguageModels #ModelPruning #NLP #AIResearch

✨One-step Language Modeling via Continuous Denoising

📝 Summary:
This paper introduces flow-based language models that use continuous denoising over one-hot token encodings. They surpass discrete diffusion models in quality and speed, particularly for few-step generation, challenging discrete diffusion's necessity for discrete data.

🔹 Publication Date: Published on Feb 18

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.16813
• PDF: https://arxiv.org/pdf/2602.16813
• Project Page: https://one-step-lm.github.io/
• Github: https://github.com/david3684/flm

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LanguageModels #GenerativeAI #DeepLearning #NLP #AI

✨Reward Prediction with Factorized World States

📝 Summary:
StateFactory transforms observations into hierarchical object-attribute structures using language models. This enables superior zero-shot reward prediction across domains by measuring semantic similarity, significantly improving agent planning performance.

🔹 Publication Date: Published on Mar 10

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.09400
• PDF: https://arxiv.org/pdf/2603.09400
• Project Page: https://statefactory.github.io/
• Github: https://github.com/yijunshens/StateFactory

✨ Datasets citing this paper:
• https://huggingface.co/datasets/YijunShen/RewardPrediction

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#RewardPrediction #AI #LanguageModels #MachineLearning #AgentPlanning

✨Training Language Models via Neural Cellular Automata

📝 Summary:
This paper introduces using Neural Cellular Automata NCA to generate synthetic data for pre-pre-training language models, addressing natural language limitations. This approach improves performance, accelerates convergence, and transfers to reasoning tasks, often outperforming extensive natural l...

🔹 Publication Date: Published on Mar 9

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.10055
• PDF: https://arxiv.org/pdf/2603.10055

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#AI #LanguageModels #NeuralCellularAutomata #SyntheticData #NLP

✨LoopRPT: Reinforcement Pre-Training for Looped Language Models

📝 Summary:
LoopRPT is a reinforcement pre-training framework for looped language models. It directly shapes intermediate representations by assigning reinforcement signals to latent steps, improving latent reasoning. This leads to better accuracy-computation trade-offs and enhanced early-stage reasoning.

🔹 Publication Date: Published on Mar 20

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.19714
• PDF: https://arxiv.org/pdf/2603.19714

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#ReinforcementLearning #LanguageModels #AI #NLP #DeepLearning

✨Diffutron: A Masked Diffusion Language Model for Turkish Language

📝 Summary:
Diffutron introduces a compact masked diffusion language model for Turkish. It uses resource-efficient LoRA-based pre-training and progressive instruction tuning. The model achieves competitive performance for non-autoregressive Turkish text generation despite its small size.

🔹 Publication Date: Published on Mar 20

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.20466
• PDF: https://arxiv.org/pdf/2603.20466

🔹 Models citing this paper:
• https://huggingface.co/diffutron/DiffutronLM-0.3B-Instruct
• https://huggingface.co/diffutron/DiffutronLM-0.3B-Base
• https://huggingface.co/diffutron/DiffutronLM-0.3B-1st-Stage

✨ Datasets citing this paper:
• https://huggingface.co/datasets/diffutron/DiffutronLM-Pretraining-Corpus

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#LanguageModels #TurkishNLP #DiffusionModels #NLP #AI

✨T5Gemma-TTS Technical Report

📝 Summary:
T5Gemma-TTS is an encoder-decoder codec language model that improves voice cloning and duration control for multilingual speech synthesis. It uses cross-attention for persistent text conditioning and Progress-Monitoring Rotary Position Embedding PM-RoPE for better target speech length tracking. I...

🔹 Publication Date: Published on Apr 2

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.01760
• PDF: https://arxiv.org/pdf/2604.01760
• Github: https://github.com/Aratako/T5Gemma-TTS

🔹 Models citing this paper:
• https://huggingface.co/Aratako/T5Gemma-TTS-2b-2b

✨ Spaces citing this paper:
• https://huggingface.co/spaces/Aratako/T5Gemma-TTS-Demo
• https://huggingface.co/spaces/litagin/T5Gemma-TTS-Demo

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#SpeechSynthesis #TTS #VoiceCloning #Multilingual #LanguageModels

🔥 Tmax: A simple recipe for terminal agents

💡 The paper presents a novel approach to training terminal agents using reinforcement learning, called Tmax. Terminal agents are a popular application of language models, but their training has been hindered by the lack of simple and effective methods, limited data, and challenging benchmarks. The authors address these issues by introducing a simplified recipe for training terminal agents, which achieves superior performance with fewer parameters than previous methods.

The method involves generating a large dataset of terminal environments using a novel taxonomy that combines difficulty control, personas, and verifier diversification. This allows for the cheap generation of large amounts of data, which is then used to train open-weight models using reinforcement learning with a simple outcome-only recipe.

The results show that Tmax achieves 27 percent on Terminal-Bench 2.0 with only 9 billion parameters, outperforming much larger models from prior work. The authors also release their terminal dataset, which is over 2.5 times larger than previously released terminal-agent datasets, as well as their models and code as a strong baseline for future academic work on terminal agents.

The contributions of the paper are threefold. First, it presents a simple and effective recipe for training terminal agents, which can be used as a baseline for future work. Second, it introduces a novel taxonomy for generating terminal environments, which allows for the cheap generation of large amounts of data. Third, it releases a large dataset of terminal environments, models, and code, which can be used by other researchers to advance the field of terminal agents. Overall, the paper provides a significant contribution to the field of terminal agents and reinforcement learning, and has the potential to advance the state of the art in this area.

📅 Published on Jun 22

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2606.23321
• PDF: https://arxiv.org/pdf/2606.23321
• Project Page: https://wai-org.com/blog/tmax/

🤖 Models citing this paper:
• https://huggingface.co/allenai/tmax-27b
• https://huggingface.co/allenai/tmax-9b
• https://huggingface.co/allenai/qwen35-9b-openthoughts

📊 Datasets citing this paper:
• https://huggingface.co/datasets/allenai/TMax-15K
• https://huggingface.co/datasets/allenai/tmax-15k-open-instruct
• https://huggingface.co/datasets/allenai/tmax-sft

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#TerminalAgents #ReinforcementLearning #LanguageModels #TmaxAlgorithm #AgentTrainingMethods