✨OmniParser for Pure Vision Based GUI Agent

📝 Summary:
OmniParser enhances GPT-4V's ability to act as a GUI agent by improving screen parsing. It identifies interactable icons and understands element semantics using specialized models. This significantly boosts GPT-4V's performance on benchmarks like ScreenSpot, Mind2Web, and AITW.

🔹 Publication Date: Published on Aug 1, 2024

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2408.00203
• PDF: https://arxiv.org/pdf/2408.00203
• Github: https://github.com/microsoft/omniparser

🔹 Models citing this paper:
• https://huggingface.co/microsoft/OmniParser
• https://huggingface.co/microsoft/OmniParser-v2.0
• https://huggingface.co/banao-tech/OmniParser

✨ Datasets citing this paper:
• https://huggingface.co/datasets/mlfoundations/Click-100k

✨ Spaces citing this paper:
• https://huggingface.co/spaces/callmeumer/OmniParser-v2
• https://huggingface.co/spaces/nofl/OmniParser-v2
• https://huggingface.co/spaces/SheldonLe/OmniParser-v2

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#GUIagents #ComputerVision #GPT4V #AIagents #DeepLearning

✨HiconAgent: History Context-aware Policy Optimization for GUI Agents

📝 Summary:
HiconAgent introduces History Context-aware Policy Optimization HCPO for GUI agents. HCPO efficiently leverages historical context using dynamic sampling and compression, achieving better performance than larger models with reduced computational cost and significant speedups.

🔹 Publication Date: Published on Dec 1

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.01763
• PDF: https://arxiv.org/pdf/2512.01763
• Github: https://github.com/JiuTian-VL/HiconAgent

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#HiconAgent #GUIAgents #AIResearch #ReinforcementLearning #ContextAwareAI

✨GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning

📝 Summary:
GUI Exploration Lab is a simulation environment to train GUI agents for screen navigation. It finds supervised fine-tuning establishes basics, single-turn reinforcement learning improves generalization, and multi-turn RL enhances exploration for superior navigation performance.

🔹 Publication Date: Published on Dec 2

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.02423
• PDF: https://arxiv.org/pdf/2512.02423

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#ReinforcementLearning #GUIAgents #AINavigation #MachineLearning #AIResearch

✨MAI-UI Technical Report: Real-World Centric Foundation GUI Agents

📝 Summary:
MAI-UI introduces a family of foundation GUI agents tackling real-world deployment challenges. It uses a self-evolving data pipeline, device-cloud collaboration, and online RL to set new state-of-the-art in GUI grounding and mobile navigation, significantly boosting performance and privacy.

🔹 Publication Date: Published on Dec 26

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2512.22047
• PDF: https://arxiv.org/pdf/2512.22047

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#GUIAgents #AI #ReinforcementLearning #MobileTech #HCI

✨PersonalAlign: Hierarchical Implicit Intent Alignment for Personalized GUI Agent with Long-Term User-Centric Records

📝 Summary:
PersonalAlign is a new framework for GUI agents to align with implicit user intents using hierarchical memory and long-term user records. Their HIM-Agent significantly improves both execution by 15.7% and proactive performance by 7.3%.

🔹 Publication Date: Published on Jan 14

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2601.09636
• PDF: https://arxiv.org/pdf/2601.09636
• Project Page: https://jiutian-vl.github.io/PersonalAlign-page/

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#PersonalAlign #GUIAgents #AI #Personalization #IntelligentAgents

✨MemGUI-Bench: Benchmarking Memory of Mobile GUI Agents in Dynamic Environments

📝 Summary:
MemGUI-Bench is a new, comprehensive benchmark designed to evaluate the memory capabilities of mobile GUI agents. It addresses current benchmarks' failure to assess memory by offering a taxonomy, 128 tasks, and an automated evaluation pipeline. Experiments with state-of-the-art agents reveal sign...

🔹 Publication Date: Published on Feb 3

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2602.06075
• PDF: https://arxiv.org/pdf/2602.06075
• Project Page: https://lgy0404.github.io/MemGUI-Bench/
• Github: https://github.com/lgy0404/MemGUI-Bench

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MobileAI #GUIagents #AIBenchmarking #MemoryAI #AIResearch

✨SlowBA: An efficiency backdoor attack towards VLM-based GUI agents

📝 Summary:
SlowBA is a novel backdoor attack targeting the response latency of VLM-based GUI agents. It induces excessively long reasoning chains using realistic pop-up window triggers, significantly increasing response length and latency while maintaining task accuracy. This reveals a new security vulnerab...

🔹 Publication Date: Published on Mar 9

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.08316
• PDF: https://arxiv.org/pdf/2603.08316
• Github: https://github.com/tu-tuing/SlowBA

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#BackdoorAttack #AISecurity #VLM #GUIagents #Cybersecurity

✨AndroTMem: From Interaction Trajectories to Anchored Memory in Long-Horizon GUI Agents

📝 Summary:
The paper presents AndroTMem, a framework and benchmark diagnosing interaction memory failures in long-horizon GUI agents. It proposes Anchored State Memory ASM, which uses causally linked intermediate-state anchors to overcome this bottleneck, improving task completion rates by up to 30%.

🔹 Publication Date: Published on Mar 19

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2603.18429
• PDF: https://arxiv.org/pdf/2603.18429

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#GUIAgents #AIMemory #AIAgents #AIResearch #HumanComputerInteraction

✨Mobile GUI Agent Privacy Personalization with Trajectory Induced Preference Optimization

📝 Summary:
Mobile GUI agents neglect user privacy personalization, as varied execution trajectories hinder standard optimization. This paper proposes Trajectory Induced Preference Optimization TIPO to address this challenge. TIPO improves persona alignment and task executability, outperforming existing meth...

🔹 Publication Date: Published on Apr 13

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.11259
• PDF: https://arxiv.org/pdf/2604.11259

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#MobileAI #PrivacyTech #Personalization #GUIAgents #MachineLearning

✨Turing Test on Screen: A Benchmark for Mobile GUI Agent Humanization

📝 Summary:
This paper introduces the Turing Test on Screen to address GUI agents detectability by digital platforms. It proposes a benchmark and methods to humanize agent behavior, balancing imitability with task performance, enabling seamless coexistence in adversarial digital environments.

🔹 Publication Date: Published on Feb 24

🔹 Paper Links:
• arXiv Page: https://arxiv.org/abs/2604.09574
• PDF: https://arxiv.org/pdf/2604.09574

==================================

For more data science resources:
✓ https://xn--r1a.website/DataScienceT

#TuringTest #GUIAgents #AIHumanization #MobileAI #AISecurity

🔥 MAI-UI Technical Report: Real-World Centric Foundation GUI Agents

💡 The paper addresses the difficulty of moving graphical user interface GUI agents from research prototypes to practical, large‑scale applications. The authors identify four major obstacles: 1 current agents do not allow direct interaction with users, 2 they are limited to operating only within a GUI, 3 there is no established architecture for deploying agents in real environments, and 4 existing agents perform poorly when the number of parallel environments grows.

To overcome these problems the authors introduce a system called MAI‑UI. MAI‑UI is built as a single evolving data pipeline that continuously expands the information used for navigation. The pipeline adds user interaction data and calls to a multi‑component processing module, incorporates a device‑cloud collaboration layer that routes execution based on task state, and includes an online reinforcement‑learning framework that applies advanced optimization to scale both the number of parallel environments and the length of each task. The design is intended to work for a wide range of model sizes, from two billion to two hundred thirty‑five billion parameters, and to support both desktop and mobile navigation.

The experimental results show that MAI‑UI establishes a new state of the art across several standard GUI benchmarks. On the Screen Spot Pro benchmark it achieves a success rate of seventy‑three point five percent, on the MMBench GUI L2 benchmark ninety‑one point three percent, on the OS World G benchmark seventy‑zero point nine percent, and on the UI Vision benchmark forty‑nine point two percent, all of which exceed the previous best results from Gemini‑3‑Pro and Seed‑1‑8. For mobile GUI navigation the system reaches a new state of the art of seventy‑six point seven percent on Android World, surpassing UI‑Tars‑2, Gemini‑2‑5‑Pro and Seed‑1‑8. On the Mobile World benchmark MAI‑UI obtains a success rate of forty‑one point seven percent, outperforming end‑to‑end GUI models and competing favorably with the Gemini‑3‑Pro agent framework.

Additional online reinforcement‑learning experiments demonstrate that scaling the number of parallel environments from thirty‑two to five hundred twelve yields an improvement of five point two in performance, while increasing the environment step budget from fifteen to fifty adds another four point three. The device‑cloud collaboration component improves on‑device performance by thirty‑three percent, cuts the number of cloud model calls by more than forty percent, and preserves user privacy.

Overall the contribution of the paper is a unified methodology that enables large‑scale, privacy‑preserving, and highly effective GUI agents for both desktop and mobile platforms, and it provides empirical evidence that the approach outperforms existing state‑of‑the‑art systems across multiple benchmarks.

📅 Published on Dec 26, 2025

🔗 Links:
• GitHub: https://github.com/huggingface
• arXiv: https://arxiv.org/abs/2512.22047
• PDF: https://arxiv.org/pdf/2512.22047

🤖 Models citing this paper:
• https://huggingface.co/Tongyi-MAI/MAI-UI-8B
• https://huggingface.co/mlx-community/MAI-UI-8B-bf16
• https://huggingface.co/mlx-community/MAI-UI-8B-6bit

━━━━━━━━━━━━━━━━━━━━━━━━
📢 By: https://xn--r1a.website/PaperNexus

#GUIAgents #HumanComputerInteraction #ScalableAI #AgentArchitecture #RealWorldDeployment