论文分享 | 智能体相关研究进展

我们从2024-11-28到2024-12-04的42篇文章中精选出5篇优秀的工作分享给读者。

1. Grid-augmented vision: A simple yet effective approach for enhanced spatial understanding in multi-modal agents
2. A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocation
3. Training Agents with Weakly Supervised Feedback from Large Language Models
4. Practical Performative Policy Learning with Strategic Agents
5. Ponder & Press: Advancing Visual GUI Agent towards General Computer Control

1.Grid-augmented vision: A simple yet effective approach for enhanced spatial understanding in multi-modal agents

Authors: Joongwon Chae, Zhenyu Wang, Peiwu Qin

https://arxiv.org/abs/2411.18270

论文摘要

Recent advances in multimodal models have demonstrated impressive capabilities in object recog nition and scene understanding. However, these models often struggle with precise spatial lo calization- a critical capability for real-world applications. We propose a simple yet effective backbone-free approach that introduces an explicit visual reference system for enhanced spatial understanding. Byoverlaying a 9×9 black grid pattern onto input images, our method provides con sistent spatial anchors without requiring architectural modifications or additional computational overhead. Experiments on the COCO 2017 dataset demonstrate that our grid-based approach achieves significant improvements in localization accuracy, with a 107.4% increase in IoU (from 0.27 to 0.56) and a 194.4% improvement in GIoU (from 0.18 to 0.53) compared to baseline per formance. These results highlight how explicit spatial references can effectively bridge the gap between conceptual understanding and precise localization. Our method’s simplicity and effec tiveness make it particularly valuable for applications requiring accurate spatial reasoning, such as robotic manipulation, medical imaging, and autonomous navigation. The project code is available at https://github.com/triumph123aaa/GRID-AUGMENTED-VISION .

论文简评

这篇论文主要探讨了一种基于网格叠加的方法来增强多模态代理的空间理解能力，并展示了在COCO 2017数据集上的显著提升。该方法简单易用，无需对现有架构进行重大更改，因此可以轻松集成到现有的系统中。实验结果表明，在多个任务指标（如IoU和GIoU）上取得了显著的进步，这些进步对于实际应用具有重要意义。这种简单的方法挑战了关于模型复杂度如何影响空间理解的传统观念，显示出了其潜在的价值。总之，这篇论文提出的网格叠加方法是一种有效的解决方案，它既简洁高效，又能够带来显著的性能改进，为解决空间理解问题提供了新的思路和可能的方向。

2.A Local Information Aggregation based Multi-Agent Reinforcement Learning for Robot Swarm Dynamic Task Allocation

Authors: Yang Lv, Jinlong Lei, Peng Yi

https://arxiv.org/abs/2411.19526

论文摘要

In this paper, we explore how to optimize task allocation for robot swarms in dynamic environments, emphasizing the necessity of formulating robust, flexible, and scalable strategies for robot cooperation. We introduce a novel framework using a decentralized partially observable Markov decision process (Dec_POMDP), specifically designed for distributed robot swarm networks. At the core of our methodology is the Local Information Aggregation Multi-Agent Deep Deterministic Policy Gradient (LIA_MADDPG) algorithm, which merges centralized training with distributed execution (CTDE). During the centralized training phase, a local information aggregation (LIA) module is meticulously designed to gather critical data from neighboring robots, enhancing decision-making efficiency. In the distributed execution phase, a strategy improvement method is proposed to dynamically adjust task allocation based on changing and partially observable environmental conditions. Our empirical evaluations show that the LIA module can be seamlessly integrated into various CTDE-based MARL methods, significantly enhancing their performance. Additionally, by comparing LIA_MADDPG with six conventional reinforcement learning algorithms and a heuristic algorithm, we demonstrate its superior scalability, rapid adaptation to environmental changes, and ability to maintain both stability and convergence speed. These results underscore LIA_MADDPG’s outstanding performance and its potential to significantly improve dynamic task allocation in robot swarms through enhanced local collaboration and adaptive strategy execution.

论文简评

该论文旨在探讨一种分布式动态任务分配框架，在机器人群中使用Local Information Aggregation Multi-Agent Deep Deterministic Policy Gradient（LIA_MADDPG）算法来解决一个实际且重要的问题——机器人群中的动态任务分配。这一方法通过允许机器人共享本地信息，从而提高决策效率。论文提供了丰富的数据支持，并进行了详细的实验分析，以比较和评估了提出的策略与现有算法之间的性能差异。总体来说，这篇论文为机器人领域的研究提供了一个有价值的视角，并展示了分布式智能系统在解决复杂问题时的巨大潜力。

3.Training Agents with Weakly Supervised Feedback from Large Language Models

Authors: Dihong Gong, Pu Lu, Zelong Wang, Meng Zhou, Xiuqiang He

https://arxiv.org/abs/2411.19547

论文摘要

Large Language Models (LLMs) offer a promising basis for creating agents that can tackle complex tasks through iterative environmental interaction. Existing methods either require these agents to mimic expert-provided trajectories or rely on definitive environmental feedback for reinforcement learning, which limits their application to specific scenarios like gaming or code generation. This paper introduces a novel training method for LLM-based agents using weakly supervised signals from a critic LLM, bypassing the need for expert trajectories or definitive feedback. Our agents are trained in an iterative manner, where they initially generate trajectories through environmental interaction. Subsequently, a critic LLM selects a subset of good trajectories, which are then used to update the agents, enabling them to generate improved trajectories in the next iteration. Extensive tests on the API-bank dataset show consistent improvement in our agents' capabilities and comparable performance to GPT-4, despite using open-source models with fewer parameters.

论文简评

这篇论文提出了一种使用弱监督反馈来训练代理的方法，该方法通过环境交互生成轨迹，并由一个critic LLM评估有效的轨迹，以进一步训练代理。作者声称他们的方法在使用较小模型的情况下可以达到与GPT-4相当的性能水平。这一创新研究为大型语言模型驱动的代理提供了更广阔的适用性，同时提出了迭代学习的方法，使代理能够持续改进，增强其适应性。此外，论文还解决了训练代理时依赖专家提供的轨迹的问题，这是一个重要的挑战。总的来说，这篇文章展现了强大的理论基础和实用价值，对于理解和应用大型语言模型驱动的代理具有重要意义。

4.Practical Performative Policy Learning with Strategic Agents

Authors: Qianyi Chen, Ying Chen, Bo Li

https://arxiv.org/abs/2412.01344

论文摘要

This paper studies the performative policy learning problem, where agents adjust their features in response to a released policy to improve their potential outcomes, inducing an endogenous distribution shift. There has been a growing interest in training machine learning models in strategic environments, including strategic classification and performative prediction. However, existing approaches often rely on restrictive parametric assumptions: micro-level utility models in strategic classification and macro-level data distribution maps in performative prediction, severely limiting scalability and generalizability. We approach this problem as a complex causal inference task, relaxing parametric assumptions on both micro-level agent behavior and macro-level data distribution. Leveraging bounded rationality, we uncover a practical low-dimensional structure in distribution shifts and construct an effective mediator in the causal path from the deployed model to the shifted data. We then propose a gradient-based policy optimization algorithm with a differentiable classifier serving as a substitute for the high-dimensional distribution map. Our algorithm efficiently utilizes batch feedback and limited manipulation patterns. Our approach achieves high sample efficiency compared to methods reliant on bandit feedback or zero-order optimization. We also provide theoretical guarantees for algorithmic convergence. Extensive and challenging experiments demonstrate our method’s practical efficacy.

论文简评

该论文聚焦于一个重要的机器学习与经济学问题：执行性政策学习（Policy Learning with Performative Actions）。研究提出了一种创新的框架，它允许代理人在响应策略时修改其特征，导致数据分布的变化。作者通过放松对行为者和数据分布参数的假设，并利用有界理性来实现低维结构的分布变化，提出了一个基于梯度优化的方法。此外，他们还引入了一个基于梯度优化的算法，并提供了理论保证以支持其收敛性。实验结果表明，这种方法不仅具有实践有效性，而且在样本效率方面也优于现有方法。综上所述，这篇论文为解决机器学习中的一个重要问题提供了一种新颖且有效的解决方案，值得进一步研究和应用。

5.Ponder & Press: Advancing Visual GUI Agent towards General Computer Control

Authors: Yiqin Wang, Haoji Zhang, Jingqi Tian, Yansong Tang

https://arxiv.org/abs/2412.01268

论文摘要

Most existing GUI agents typically depend on non-vision inputs like HTML source code or accessibility trees, limiting their flexibility across diverse software environments and platforms. Current multimodal large language models (MLLMs), which excel at using vision to ground real-world objects, offer a potential alternative. However, they often struggle with accurately localizing GUI elements—a critical requirement for effective GUI automation—due to the semantic gap between real-world objects and GUI elements. In this work, we introduce Ponder & Press, a divide-and-conquer framework for general computer control using only visual input. Our approach combines a general-purpose MLLM as an 'interpreter', responsible for translating high-level user instructions into detailed action descriptions, with a GUI-specific MLLM as a 'locator' that precisely locates GUI elements for action placement. By leveraging a purely visual input, our agent offers a versatile, human-like interaction paradigm applicable to a wide range of applications. Ponder & Press locator outperforms existing models by +22.5% on the ScreenSpot GUI grounding benchmark. Both offline and interactive agent benchmarks across various GUI environments—including web pages, desktop software, and mobile UIs—demonstrate that the Ponder & Press framework achieves state-of-the-art performance, highlighting the potential of visual GUI agents. Refer to the project homepage here.

论文简评

这篇关于GUI自动化框架Ponder & Press的研究报告，深入探讨了如何利用视觉输入来改善现有依赖非视觉输入的GUI自动化的局限性。该研究提出了一个基于视觉指令解释的通用机器学习模型（MLLM）以及一个专门针对GUI元素精确定位的GUI特定MLLM。作者声称他们的方法在ScreenSpot等基准测试中取得了显著改进，展示了视觉GUI代理潜在能力的巨大潜力。

核心观点在于，提出的方法有效地解决了现有依赖于非视觉输入的GUI自动化的限制，并且通过采用分治策略，清晰地分离了任务解释与GUI定位的任务，从而可能提高准确性和可扩展性。此外，大量的定量评估证明了这种方法的强大性能，特别是在多个基准测试中的卓越表现，支持了作者对这一发现的认可。

总的来说，这篇论文提供了一个创新的解决方案，旨在克服现有技术面临的挑战，为GUI自动化领域开辟了一片新的天地。它不仅展现了视觉输入的优势，还展示了其在提升GUI自动化效率方面的巨大潜力。

我们欢迎您在评论区中留下宝贵的建议！包括但不限于：

• 可以提出推文中论文简评的不足！
• 可以分享最近更值得推荐的论文并给出理由！

#论文分享 #论文 #人工智能 #ai #算法 #深度学习 #智能体 #大模型 #Agent #LLM

推荐阅读