Concetti Chiave
MobileGPT introduces an innovative LLM-based mobile task automator with human-like memory, enhancing task automation efficiency and adaptability.
Sintesi
The content discusses the development of MobileGPT, an LLM-based mobile task automator that mimics human cognitive processes for efficient task automation. It introduces the concept of Explore-Select-Derive-Recall phases, highlighting the system's ability to learn and adapt tasks with high accuracy and reduced latency. The paper outlines the structure of MobileGPT's memory, its hierarchical organization, and the mechanisms for self-correction and human-in-the-loop task repair. Evaluation results demonstrate MobileGPT's superior performance in task completion rates, efficiency, and accuracy compared to baseline systems.
Abstract:
Introduction of MobileGPT, an LLM-based mobile task automator.
Description of Explore-Select-Derive-Recall phases.
Emphasis on learning tasks efficiently with high accuracy.
Introduction:
Overview of mobile app automation demand.
Comparison of different automation methods.
Limitations of existing approaches.
System Overview:
Description of system workflow: Explore-Select-Derive phases.
Illustration of how MobileGPT operates similarly to human cognitive learning processes.
Accurate and Reliable Task Execution:
Explanation of prompting mobile screens to LLM.
Detailed breakdown of Explore, Select, Derive phases.
Dual Strategy Failure Handling:
Self-correcting through feedback generation.
Human-in-the-loop (HITL) task repair mechanism explained.
Hierarchical App Memory:
Memory structure organization in a transition graph format.
Sub-task based Screen classification:
Methodology for screen classification based on sub-tasks requirements.
Flexible Task Recall:
Attribute-based action adaptation process detailed.
In-context action adaptation using few-shot learning capability discussed.
Implementation:
Utilization of multiple LLM agents for different operational requirements.
App launch process described for efficient execution.
Evaluation:
Dataset creation rationale explained.
Experimental setup details provided for evaluation stages (cold-start, warm-start).
Efficiency analysis comparing latency and cost between systems.
Accuracy assessment including task completion rates and step accuracy breakdown.
Statistiche
MobileGPTは、新しいLLMベースのモバイルタスクオートメーターを導入します。
MobileGPTは、Explore、Select、Deriveフェーズを備えた革新的なLLMベースのモバイルタスクオートメーターを紹介します。
Citazioni
"MobileGPT demonstrates higher step accuracy compared to the GPT4-baseline."
"MobileGPT exhibits near-perfect accuracy during warm-starts."