LLM + OS
Design prompts for a LLM-based agent framework intended to operate smartphone application
This project is based on AppAgent, a novel LLM-based multimodal agent framework designed to operate smartphone applications.
Figures from left to right demonstrates the process of AppAgent operating an smartphone app.
AppAgent learns to navigate and use new apps, however, it shows weakness when executing complex tasks. We aim at designing a effective prompt and a knowledge base for it to help reduce LLM hallucination.