agents that uses the language to act on behave of another person or group.

Challenges

See Challenges of Language Model Agents

Methods

ReAct

See ReAct

Aguvis

Take the AgentNet dataset, and then tune a vison LM to roll out the rest of the sequence of actions given screenshots as input on top of a Qwen base model.

We can also add on top Chain of Thought to get more thinking as well.

Formulations

OSWorld

A unified task setup and evaluation.

the