WebVoyager

WebVoyager is a Python AI agent that controls a browser on your machine or a Linux server to carry out tasks on live websites. It's a research project for developers studying browser automation. Model decisions depend on OpenAI's cloud API, which receives page observations and requires an API key and internet access; the local browser doesn't provide local model inference.

The agent uses Selenium with Chrome or Chromium. Its visual mode combines screenshots with page text so the model can assess the page's appearance when choosing actions. A text-only mode also supports models without image input. The agent can download PDFs for analysis.

The repository includes website tasks and reference answers for comparing results against defined goals. These cover sites such as Booking and Google Flights, alongside browser tasks drawn from the GAIA validation dataset. Some travel tasks contain dates that need updating before they remain useful as evaluation cases.

WebVoyager saves interaction messages and screenshots, giving researchers a record of the actions behind each answer. Its automated evaluation tool sends the task, agent response and final screenshots to GPT-4V to assess completion. Developers can adapt the agent's prompts for their own experiments.

The code is open source under Apache 2.0. Website changes, pop-ups and variation in OpenAI model responses can affect results, and the project doesn't guarantee the accuracy of generated answers.