WebArena

What it does

Demo environments

The demos include social forum, online shopping, content management, collaborative software development, map, wiki, calculator, and scratchpad sites.checked 5 Oct 2026

Embedded resources

The environment includes tools and knowledge resources as independent websites.checked 5 Oct 2026

Environment

It recreates websites from four popular categories with functionality and data modeled on real-world equivalents, and includes tools and knowledge resources as separate websites.checked 11 Oct 2026

Evaluation

Evaluation checks information-seeking answers against annotated references and can programmatically check intermediate execution states.checked 11 Oct 2026

Evaluation limit

The repository instructs users to reset the environment after evaluating its 812 examples.checked 11 Oct 2026

Evaluation requirement

The repository says to set up your own WebArena websites for correct evaluation and describes the demo sites as browsing-only.checked 5 Oct 2026

Example tasks

Shown tasks include creating a repository and checking the status and arrival time of an order.checked 5 Oct 2026

External model key

The repository’s evaluation instructions require exporting a valid OpenAI API key before launching evaluation.checked 5 Oct 2026

Included sites

The demos include a social forum, online shopping, content management, collaborative software development, a map, a wiki, a calculator, and a scratchpad.checked 11 Oct 2026

License

The WebArena code repository identifies its license as Apache-2.0.checked 11 Oct 2026

Licensing

The WebArena website states that it is licensed under Creative Commons Attribution-ShareAlike 4.0 International.checked 5 Oct 2026

Model use

The repository's example evaluation command uses GPT-3.5 Turbo and requires an OpenAI API key.checked 11 Oct 2026

Observation formats

Agents can observe a page through its URL and content, represented as a screenshot, HTML DOM tree, or accessibility tree.checked 5 Oct 2026

Observation options

Observations include a page URL and content, represented as a screenshot, HTML DOM tree, or accessibility tree.checked 11 Oct 2026

Purpose

WebArena is a standalone, self-hostable web environment and benchmark for building autonomous agents.checked 11 Oct 2026

Related framework

The repository describes AgentLab as supporting parallel experiments through BrowserGym, benchmark integration including VisualWebArena, unified leaderboard reporting, and improved handling of environment edge cases.checked 11 Oct 2026

Research affiliations

The page lists Carnegie Mellon University and Inspired Cognition as author affiliations.checked 5 Oct 2026

Research provenance

The project page lists Carnegie Mellon University and Inspired Cognition affiliations for the paper's authors.checked 11 Oct 2026

Self-hosting

The project provides Docker resources and a pre-installed Amazon Machine Image for hosting the benchmark websites.checked 11 Oct 2026

Setup

The canonical repository README specifies Python 3.10 or newer and installation of its requirements and Playwright browser.checked 5 Oct 2026

Setup requirement

The repository installation instructions specify Python 3.10 or later and Playwright.checked 11 Oct 2026

Simulated websites

It creates websites across four popular categories with functionality and data modeled on real-world equivalents.checked 5 Oct 2026

Task input

The benchmark evaluates agents that turn high-level natural-language commands into web interactions.checked 5 Oct 2026

Tasks

The benchmark evaluates agents that translate realistic natural-language instructions into concrete web interactions.checked 11 Oct 2026