McGill-NLP/webllama

What it solves

WebLlama aims to create human-centric web browsing agents that act as assistants rather than replacements for users. It addresses the difficulty of training agents to navigate real-world websites—including booking, shopping, and knowledge lookup—while interacting with users through multi-turn dialogue.

How it works

The project provides a framework for training and evaluating Llama-based agents. Its primary model, Llama-3-8B-Web, is a finetuned version of Meta-Llama-3-8B-Instruct, trained on a curated subset of the WebLINX 1.0 dataset containing over 24,000 instances of web interactions (such as clicks, text input, and dialogue acts).

To deploy these agents, the project integrates with browser automation tools like BrowserGym, Playwright, and Selenium, allowing the model to translate text representations of a web page's state into executable actions.

Who it’s for

Researchers and developers building web-navigating AI agents who need specialized models, training scripts, and standardized benchmarks for real-world website navigation.

Highlights

• High Performance: Llama-3-8B-Web outperforms GPT-4V (zero-shot) by 18% on the WebLINX 1.0 benchmark.
• Human-Centric Design: Focuses on dialogue-guided browsing where the agent can talk to the user.
• Comprehensive Evaluation: Includes benchmarks for testing generalization across new websites, domains, and geographic locations.
• Extensible Ecosystem: Integrates with BrowserGym and provides tools for visualizing evaluation results via a Streamlit app.