oxylabs/oxylabs-ai-studio-js

What it solves

This SDK provides a streamlined way for developers to integrate AI-powered web data extraction and navigation tools into JavaScript applications. It eliminates the need to manually handle complex API requests for scraping, crawling, and browsing, providing a structured interface for extracting structured data from the web using AI.

How it works

The SDK acts as a wrapper around the Oxylabs AI Studio API, allowing users to perform several AI-driven tasks:

• AI-Scraper: Extracts data from a specific URL using a provided JSON schema or a natural language prompt to generate a schema.
• AI-Crawler: Discovers and extracts data from multiple pages across a site based on user instructions.
• Browser-Agent: An AI agent that can navigate a website (e.g., clicking and inputting text) and extract data based on a user prompt.
• AI-Search: Performs web searches and can return content from the results using optimized endpoints for either real-time or polling-based responses.
• AI-Map: Maps a website's structure to discover relevant URLs based on keywords or prompts.

Who it’s for

Developers building applications that require automated, AI-driven web data collection, structured data extraction, and autonomous web navigation.

Highlights

• Schema-driven extraction: Supports JSON, CSV, and Markdown output formats with custom JSON schemas for structured data.
• Autonomous navigation: The Browser-Agent can perform actions like clicking and inputting text before extracting data.
• Crawl and Map: Tools for discovering URLs and mapping site structures at scale.
• Geo-location simulation: Ability to simulate requests from specific countries to bypass regional restrictions.
• JavaScript rendering: Support for rendering JavaScript to handle dynamic content on target pages.