oxylabs/perplexity-scraper

What it solves

This project provides a way to programmatically extract data from Perplexity AI, which lacks a public API for all its features. It allows developers to automate the collection of AI-generated responses, structured metadata, and associated sources without having to manage complex proxy infrastructures or bypass anti-bot systems.

How it works

The tool is an API-based scraper that sends prompts to Perplexity and returns the results in various formats. Users provide a prompt, credentials, and optional parameters (such as geographic location) via a POST request. The system handles the rendering of dynamic content and can deliver the output as parsed JSON, Markdown, and raw HTML, or even as a website PNG or XHR/Fetch requests.

Who it’s for

Developers and researchers who need to automate AI content auditing, track research trends, and analyze SEO performance by monitoring how Perplexity summarizes information and mentions brands.

Highlights

• Multiple Output Formats: Supports JSON, Markdown, HTML, and PNG images of the page.
• Structured Data Extraction: Parses specific elements like the AI's answer, the model used, related queries, and inline products.
• Comprehensive Metadata: Collects additional results from tabs such as images, sources, videos, and hotels.
• Managed Infrastructure: Integrates with a global proxy network to handle IP rotation and anti-bot measures automatically.