oxylabs/AI-Search-py

What it solves

AI-Search simplifies the process of retrieving and extracting information from the web for developers. It removes the need to build custom search scrapers from scratch by providing a unified way to search the web using natural language and optionally scrape the content of the resulting pages.

How it works

The tool operates as an intelligent search agent that integrates into Python applications via an API. Users provide a search query in plain English, specify the number of results they want (up to 50), and can toggle whether the agent should scrape the full content of the pages or just return URLs and titles. To handle modern, JavaScript-heavy websites, the tool includes a JavaScript rendering option to ensure data is extracted correctly from dynamic content.

Who it’s for

It is designed for developers who need to automate data discovery and collection tasks, such as gathering latest news, researching competitors, or aggregating industry updates.

Highlights

• Natural Language Queries: Search the web using plain English.
• Dynamic Content Handling: Supports JavaScript rendering to extract data from Single Page Applications.
• Content Extraction: Optionally scrapes and returns markdown content from search results.
• Localized Search: Ability to localize search results using the geo_location parameter with ISO 2-letter country codes or coordinates.