karust/openserp

OpenSERP – Open‑source Search‑Engine Results API & CLI

What it is

• A free, self‑hostable service that scrapes the public web‑search pages of Google, Bing, Yandex, Baidu, DuckDuckGo and Ecosia and returns the results in a clean, unified JSON (or markdown/text/NDJSON) format.
• Provides a REST API and a command‑line client, so you can call it from code, from scripts, or directly from a terminal.
• No API keys or per‑search billing – you run it locally (or on your own server) and it works like a private SERP endpoint.

Why it matters for AI/LLM workflows

• LLMs often need up‑to‑date web information. OpenSERP lets you fetch live search results and even the extracted page content in one request, which you can feed to prompting or retrieval‑augmented generation pipelines.
• The same JSON schema works across all six supported engines, simplifying multi‑engine aggregation and fallback strategies.
• A hosted cloud version is also offered, so you can switch between self‑hosted and SaaS without code changes.

Key features

• Dedicated endpoints for each engine (/google/search, /bing/image, …) with a common response schema.
• Megasearch – query several engines at once, merge results, deduplicate URLs, and get clustering information.
• URL extraction – optionally pull the target page, clean it to markdown, and embed it in the search response.
• Supports SERP extras such as AI summaries, answer boxes, “people also ask”, related searches, and image search.
• Output formats: JSON, markdown, plain text, NDJSON.
• Proxy support (HTTP & SOCKS5), caching, resilient mode, and pre‑built Docker images.
• Official SDKs for JavaScript/TypeScript, Python, an MCP server for AI agents, and an n8n node.

Typical use cases

• LLM grounding – retrieve fresh web snippets or full pages to augment model responses.
• SEO / rank tracking – run automated queries and store structured results for analysis.
• Multi‑engine research – combine Google, Bing and others in a single call to improve coverage.
• Automation – integrate into CI pipelines, bots, or no‑code tools (n8n) that need search data.

Getting started

1. Docker (quickest)

   docker run --rm -p 127.0.0.1:7000:7000 karust/openserp:latest serve -a 0.0.0.0 -p 7000
   

2. Go install

   go install github.com/karust/openserp@latest
   openserp search duckduckgo "open source serp api" --format markdown
   

3. From source

   git clone https://github.com/karust/openserp.git
   cd openserp
   go build -o openserp .
   ./openserp serve
   

4. First API call (megasearch, fast fallback mode):

   curl "http://127.0.0.1:7000/mega/search?engines=bing,google&text=golang+vs+rust&extract=1&mode=any"
   

Example response (truncated) shows the query metadata, an array of results with fields like title, url, snippet, and an optional extracted markdown block for the page content.

SDK usage (Python example)

import openserp
client = openserp.OpenSERP(base_url="http://localhost:7000")
resp = client.search(engine="google", text="openserp", limit=5)
print(resp["results"])

Documentation & API explorer

• Swagger UI: http://127.0.0.1:7000/docs
• OpenAPI spec: http://127.0.0.1:7000/openapi.yaml
• Architecture overview: https://openserp.org/docs/architecture/

License

• MIT License – you can freely modify, redistribute, or embed the software.

Community

• Issues & feature requests on GitHub.
• Feedback via email or the Telegram channel.
• Contributions welcomed via the CONTRIBUTING.md guide.

---

OpenSERP gives developers a privacy‑preserving, cost‑free way to add live web search to AI agents, bots, and data pipelines.