KKKKhazix/AIHOT

AIHOT – An Open‑Source Engine for Building Industry‑Specific AI‑Curated News Sites

What it is

• AIHOT is a full‑stack, Docker‑ready framework that collects raw information from a variety of feeds (RSS, web pages, JSON APIs, X/Twitter accounts, WeChat public accounts, or custom scripts), runs the data through large language models (LLMs) to filter, score, translate and summarise it, then clusters related items into events and ranks them by how many independent sources are talking about them. The result is a daily (and weekly/monthly) news‑digest website that can be re‑branded for any vertical – law, HR, finance, metals, etc.

Why it matters

• The code shows a concrete, end‑to‑end pipeline that combines classic web‑scraping, duplicate detection, LLM‑based relevance scoring, automated Chinese headline/summary generation, and a lightweight clustering‑by‑semantic‑similarity step. All prompts, thresholds and selection logic are stored in the repo, so users can replace the “AI‑news” knowledge base with their own domain expertise without rewriting code.

Key components

Component	Role
Source adapters (industry/sources.json)	Six built‑in feed types (RSS, web list, JSON, X, WeChat, custom push) with three credibility tiers that affect selection thresholds.
Selection pipeline (industry/prompts/, industry/selection.ts)	Pre‑filter → two independent LLM scoring passes → threshold check → deduplication. All prompt texts are version‑controlled.
LLM‑driven writing (industry/prompts/write-*.md)	Generates Chinese titles, answer‑first abstracts, recommendation reasons, and translates foreign‑language full texts.
Clustering & event creation (industry/prompts/group-*.md)	Uses vector similarity of titles/abstracts (fallback to text overlap) to find candidate reports within the last two weeks, then asks an LLM to decide if they describe the same event, a follow‑up, or separate stories.
Hotness scoring	Counts unique sources per event (max once per source per 48 h, half weight after 24 h) to produce a real‑time “hot” list.
Publication engine (site/site.ts)	Generates daily, weekly and monthly digests automatically at configurable times; builds static pages with React‑SSR, Tailwind CSS UI, and provides RSS/JSON/OpenAPI endpoints for agents.
Admin UI (/admin)	Source management, manual scoring corrections, pipeline diagnostics, budget throttling for paid LLM services, and alert logs.

How to run it

1. Clone the template, run node scripts/init-env.ts with an OpenAI‑compatible API key (DeepSeek, 千问, 智谱, etc.).
2. docker compose up -d --build builds three services (Fastify API, Next‑style SSR site, PostgreSQL). The first half‑hour processes the demo AI‑news feeds; after that the site is populated at http://localhost:3000.
3. Customize:
   • Replace the demo sources in industry/sources.json with feeds from your target industry.
   • Edit prompts in industry/prompts/ to reflect your domain’s “Know‑How”.
   • Adjust thresholds in industry/selection.ts and branding in site/brand/.
   • Re‑run the pipeline (npm run typecheck && npm test && node scripts/smoke.ts).

Who might use it

• Companies or newsletters that need a continuously updated, AI‑filtered news digest for a specific sector.
• Researchers building datasets of curated, LLM‑filtered articles.
• Developers building LLM‑aware agents that consume a clean, structured news feed via the provided RSS/JSON/API endpoints.

Tech stack

• Runtime: Node.js 24, TypeScript
• Web: Fastify (API), React Router with server‑side rendering, Tailwind CSS
• Database: PostgreSQL 17 + pg‑boss job queue
• Containerisation: Docker Compose
• LLM integration: Any OpenAI‑compatible endpoint; prompts are plain markdown files.

License

• MIT for the code. The name “AIHOT” and its logo are excluded from the license, so you must re‑brand when you deploy your own instance.

---

AIHOT is essentially a reusable AI‑powered news‑curation engine. By swapping out the source list and tweaking the prompt files, you can spin up a domain‑specific “hot‑topic” portal without writing the LLM plumbing yourself.