666ghj/BettaFish

BettaFish – AI‑powered Multi‑Agent Public‑Opinion (舆情) Analysis Platform

What it is – BettaFish (中文名 “微舆”) is an open‑source system that stitches together several specialized AI agents to automatically crawl, analyse and report on public opinion across dozens of Chinese and international social‑media platforms. Users interact with the system via a simple chat‑style query; the platform then runs a full‑pipeline that:

1. Collects data 24/7 with a distributed web‑crawler (Weibo, 小红书, 抖音, 快手, etc.).
2. Processes multimodal content – text, images and short videos are fed to dedicated “Media” agents that extract captions, OCR, sentiment and other signals.
3. Runs a suite of analysis agents – a Query agent for broad web search, an Insight agent that digs into a private comment database, and a Report agent that assembles the findings.
4. Collaborates via a “forum” engine where agents debate under the guidance of an LLM “moderator”, allowing chain‑of‑thought reasoning and reducing single‑model bias.
5. Generates an interactive HTML (and optional PDF/Markdown) report with charts, summaries and source excerpts.

---

Core Components

Component	Role	Key Tech
QueryEngine	Broad news & web search across 30+ sources	LLM‑driven search nodes, custom tool wrappers
MediaEngine	Multimodal analysis of videos, images, short‑form clips	Vision‑LLM adapters, OCR, audio‑to‑text pipelines
InsightEngine	Deep mining of a private comment database (SQLAlchemy async)	Fine‑tuned sentiment models (BERT‑LoRA, Qwen‑3), keyword optimisation
ReportEngine	Multi‑round report generation, template selection, layout, PDF export	Markdown template engine, IR (intermediate representation) schema, WeasyPrint for PDF
ForumEngine	Coordination layer where agents post, read and react to each other	LLM “host” that acts as a debate moderator, looped reflection cycles
MindSpider	Stand‑alone social‑media crawler used by the agents	Playwright‑driven browser automation, topic extraction, deep sentiment crawling

---

Highlights (the “六大优势” from the README)

1. AI‑driven 24/7 monitoring – a crawler cluster continuously harvests data from >10 major platforms.
2. Hybrid analysis engine – combines five purpose‑built agents, fine‑tuned models and statistical methods.
3. Strong multimodal capability – can parse short videos and extract structured cards (weather, stock, calendar, etc.).
4. Forum‑style collaborative reasoning – agents debate under a LLM moderator, producing richer, less homogeneous insights.
5. Private‑public data fusion – secure API lets you inject your own business databases for combined external‑internal analysis.
6. Lightweight, extensible Python design – modular packages, one‑click Docker deployment, easy to replace or extend any agent.

---

Quick‑Start (Docker)

# 1. Copy env template and edit credentials
cp .env.example .env
# 2. Pull and start all services
docker compose up -d
# 3. Open the UI
open http://localhost:5000

If you prefer a local run, the README provides a Conda/uv environment setup, Playwright browser driver installation, and optional PDF‑export dependencies.

---

Typical Use‑Case Flow

1. Ask a question – e.g., “Analyse the reputation of Wuhan University over the past month.”
2. Agents launch in parallel – Query, Media and Insight agents start gathering data.
3. Forum loop – The agents exchange findings, the moderator steers deeper searches, and reflections iterate until convergence.
4. Report synthesis – The ReportEngine selects a markdown template, plans section length, validates JSON IR, and renders an interactive HTML report (PDF optional).
5. Result – A polished, citation‑rich report is saved under final_reports/ and can be downloaded or further customized.

---

Extensibility

• Swap models – Any OpenAI‑compatible API can replace the default LLMs; fine‑tuned sentiment models are already included.
• Add domains – By editing the agent‑specific prompt files (prompts/) and tool configurations, the same pipeline can be repurposed for finance, market research, product feedback, etc.
• Custom data sources – Plug in additional crawlers or database connectors; the system will automatically incorporate them into the forum discussion.

---

License & Community

• License: GPL‑2.0
• Contributions: Guidelines in CONTRIBUTING.md; active discussion threads on the Chinese “L站” forum (links provided in the README).
• Sponsors: Inferera (LLM API), Anspire Open (search/file‑parse services), APIMart (image/video generation).

---

Bottom line: BettaFish is a full‑stack, multi‑agent framework for automated, multimodal public‑opinion analysis, packaged as a Docker‑ready Python project with a web UI, modular agents, and a sophisticated collaborative reasoning engine.