CaptainYifei/fake-news-detector

📚 Project Overview

AI 虚假新闻检测器 – a Streamlit‑based web app that lets users verify the factuality of news articles. It chains together large language models (LLMs) and semantic‑embedding models to (1) extract core claims from a piece of text, (2) rewrite those claims into search queries, (3) retrieve web evidence, and (4) decompose and evaluate each claim step‑by‑step. The system works in Chinese, English, Japanese and Korean and can run on any OpenAI‑compatible model service – from local Ollama models to OpenAI’s GPT.

✨ Core Features

Feature	What it does
Multi‑language support	Auto‑detects input language (zh, en, ja, ko) and can output in any of those languages. UI is bilingual.
Pluggable model providers	Works with Ollama (default Qwen2.5 3B + BGE‑M3 embedding), LM Studio, OpenAI, or any custom API that follows the OpenAI schema.
Three‑node fact‑checking pipeline	1️⃣ Claim extraction – LLM pulls the central verifiable statements from the article. 2️⃣ Query rewriting & evidence retrieval – LLM creates multiple search queries; DuckDuckGo (or SearXNG/Parallel) returns web snippets; BGE‑M3 embeddings rank them, applying similarity and entity‑gate filters. 3️⃣ Multi‑step claim verification – LLM breaks complex claims into atomic statements, matches them with the filtered evidence, and produces a final verdict.
Evidence gating	Configurable `top_k`, similarity threshold, and entity‑hit requirement ensure only high‑quality web snippets reach the verification step.
History & PDF reports	All checks are stored per user; results can be exported as a nicely formatted PDF report.
User system	Multiple users can log in and keep separate histories.
REST API	The same pipeline is exposed via a lightweight FastAPI‑style endpoint (`/check`).

🛠️ Getting Started

Prerequisites

Python 3.12+
Ollama (recommended) or any service exposing an OpenAI‑compatible API
Internet access for DuckDuckGo (default) – optional local SearXNG or Parallel search backend

Installation Steps

# 1. Clone the repo
git clone https://github.com/CaptainYifei/fake-news-detector.git
cd fake-news-detector

# 2. Install Python dependencies
pip install -r requirements.txt

# 3. (Optional) Install Ollama and pull the default models
curl -fsSL https://ollama.com/install.sh | sh
ollama pull qwen2.5:3b
ollama pull bge-m3:latest

Run the web UI

streamlit run app.py

Open http://localhost:8501 in a browser.

Run the API server

python api.py   # starts a server on port 8080 by default

You can then POST JSON to http://localhost:8080/check (see the README for the exact payload).

⚙️ Configuration

All tunables live in model_config.json:

Providers – define base URL and model list for each service (e.g., Ollama, OpenAI).
Defaults – pick the default LLM, embedding model, search engine, output language, and evidence‑filter parameters (evidence_top_k, evidence_min_similarity, evidence_entity_gate, evidence_ranker_multiplier).
Search providers – switch between duckduckgo, searxng, or parallel without code changes.

📂 Project Structure

fake-news-detector/
├─ app.py                 # Streamlit entry point (UI)
├─ api.py                 # FastAPI‑style REST endpoint
├─ fact_checker.py        # Orchestrates the three‑node pipeline
├─ fact_checking/         # Implementation of each node
├─ model_manager.py       # Loads `model_config.json` and creates client objects
├─ auth.py                # Simple user authentication
├─ db_utils.py            # SQLite (or similar) persistence for histories
├─ pdf_export.py          # Generates PDF reports from results
├─ model_config.json      # Central config file
├─ requirements*.txt      # Core + optional Parallel‑search deps
└─ docs/ …                # Usage guide, API docs, screenshots

🚀 How It Works (High‑level Flow)

Claim Extraction – The chosen LLM reads the article and outputs a list of concise, verifiable statements plus key entities.
Query Rewriting & Retrieval – For each claim the LLM creates several search queries. The queries are sent to the selected search backend; returned snippets are embedded with BGE‑M3, ranked, filtered by similarity (≥0.45 by default) and by whether they mention the claim’s entities, then the top K (default 8) are kept.
Multi‑step Verification – The LLM breaks each claim into atomic sub‑claims, asks the evidence snippets for answers, aggregates the answers, and finally labels the original claim as TRUE, FALSE, or UNVERIFIABLE.

📖 Documentation & Help

Usage guide – docs/usage.md (step‑by‑step UI walkthrough).
API reference – docs/api_doc.html.
Issue tracker – https://github.com/CaptainYifei/fake-news-detector/issues for bugs and feature requests.

📜 License

MIT – free to use, modify, and distribute.

⭐ Quick Takeaway

The AI Fake News Detector is a ready‑to‑run, locally‑hosted fact‑checking system that leverages modern LLMs and embedding models to automate claim extraction, evidence gathering, and verification across multiple languages. It’s especially useful for developers or researchers who want an end‑to‑end pipeline without relying on proprietary cloud services.