ZJU-OmniAI/Data-Copilot

📚 What is Data‑Copilot?

Data‑Copilot is an open‑source LLM‑driven assistant that can take a natural‑language request about Chinese financial data and automatically turn it into a full data‑analysis workflow. It discovers the user’s intent, plans a sequence of data‑retrieval, processing, prediction and visualization steps, calls the appropriate APIs, and finally returns a concise textual summary together with charts or tables – all without any manual coding.

The system is built on large language models (GPT‑3.5, Azure‑GPT‑3.5 or Qwen‑72B‑Chat) and uses the Tushare financial data platform for stocks, funds, macro‑economic indicators and company reports. A Gradio web UI lets anyone try the assistant in a browser, and the codebase also ships a command‑line entry point.

🚀 Quick‑Start (from the repo)

# 1️⃣ Clone & create environment
git clone https://github.com/ZJU-OmniAI/Data-Copilot.git
cd Data-Copilot
conda create -n data-copilot python=3.10 -y
conda activate data-copilot
pip install -r requirements.txt

# 2️⃣ Export required keys (Tushare for data, OpenAI/Azure or Alibaba DashScope for the LLM)
export TUSHARE_TOKEN="your‑tushare‑token"
export OPENAI_KEY="sk‑..."   # or DASHSCOPE_API_KEY for Qwen

# 3️⃣ Run a demo request from the command line
python main.py   # edits to `instruction` inside the file change the query

# 4️⃣ Or launch the web UI
python app.py   # then open http://127.0.0.1:7860 in a browser

The web UI asks for the LLM key, lets you type a request (e.g., “Show China’s quarterly GDP growth over the past ten years”), and displays the intermediate workflow, the generated chart/table, and a natural‑language summary.

🛠️ How it works (high‑level architecture)

Component	Role
`main.py`	Orchestrates the five‑stage pipeline: intent detection → task planning → tool execution → visualization → summary.
`tool.py` + `tool_lib/`	Implements atomic “interface tools” such as fetching stock prices, computing net inflows, or calling a prediction model.
`prompt_lib/`	Stores the system prompts and few‑shot examples that guide the LLM at each stage (intent detection, planning, etc.).
LLM callers (`lab_gpt4_call.py`, `lab_llms_call.py`, `lab_llm_local_call.py`)	Thin wrappers that send prompts to OpenAI, Azure‑OpenAI, or Alibaba DashScope (Qwen) and return the model’s response.
Data sources	Tushare API (Chinese market data) and a local static list of A‑share stocks/funds for name matching.
Gradio UI (`app.py`)	Simple web front‑end that forwards the user query to `main.py` and streams the intermediate steps back to the browser.

The LLM acts as both designer (it writes a description of the needed tool calls) and dispatcher (it decides the order, parallelism, or loops). The Python code then executes the generated plan, collects raw results, feeds them back to the model for visualization prompts, and finally asks the model to summarize.

🎬 Demo & Results (as shown in the README)

Supported models – GPT‑3.5 (OpenAI/Azure) and Qwen‑72B‑Chat. All can handle the Chinese financial domain.
Supported data – Chinese stocks, funds, macro‑economic indicators, and company financial statements (via Tushare). The demo includes:
- Net inflow of north‑bound capital for the current year.
- Quarterly GDP growth over the past decade.
- Loop workflow: Q1 net‑profit growth for every constituent of the SSE‑50 index.
The web demo visualizes results as line/bar charts (using a Chinese‑compatible font) and tables, and prints a readable textual summary.

📦 Project layout (quick reference)

Data-Copilot/
├─ main.py                # core pipeline driver
├─ app.py                 # Gradio front‑end
├─ tool.py                # generic tool executor
├─ lab_*.py               # LLM API wrappers (OpenAI, Qwen, local InternLM)
├─ prompt_lib/            # stage‑specific prompts & demos
├─ tool_lib/              # JSON‑like descriptions of each atomic tool
├─ create_tool/           # helper scripts to generate Tushare tool specs
├─ data/                  # static stock/fund lists used for name matching
├─ output/                # where generated tables are saved
└─ docs/flowchart.md      # visual flowchart of the pipeline

📄 License & Citation

The code is released under the MIT License. If you use the system in research or a product, please cite the accompanying arXiv paper:

@article{zhang2023data,
  title={Data-Copilot: Bridging Billions of Data and Humans with Autonomous Workflow},
  author={Zhang, Wenqi and Shen, Yongliang and Tan, Zeqi and Hou, Guiyang and Lu, Weiming and Zhuang, Yueting},
  journal={arXiv preprint arXiv:2306.07209},
  year={2023}
}

🙋‍♀️ Getting help

Open an issue on GitHub or email zhangwenqi@zju.edu.cn for questions about setup, missing Tushare credits, or extending the system to other data domains.