xming521/WeClone

What it solves

WeClone provides an end-to-end pipeline to create a digital avatar that mimics a specific person's chatting style and personality. It solves the problem of manually crafting complex prompts to simulate a persona, instead using actual chat history to fine-tune a Large Language Model (LLM).

How it works

The system follows a four-step process:

1. Data Export & Preprocessing: It imports chat records (currently supporting Telegram) and uses Microsoft Presidio to filter out sensitive private information (like phone numbers and emails) and a custom blocklist to remove unwanted content.
2. Fine-tuning: It uses LLaMA Factory to perform Supervised Fine-Tuning (SFT) on a base model (defaulting to Qwen2.5-VL-7B-Instruct) using methods like LoRA or QLoRA to infuse the model with the user's specific conversational "flavor."
3. Inference: The fine-tuned model is served via an API or a browser-based web chat demo.
4. Deployment: The avatar can be integrated into messaging platforms like Telegram, Discord, Slack, or WeChat via chatbot frameworks such as AstrBot or LangBot.

Who it’s for

Users who want to create a personalized AI version of themselves or others based on real conversation data, as well as researchers exploring persona-based LLM fine-tuning.

Highlights

• End-to-End Workflow: Covers everything from raw data export to final bot deployment.
• Multimodal Support: Supports fine-tuning with image modal data to better capture conversational context.
• Privacy-Focused: Includes built-in PII (Personally Identifiable Information) filtering to secure data before training.
• Flexible Deployment: Compatible with multiple IM platforms and existing LLM bot frameworks.