Warma10032/cyber-doctor

赛博医生项目——”赛博华佗“，基于多模态大模型的多功能智能体，一键搭建本地多模态大模型。接入医疗健康相关的知识图谱和知识库后可以进行疾病初诊，病历分析，专业知识问答等功能，成为你的私人医生。赛博华佗项目能帮助实现医疗资源的跨地域传播，让更多人借助大模型改善健康水平。"Cyber ​​Huatuo" - Easy to build a personal doctor agent based on LLM and Knowledge Graph/Knowledge Database.

What it solves

Cyber-doctor is designed to address the imbalance of medical resources, providing users with a multimodal AI assistant for basic disease diagnosis, medical record analysis, and professional health knowledge queries. It aims to lower the barrier to accessing health guidance, particularly for those in underdeveloped regions.

How it works

The project integrates multiple multimodal large language models (LLMs) and uses an AI agent to determine the task type and coordinate between different models. It employs several enhancement techniques to improve accuracy:

  • RAG (Retrieval-Augmented Generation): Uses local knowledge bases (files) and web crawling for real-time information.
  • Knowledge Graph: Integrates Neo4j to provide structured professional knowledge to the LLM.
  • Multimodal Processing: Combines STT (Whisper) and TTS (edge-tts) for voice interactions, and uses multimodal LLMs for image and video generation/recognition.

Who it’s for

  • Individuals concerned about their health who need basic medical advice and record analysis.
  • Developers looking for a framework to build domain-specific expert systems using RAG and Knowledge Graphs.

Highlights

  • Multimodal Integration: Supports text, voice, image, and video inputs and outputs.
  • huile-enhanced RAG: Combines traditional document retrieval, real-time web search, and Neo4j knowledge graph queries.
  • Accessibility: Includes a dedicated voice-only dialogue mode to reduce the learning curve for non-technical users.
  • Document Generation: Can automatically generate formatted Word and PPT documents based on AI-generated content.