BIT-DataLab/Edit-Banana

What it solves

Edit Banana transforms static images of diagrams, flowcharts, and technical schematics into fully editable DrawIO (XML) files. It eliminates the need to manually recreate complex diagrams from non-editable formats by reconstructing the original layout, logical relationships, and text.

How it works

The project uses a multi-stage pipeline to reconstruct diagrams:

1. Segmentation: A fine-tuned SAM 3 (Segment Anything Model 3) identifies and masks individual diagram elements.
2. Text Extraction: The system uses local OCR (Tesseract or PaddleOCR) for text localization and Pix2Text for converting mathematical formulas into LaTeX.
3. VLM Guidance: Multimodal Large Language Models (VLMs) guide the extraction process through multi-round scanning.
4. XML Generation: Spatial data from the segmentation phase and text from the OCR phase are merged to generate a DrawIO-compatible XML file.

Who it’s for

It is designed for researchers, engineers, and students who need to edit existing diagrams found in papers, presentations, or legacy documents without having the original source files.

Highlights

• High-Fidelity Reconstruction: Preserves layout logic, color matching, element hierarchy, and arrow styles (thickness and dashed lines).
• Formula Support: Specifically handles scientific formulas via LaTeX conversion.
• Editable Output: Produces standard DrawIO XML, allowing users to drag, style, and modify elements individually.
• Performance Optimizations: Includes a Global Lock for thread-safe GPU access and an LRU Cache for persisting image embeddings.