Kohulan/DECIMER-Image_Transformer

DECIMER Image Transformer is a deep-learning-based tool designed for automated recognition of chemical structure images. Leveraging transformer architectures, the model converts chemical images into SMILES strings, enabling the digitization of chemical data from scanned documents, literature, and patents.

What it solves

Chemists often encounter chemical structure images in papers, databases, or hand-drawn sketches. Manually transcribing them into machine-readable formats is slow and error-prone. DECIMER automates Optical Chemical Structure Recognition (OCSR) — it takes an image of a molecule and outputs the corresponding SMILES string, the standard text representation used in cheminformatics.

How it works

DECIMER uses a deep learning pipeline with two main parts:

  • Image feature extraction: an EfficientNet-V2 convolutional network reads the image and extracts visual features.
  • SMILES prediction: a transformer model (similar to those used in language tasks) generates the SMILES sequence from those features. The project was trained on a large dataset (over 1 million images) using Google TPUs for speed. It supports both printed and hand-drawn chemical structures — the hand-drawn model is a separate component that can be loaded on demand.

The software is easy to install via pip (pip install decimer) and provides a simple Python interface. The first time you run prediction, the model weights (about 300 MB for the printed-structure model) are downloaded from Zenodo and cached. You can pre-load both models with DECIMER.load_models() to avoid the download later, and you can point to a local model directory to run entirely offline.

Who it's for

  • Chemists and cheminformaticians who need to extract molecule structures from images.
  • Researchers digitizing publications, lab notebooks, or hand-drawn structures.
  • Developers building tools that need to convert chemical images to SMILES.

Highlights

  • Open source (MIT) and available on PyPI.
  • High performance: trained on over 1 million images using TPUs.
  • Two models: one for printed structures, one for hand-drawn structures.
  • Simple Python API: predict_SMILES(image_path) returns a SMILES string.
  • On-demand model download and caching; supports offline use with pre-installed models.
  • Ongoing development by the Steinbeck Group at Friedrich Schiller University Jena.

Related

  • Project
    hongsong-wang/CV_Paper_Portal

    A portal that aggregates abstracts from top computer vision, language, machine learning, and robotics conferences into searchable web pages, helping researchers quickly find papers by keyword across many years and venues.

  • Project
    google-deepmind/representations4d

    DeepMind’s representations4d provides research‑grade video models that learn 4‑D (spatial + temporal) features via masked auto‑encoding, moving‑token attention, recurrent masking, multimodal alignment, and diffusion‑based perception. It ships pretrained checkpoints (up to 22 B parameters), Colab demos, and a small Python library for inference on depth, tracking, segmentation, and text‑prompted multi‑task vision.

  • Project
    ZZZHANG-jx/Recommendations-Document-Image-Processing

    This repository is a carefully organised collection of academic papers, datasets, and benchmark results for document image processing, covering tasks such as shadow removal, dewarping, deblurring, and appearance enhancement. It serves as a reference hub for researchers and developers looking to understand or apply state-of-the-art methods to clean up photographed documents.

  • Project
    yandexdataschool/deep_vision_and_graphics

    A course repository from Yandex School of Data Analysis that teaches deep learning applied to computer vision, with weekly lectures, seminars, and homework covering topics from convolutional networks to diffusion models and neural rendering.