grobidOrg/grobid-client-python

grobid-client-python – A Python wrapper for the GROBID document‑parsing service

What it is – A thin, well‑typed Python library (and accompanying CLI) that talks to a running GROBID server via its REST API. GROBID is a machine‑learning‑based tool that extracts structured information (metadata, references, full‑text, figures, tables, etc.) from scientific PDFs and patents. This client adds convenience features such as:

Concurrent processing – send many files in parallel (default pool size = 10, configurable).
Flexible inputs – single PDFs, directories, glob patterns, zip/tar archives, or even s3:// objects (range‑streamed, no full download).
Multiple output formats – raw TEI XML, a CORD‑19‑style JSON representation, or clean Markdown.
Optional enrichments – PDF coordinate extraction, layout‑aware sentence segmentation, citation consolidation, ID generation, etc.
Configurable – server URL, time‑outs, batch size, logging, and many processing flags can be set via a JSON config file or command‑line options.

How you use it

CLI – grobid_client --input ./pdfs --output ./out processFulltextDocument (or any of the other services listed in the README).
Library – instantiate GrobidClient and call client.process(service, input_path, output_path, …) with the same flags you would use on the command line.

Why it matters – GROBID’s models are trained to understand the structure of scholarly articles and patents, a classic NLP/AI task. By handling the plumbing (parallel HTTP calls, archive streaming, S3 access, format conversion) this client lets researchers and developers focus on downstream analysis rather than on low‑level data wrangling.

Key Features (from the README)

Concurrent processing of many documents.
Archive & S3 streaming – never fully decompress or download large collections.
JSON & Markdown conversion of the TEI XML output, matching the CORD‑19 schema.
Type‑hints and a py.typed marker for static‑type checking.
Optional coordinate extraction (--tei_coordinates) and sentence segmentation (--segment_sentences).
Command‑line and library interfaces with identical option sets.
Configurable via JSON (server URL, batch size, time‑outs, logging, etc.).

Typical Workflow

Run a GROBID server (local Docker image, local install, or a hosted demo).
Install the client: pip install grobid-client-python (add [s3] extra if you need S3 support).
Process PDFs – either with the CLI or from Python code, optionally requesting JSON/Markdown output.
Consume the results – the JSON format provides a ready‑to‑use, structured representation of the article’s metadata, body text, figures, tables, and citations.

When to Choose This Tool

You already have GROBID running and need a Pythonic way to batch‑process large corpora of PDFs.
You want to stream archives or S3 objects without exhausting local disk.
You need the extracted data in a machine‑readable JSON format for downstream NLP pipelines.
You prefer a single package that handles both the HTTP calls and the TEI‑to‑JSON/Markdown conversion.

Limitations Mentioned

Citation consolidation (which queries external services like CrossRef) can be slow and requires a higher client timeout (≥ 120 s).
Windows support is only via Docker containers.
The client assumes a reachable GROBID server; it does not include the extraction models itself.

Bottom line – grobid-client-python is a practical, production‑ready helper for anyone using GROBID’s AI‑driven document parsing capabilities, turning raw PDFs into structured, searchable data with minimal effort.