jannisborn/paperscraper

A bibliometrics tool for publication metadata (pubmed, arxiv, medrxiv, biorxiv, chemrxiv) and citation analyses

What it solves

Researchers often need to gather publication metadata and full texts from multiple sources (PubMed, arXiv, preprint servers) and track citation metrics, which is tedious and error-prone by hand. Paperscraper provides a single Python interface to automate these tasks, including bulk downloads of preprint archives and citation counting.

How it works

It is a Python package that you install with pip. You can query PubMed, arXiv, or Google Scholar directly. For bioRxiv, medRxiv, and chemRxiv, you first download a local dump of the full archive (JSONL files) and then search locally. The package also provides functions to retrieve PDFs by DOI, get citation counts from Google Scholar or Semantic Scholar, and compute self-citation/reference scores for papers. Keyword searches support Boolean logic via nested lists (AND/OR).

Who it's for

Anyone who works with scientific literature programmatically: researchers, literature-reviewers, and data scientists who want to gather metadata, download papers, or analyze citation patterns.

Highlights

  • Scrapes publication metadata from PubMed, arXiv, bioRxiv, medRxiv, chemRxiv.
  • Downloads full-text PDFs or XML by DOI.
  • Retrieves citation counts and search results from Google Scholar (via SearchApi) and Semantic Scholar.
  • Provides journal impact factor queries.
  • Supports date-bounded scraping and bulk archive downloads for preprint servers.
  • Includes self-citation and self-reference analysis per paper.
  • Keyword search with Boolean (AND/OR) logic.

Related

  • Project
    Belluxx/Aleph

    Aleph is a geographic data capture tool that lets you draw a rectangle or polygon on a map and download everything inside it, including buildings, elevation, satellite imagery, Street View photos, and 3D models. It also provides on-demand data for specific points via a command-line interface.

  • Project
    jerryjliu/docjev

    A document classification and splitting tool that uses OCR and natural-language rules to categorize files and divide large PDF packets into separate documents.

  • Project
    supervisely/supervisely

    Supervisely is a web‑based computer‑vision platform that combines data labeling, model training, inference and collaboration tools. It offers a Python SDK and REST API for automation, and an app ecosystem where developers can create head‑less scripts, interactive UI tools, or labeling‑tool extensions—all deployable with a single click.

  • Project
    cedya77/aiometadata

    AIOMetadata is a Stremio metadata addon that aggregates and enriches movie, series, and anime metadata from multiple sources, giving users full control over catalogs, artwork, and search. It solves the problem of limited default metadata by providing deep customization, anime power, and self-hosting options.