Pulpie: Pareto-Optimal Models for Cleaning the Web

Pulpie achieves SOTA extraction quality at 20x lower cost

Pulpie is a family of Pareto-optimal encoder models designed to extract main content from HTML pages by labeling blocks as either content or boilerplate. By moving from a decoder-based architecture to an encoder-based one, Pulpie matches the extraction quality of leading models like Dripper while drastically reducing computational overhead. For example, the smallest model, pulpie-orange-small (210M parameters), processes 13.7 pages per second on an NVIDIA L4 GPU, compared to Dripper's 0.68 pages per second.

This efficiency results in massive cost savings at scale: cleaning 1 billion pages costs approximately $7,900 with Pulpie Small on an L4 instance, compared to $159,000 with Dripper.

The Impact of Clean Web Data on LLM Performance

High-quality web extraction is critical for both the pre-training and inference phases of large language models (LLMs).

Pre-training Gains

Research by Ma et al. (2025) demonstrates that model-based parsing significantly outperforms heuristic-based extraction. A model trained on a corpus extracted via a model-based parser scored 1.08 percentage points higher in average accuracy across 13 benchmarks than one trained on heuristic-extracted data. Notably, this approach outperformed models trained on heavily filtered corpora like FineWeb and RefinedWeb, proving that the quality of the initial extractor is a primary lever for model performance.

Inference Accuracy

Noise in the input context during inference can derail model answers. As shown by Shi et al. (2023), a single irrelevant passage can significantly reduce accuracy. Removing boilerplate—which typically makes up 70% of a standard HTML page—ensures models are more accurate and efficient.

Heuristics vs. Model-Based Extraction

Heuristic extractors (e.g., Trafilatura, Readability) rely on surface signals like tag density and DOM structure. While fast, they often struggle with structured content. Model-based extractors use transformers to read the content, leading to significantly higher preservation of complex elements:

Content Trafilatura (heuristic) Model-based
Code blocks 0.13 0.91
Formulas 0.61 0.94

Architectural Shift: From Decoders to Encoders

Pulpie's performance gains stem from its transition from a bandwidth-bound decoder to a compute-bound encoder.

  • Reading Extractors (Decoders): Models like Dripper generate labels one token at a time. This requires reading the full model from GPU memory for every single token, making speed dependent on memory bandwidth.
  • Pulpie (Encoders): Pulpie uses an encoder architecture that labels every HTML block in a single forward pass. This shifts the bottleneck to compute (dense matrix multiplication), which is significantly more efficient on modern GPUs.

This architectural difference is most pronounced on lower-end GPUs. On an NVIDIA L4, the throughput gap between Pulpie and Dripper is 20x, whereas on an A100, it is 7.1x. This is because the L4 has a much lower memory bandwidth ratio relative to its compute power compared to the A100.

Model Family and Performance Benchmarks

Pulpie was developed by distilling a 2.1B parameter teacher model (fine-tuned from EuroBERT) into two smaller student models.

Quality Benchmarks (ROUGE-5 F1)

Model Parameters ROUGE-5 F1 Empty Pages
Pulpie Orange Large (Teacher) 2.1B 0.873 21
Dripper 0.6B 0.864 135
Pulpie Orange Base 610M 0.863 36
Pulpie Orange Small 210M 0.862 45
magic-html - 0.700 384
Trafilatura - 0.619 16

Pulpie Orange Small matches Dripper's quality while being one-third the size. Additionally, Pulpie avoids the context window failures that plague Dripper; while Dripper failed on 135 pages (mostly due to 32k-token limits), Pulpie packs blocks into 8,192-token chunks, ensuring no page is too long to process.

Throughput and Cost

On an NVIDIA L4 GPU, the throughput and cost for 1 billion pages are as follows:

Model Pages/sec Cost / 1B Pages
Pulpie Orange Small 13.7 ~$7,900
Pulpie Orange Base 3.9 ~$28,000
Pulpie Orange Large 1.3 ~$83,000
Dripper 0.68 ~$159,000

Implementation and Usage

Pulpie is open-source and available via pip install pulpie. The pipeline follows four stages: simplifying HTML, chunking blocks into 8,192-token segments, classifying blocks as content or boilerplate, and returning the result as HTML or Markdown.

from pulpie import Extractor

extractor = Extractor() # defaults to Pulpie Orange Small
result = extractor.extract(html)

print(result.markdown) # clean markdown

For bulk processing, the Pipeline class allows for overlapping CPU preprocessing with GPU inference across multiple devices.

Sources

Related

  • Project
    lupinemachines/lupine

    LUPINE is an open‑source GPU‑over‑IP bridge that lets CPU‑only machines (including Macs) use remote NVIDIA GPUs as if they were local. It ships Docker images for a server (runs on the GPU host) and a client (provides CUDA driver shims). Features include multi‑GPU across multiple servers, TCP keep‑alive for long‑running jobs, optional checkpointing, trace logging, and device‑printf forwarding. Use it by running the server container on a GPU box, the client container on the CPU box, and setting `LUPINE_SERVER` to the server address. Ideal for AI/ML workloads that need remote GPU access, though large data transfers can be limited by network latency and authentication/TLS must be added via a proxy.

  • Dispatch
    Soup Enables Fine‑Tuning an 8B Model on a 4 GB Laptop GPU via Layer Streaming

    Soup lets you fine‑tune an 8B parameter LLM on a 4 GB laptop GPU using layer streaming and QLoRA, achieving bit‑exact results with minimal overhead.

  • Dispatch
    colibrì runs GLM‑5.2 744B‑parameter MoE on a 25 GB RAM consumer machine

    colibrì demonstrates that the 744‑billion‑parameter GLM‑5.2 Mixture‑of‑Experts model can be run on a 25 GB‑RAM consumer PC using a pure‑C engine that streams experts from disk.

  • Project
    aiptimizer/TurboOCR

    TurboOCR is a C++/CUDA/TensorRT server that runs PP‑OCRv6 on an NVIDIA GPU to provide ultra‑fast OCR, layout detection, table extraction (HTML) and formula conversion (LaTeX). It processes images or PDFs via HTTP/gRPC, reaching up to 559 images / s on a RTX 5090, with optional model tiers for higher accuracy. The project ships a Docker image that builds TensorRT engines on first start and offers Prometheus metrics and health probes. Licensed under MIT.

  • Dispatch
    pxpipe: Reducing Claude Fable 5 Token Costs by Converting Context to Images

    pxpipe is a local proxy that reduces input token usage for Claude Fable 5 by rendering dense text context as images, achieving an end-to-end cost reduction of 59-70%.