EvanZhouDev/llm.pdf

Run LLMs inside a PDF file.

What it solves

It demonstrates that a Large Language Model (LLM) can be executed entirely within a standalone PDF file, removing the need for external servers or traditional software installations.

How it works

The project uses Emscripten to compile llama.cpp into asm.js, which is then executed within the PDF via a JavaScript injection technique. The LLM model file itself is embedded directly into the PDF using base64 encoding.

Who it’s for

Developers and researchers interested in proof-of-concept experiments in LLM deployment and the creative use of PDF capabilities.

Highlights

  • Runs LLM inference locally within a PDF document.
  • Supports GGUF quantized models.
  • Includes a script to generate custom PDFs with different compatible models.
  • Leverages asm.js and llama.cpp for execution.

Related

  • Project
    jjang-ai/vmlx

    A self-hosted inference server for Apple Silicon that provides OpenAI and Anthropic compatible APIs for running LLMs, VLMs, and image generation models locally.

  • Project
    AgnostiqHQ/covalent

    A Python library that abstracts infrastructure management, allowing AI/ML and scientific research code to run on any cloud or on-prem cluster with a single line of code.

  • Project
    georgian-io/LLM-Finetuning-Toolkit

    A config-based CLI tool for launching and managing LLM fine-tuning experiments, allowing users to run ablation studies across different models, prompts, and optimization techniques.

  • Project
    bacalhau-project/bacalhau

    Bacalhau is a distributed compute orchestration framework that enables "Compute Over Data," allowing users to run jobs directly where large datasets are stored to reduce latency and costs.