Riddle: Transforming reMarkable Paper Pro into an AI-Powered Diary

Riddle: Transforming reMarkable Paper Pro into an AI-Powered Diary

Riddle turns the reMarkable Paper Pro into an interactive AI diary

Riddle is an open-source application that transforms the reMarkable Paper Pro into a digital version of "Tom Riddle's diary" from Harry Potter. The system allows users to write with a pen on the e-ink screen; after a brief pause, the application "drinks" the ink (fades the user's handwriting) and generates a handwritten response from an AI that appears stroke-by-stroke on the page.

Technical Architecture and Ink Processing

Handwriting Synthesis and Rendering

Riddle is written in Rust and handles the entire pipeline from pen input to visual output. The system captures raw evdev events with 4096-level pressure sensitivity. To create the realistic effect of an AI "writing" back, the application follows a specific synthesis process:

  1. Rasterization: The response text is rendered using the Dancing Script font.
  2. Skeletonization: The rendered text is processed via Zhang-Suen thinning to create single-pixel pen paths.
  3. Stroke Tracing: These paths are converted into strokes that can be animated.
  4. Animated Replay: The strokes are drawn back onto the screen in a flowing hand.

Display Backends

The project implements two distinct display modes to manage latency and system integration:

  • Windowed (qtfb): Runs as a standard application within the xochitl (AppLoad) environment.
  • Takeover (quill): A high-performance mode that stops the vendor UI (xochitl) and drives the e-ink engine directly via a shim over the proprietary libqsgepaper.so waveform engine. This "takeover" mode provides the lowest possible latency for ink rendering.

AI Integration and the "Oracle"

Vision-Capable LLM Backends

The "spirit" of the diary is powered by a vision-capable Large Language Model (LLM) that reads the user's handwriting by processing a committed page sent as an inline PNG. The system supports two primary integration paths:

  • OpenAI-Compatible APIs: Users can configure the app to communicate with any /chat/completions endpoint, including OpenAI, OpenRouter, Groq, or local servers. It is optimized for vision models like gpt-4o-mini.
  • Pi Integration: If the user is running pi-mono, Riddle uses a resident pi --mode rpc process to reduce latency, paying only the model's inherent response time.

Performance and Latency

The system is designed for streaming responses. The quill begins writing the AI's reply sentence-by-sentence as the model streams the text, rather than waiting for the full response to be generated. On-device testing has shown a first-ink latency of approximately 0.9 to 1.1 seconds.

Installation and System Requirements

Hardware and Software Dependencies

Riddle requires a reMarkable Paper Pro (specifically tested on aarch64, OS 3.26–3.27). The device must be in developer mode with a launcher installed. The project recommends using remagic for the initial setup of developer mode and installation.

Installation Methods

Users can install Riddle through three primary methods:

  1. remagic: A one-command installation process (remagic install riddle).
  2. Prebuilt Bundle: Manually copying a riddle-<version>.zip bundle to the device via SCP.
  3. Building from Source: Cross-compiling from x86_64 using Cargo for the windowed version, or using the reMarkable SDK toolchain for the takeover version to link against the device's proprietary libraries.

User Interface and Gestures

Riddle replaces traditional UI elements with a gesture-based interaction model:

Gesture Action
Rest the pen Triggers the AI to process the page and reply
Flip the marker Erase
Draw a large "?" Summons the built-in guide
Five-finger tap Exits the diary and returns to the system UI
Power button Suspends the device with a "The diary sleeps" message

Sources