Edsger: A Handwritten Clojure REPL for reMarkable 2

Overview

Edsger is a handwritten Read-Eval-Print Loop (REPL) designed for the reMarkable 2 tablet. It allows users to write Clojure code by hand on the e-paper display and receive the evaluated results back on the screen. The project serves as a functional experiment in "digital executable paper," bridging the gap between traditional handwriting and live programming environments.

Technical Architecture

The system operates by capturing handwritten input from the reMarkable 2 and processing it through a pipeline of transcription and evaluation. Based on the project's architecture, the workflow involves:

  1. Handwriting Capture: The user writes code directly on the reMarkable 2 screen.
  2. Transcription: The handwritten text is converted into machine-readable code. The project utilizes Claude (an LLM) for the transcription process.
  3. Evaluation: The transcribed Clojure code is sent to a REPL for execution.
  4. Rendering: The result of the evaluation is rendered back onto the e-paper display of the reMarkable 2.

Performance and Latency Challenges

A primary technical hurdle for Edsger is the latency between the completion of a handwritten stroke and the appearance of the result.

  • Processing Delay: Observers noted a delay of approximately 14 seconds between the pen stopping and the result appearing. This latency is attributed to the cumulative time required for the reMarkable's internal processing, the Claude transcription API call, and the REPL startup or processing time.
  • Display Constraints: The reMarkable 2 uses e-paper technology, which requires multiple frames to change pixel states (white to black or vice versa). This makes real-time updates challenging, as the display driver must handle specific waveform files (.wbf) to manage pixel transitions.

Community Insights and Alternatives

Technical discussions surrounding the project highlight several ways to optimize the performance and expand the functionality of the reMarkable 2:

Local OCR Alternatives

To reduce the latency caused by cloud-based LLM transcription, community members suggested using local OCR models. Specifically, PaddlePaddle OCR was recommended as a faster, local alternative to Sonnet calls for handwriting conversion.

Display Driver Optimizations

Driving the reMarkable 2 display directly is complex due to the OEM firmware's requirements. Discussion points included:

  • Xochitl: The most common approach is hooking into xochitl (the system's user-facing application), though this is brittle and often breaks during software updates.
  • Waved: A C++ library that allows direct display driving via a sane API, though it is less frequently updated.
  • Direct Framebuffer Access: Using the framebuffer directly (e.g., via remarkable2-framebuffer) can provide more instant updates, which could potentially be paired with speculative decoding to mask latency.

Architectural Suggestions

Some users suggested evolving the project into a Jupyter-like notebook format. In this model, a canonical representation of the session would be stored as a notebook containing handwritten blocks, their interpretations, and the resulting output, which is then rendered back to the device as a PDF or via a custom Qt application.

"The RM line is very hackable, and I still wish it were even easier, the hardware just makes you think of so many possibilities."

Conclusion

While currently an experimental tool with significant latency, Edsger demonstrates the possibility of executing handwritten code on e-paper hardware. It leverages the reMarkable 2's hackability and Clojure's powerful REPL to explore the intersection of handwriting and software development.

Sources