F3: A Next-Generation Open-Source Data File Format

F3 is a research prototype for a next-generation open-source columnar data file format designed to solve the rigidity of legacy formats like Apache Parquet and ORC. Its primary innovation is the use of embedded WebAssembly (Wasm) decoders, which allow files to remain self-describing and readable even when native decoders are unavailable or when new encoding schemes are introduced.

Solving Format Rigidity with Embedded Wasm Decoders

F3 ensures long-term interoperability and extensibility by embedding the logic required to decode data directly within the file itself. While traditional formats rely on external SDKs or libraries to implement specific encoding schemes, F3 includes Wasm binaries that act as fallback decoders.

This approach provides several technical advantages:

• Forward Compatibility: Developers can implement new encoding schemes without waiting for a global update to the file format specification or the widespread adoption of new SDKs.
• Platform Independence: Because Wasm is a portable binary instruction format, the embedded decoders can run on any platform with a Wasm runtime.
• Minimal Overhead: The project documentation notes that embedding these decoders requires only a few kilobytes of additional storage.

Comparison with Legacy Columnar Formats

F3 is positioned as a successor to "last-generation" formats such as Parquet. The project aims to rectify layout shortcomings inherent in these older formats, which were designed for hardware and workload environments that have since evolved.

By providing a general-purpose API for adding new encoding schemes, F3 intends to obviate the need to create entirely new file formats every time there is a shift in data processing or computing paradigms. The format's structure is defined using FlatBuffers, which provides an efficient way to access serialized data.

Technical Implementation and Research Status

F3 is currently a research prototype associated with a SIGMOD 2026 paper. It is not intended for production use. The current implementation is written in Rust and includes several key components:

• fff-poc: The main proof-of-concept code.
• fff-ude-wasm: The implementation of User-Defined-Encoding (UDE) via Wasm.
• fff-bench: A suite of benchmarks and experiments used to validate the efficacy of the storage layout.

Community Critique and Technical Trade-offs

While the concept of embedded decoders is viewed by some as a "genius" solution to the SDK dependency problem, other technical observers have raised significant concerns regarding security and long-term viability:

Security and Attack Surface

Critics argue that introducing an active execution layer (a VM) into a data format significantly increases the attack surface. Specifically, there are concerns regarding:

• Remote Code Execution (RCE): The ability to execute Wasm binaries from within a file could be exploited for malicious purposes.
• Resource Exhaustion: Maliciously crafted Wasm decoders could be used to launch denial-of-service attacks via resource exhaustion.

Long-term Archival Concerns

For cold storage and long-term archiving (e.g., 10+ years), some argue that a "dead-simple, heavily documented byte specification" is preferable to a binary decoder. The reliance on a Wasm interpreter introduces a dependency on the continued availability and performance of Wasm runtimes in the distant future.

Functional Limitations

Some observers note that a decoder only solves the problem of converting a bitstream; it does not solve the higher-level problem of how the decoded data is used. Concerns were raised regarding whether F3 supports critical performance features like efficient mmap or partial seeking without requiring the execution of the embedded Wasm blob.

"Embedding the decoders in each file requires minimal storage (kilobytes) and ensures compatibility on any platform in case native decoders are unavailable."

"Putting the decoding logic inside an WASM binary introduces an active execution layer into what should be a cold storage."