OpenAI Core Dump Epidemiology: Fixing an 18-Year-Old libunwind Bug

OpenAI has resolved a series of inexplicable crashes within Rockset, a C++ based data infrastructure service used for ChatGPT's data plugins and conversation searching. The investigation revealed that what appeared to be a single issue was actually two unrelated bugs: a silent hardware corruption on a specific Azure host and a long-standing race condition in the GNU libunwind library.

Population-Level Diagnosis vs. Case-by-Case Debugging

The turning point in the investigation was the shift from a "doctor" approach (analyzing individual core dumps) to an "epidemiologist" approach (analyzing the entire population of crashes). By building an automated pipeline to analyze a year's worth of production core dumps, OpenAI was able to categorize crashes into distinct clusters based on register state and patterns.

This data-driven approach revealed two separate crash populations:

• Misaligned-stack crashes: These were localized to a single physical host in one region with a clear start date, indicating a hardware failure.
• Return-to-null crashes: These were spread across multiple clusters and regions, suggesting a software-level systemic issue.

Bug 1: Silent Hardware Corruption

One cluster of crashes was traced back to a single physical Azure host where the CPU was performing calculations incorrectly. This caused the stack pointer (%rsp) to become misaligned during execution, leading to a crash upon function return.

Because the corruption was silent and could not be reproduced in a controlled environment, OpenAI mitigated the issue by denylisting the host. To prevent future occurrences, the team improved the fatal signal handler to include register state in logs, allowing for faster detection without requiring full core dumps, and updated the control plane to favor VM reuse over recycling to make bad-node detection easier.

Bug 2: The 18-Year-Old GNU libunwind Race Condition

The second, more complex issue involved a race condition in GNU libunwind, a widely used open-source library for C++ exception unwinding. The bug occurs during the transfer of control to a cleanup handler or catch block.

The Technical Mechanism

When GNU libunwind performs an unwind transfer, it synthesizes a ucontext_t struct on the stack to hold the desired register state. The vulnerability exists in the _Ux86_64_setcontext assembly routine:

1. The routine updates the stack pointer (%rsp) to the new bottom of the active stack.
2. Once %rsp is updated, the ucontext_t struct is no longer part of the active stack or the "red zone" (the 128 bytes below %rsp protected by the kernel).
3. If a signal (such as SIGUSR2) is delivered at this exact moment, the kernel builds the signal frame at %rsp-128, potentially overwriting the ucontext_t memory.
4. If the restored instruction pointer (%rip) is read from this corrupted memory, the program crashes—often returning to NULL.

The Race Window and Probability

The race window is approximately one instruction wide, estimated at roughly 100 picoseconds. While extremely narrow, the crash frequency became operationally visible due to three factors in Rockset's environment:

• High Exception Rate: Rockset uses exceptions for internal ingest backpressure, throwing up to $10^4$ exceptions per second on overloaded hosts.
• High Signal Rate: The coarse_thread_cputime_clock delivers SIGUSR2 signals every few milliseconds of CPU time.
• Increased Stack Usage: A recent update to the SIGUSR2 handler added a call to timer_getoverrun, increasing the amount of stack used by the handler and making it more likely to overwrite the stale ucontext_t memory.

Resolution and Mitigation

OpenAI resolved the libunwind issue by switching from GNU libunwind to libgcc's unwinder, which also offered performance benefits by reducing lock contention on large VMs. Additionally, OpenAI upstreamed a self-contained reproducer and a fix to the GNU libunwind project to resolve the bug for the broader community.