Postgres LISTEN/NOTIFY Scalability Analysis
Postgres LISTEN/NOTIFY Scalability Analysis
Postgres LISTEN/NOTIFY can scale to 60k messages per second
Postgres LISTEN/NOTIFY is capable of scaling to 60,000 messages per second, challenging the common industry perception that it is unsuitable for high-throughput production environments. While often dismissed as a tool for low-volume signaling, benchmarks demonstrate that with proper optimization—specifically batching commands to amortize per-request costs—the throughput ceiling is significantly higher than previously assumed.
Performance Benchmarks and Hardware Constraints
Recent testing indicates that LISTEN/NOTIFY can achieve 60k messages per second on high-end hardware. However, the context of the hardware used in these experiments is critical for interpreting the results:
- Hardware Specs: Benchmarks were conducted on a server with 96 cores and 384 GB of RAM.
- Vertical Scaling: The performance gains are a result of vertical scaling. While 60k is a substantial number for many applications, it may still be considered low relative to the raw compute power of a 96-core machine.
- Batching: Throughput increases significantly when clients batch commands, reducing the overhead of individual requests.
Technical Limitations and Trade-offs
Despite the ability to scale throughput, LISTEN/NOTIFY has inherent architectural limitations that may make it unsuitable for certain use cases:
Payload Size Limits
There is a hard limit of 8,000 bytes for notification data. If the event data exceeds this limit, developers must store the actual payload in a database row and use the notification only to transmit the row ID.
Global Queue Contention
Historically, NOTIFY has relied on a global queue. This architecture can lead to lock contention and, in some older versions of Postgres, a single slow reader on one channel could potentially block writes to all other channels.
Reliability and Message Loss
Some practitioners report that LISTEN/NOTIFY can lose messages under extremely high load, suggesting it should not be used as a guaranteed delivery queue without a secondary fallback mechanism (such as a sequence-numbered table) to track processed offsets.
Industry Perspectives: Simplicity vs. Specialized Brokers
There is a significant debate among engineers regarding when to use LISTEN/NOTIFY versus a dedicated message broker like Redis or SQS.
The Case for Simplicity
Many developers argue that using the database for pub/sub reduces "emotional support infrastructure"—the overhead of managing, deploying, and monitoring an additional service.
"The integration with the rest of the DB, its availability, its not being another service you have to devops, it's definitely not something that should be simply dismissed out of hand as an option."
The Case for Specialized Tools
Conversely, some argue that building complex queue semantics on top of Postgres internals leads to "scary" systems that are difficult for new engineers to maintain and debug.
"The fewer mechanistic responsibilities per core data store, the better in my experience. Unless I need very strong data consistency guarantees, it's always better to use something like SQS, Redis queues, etc."
Practical Implementation Strategies
For those implementing LISTEN/NOTIFY in production, two successful patterns have emerged:
- The Broker Pattern: Use a small number of high-performance connections (e.g., written in Rust) to act as brokers. These brokers maintain a few
LISTENconnections to the database and distribute messages to tens of thousands of end-user subscriptions, drastically reducing the connection load on Postgres. - The Fallback Pattern: Combine
NOTIFYwith a table of events. TheNOTIFYsignal tells the consumer to check the table for new rows, and the consumer tracks its own offset (sequence number) to ensure no messages are missed during downtime or high-load spikes.