PgDog Funding Announcement and Horizontal Scaling for PostgreSQL

PgDog enables horizontal scaling for PostgreSQL via a Rust-based proxy

PgDog is an open-source sharder, connection pooler, and load balancer designed to allow PostgreSQL to scale horizontally to 100 TB+ tables and 1M queries per second. It functions as a proxy layer placed in front of existing PostgreSQL deployments, enabling massive scale without requiring changes to the application code.

Key technical capabilities include:

  • Deployment Flexibility: PgDog can be deployed on-premises, in private cloud accounts, or on laptops via Docker images.
  • Performance: The tool is written in Rust and utilizes multithreaded code to maximize CPU efficiency.
  • Production Proven: The team reports that PgDog is currently serving over 2M queries per second across dozens of deployments and has sharded over 20 TB of data.
  • Ease of Integration: Users can integrate PgDog by pulling the Docker image and updating their DATABASE_URL.

Funding and Strategic Direction

PgDog has raised $5.5M from investors including Basis Set, YC, and Pioneer Fund. This funding provides the startup, a three-person team of infrastructure and application engineers, with several years of runway to refine the product.

While the core product is open source, the company is developing an Enterprise edition specifically designed to simplify operations within AWS, which will include SLA-backed support.

Technical Insights and User Feedback

Community discussion and user reports highlight several specific advantages and pain points regarding PgDog's implementation compared to traditional tools like PgBouncer.

Advantages over traditional poolers

Users have reported that PgDog handles specific PostgreSQL features more effectively than other proxies:

  • LISTEN/NOTIFY: One user noted that PgDog successfully solves the LISTEN/NOTIFY problem on a transaction pooler.
  • Connection Settings: PgDog transparently handles different connection settings (such as statement_timeout) per connection, a feature reportedly missing or difficult in RDS Proxy and PgBouncer.
  • Migration Utility: Users have cited PgDog as being smoother than PgBouncer when used as a proxy during backend migrations (e.g., moving from Heroku to EC2 to RDS).
  • Aurora Integration: The tool includes an "auto mode" to support Aurora failovers.

Technical Limitations and Concerns

Technical critics and users have raised several points regarding the architecture and operational overhead:

  • Sharding Logic: A significant concern was raised regarding the use of modulo distribution (shard_number = hash(data) % num_shards). Critics argue that this approach requires a full resharding of data to replace or add shards, unlike the linear range/virtual shard approach used by systems like Citus.
  • Consistency: Documentation indicates that writes are not fully consistent across the cluster; partial updates may be visible to other clients even if a transaction is committed.
  • $‚$INSERT (SELECT ...): Users have noted that INSERT (SELECT ...) may not be supported, with documentation only mentioning VALUES inserts.
  • Configuration: Some users found the configuration (via .toml files) awkward for Kubernetes environments where multi-tenancy and dynamic database creation are common.
  • Authentication Caching: There have been reports of authentication requests being cached during passthrough, leading to issues when role passwords are changed.

Comparison to Other Distributed SQL Solutions

Industry observers have compared PgDog to other tools such as Vitess (for MySQL), Citus, and Yugabyte. The primary distinction is that PgDog operates as a proxy rather than a core database extension or a full rewrite of the database engine, which allows it to be added to existing PostgreSQL setups without modifying the underlying database core.

Sources