Implementing Neural Networks in SQL via Xarray-SQL

Neural Networks as Relational Operations

Implementing a neural network in SQL is possible by treating N-dimensional (Nd) arrays as relational tables and leveraging relational algebra to perform tensor operations. This approach, demonstrated through the Xarray-SQL project, posits that every Nd array can be mapped to a 2D tabular representation where the orthogonal dimensions of the array serve as primary keys.

By redefining array operations in relational terms, complex mathematical processes—including matrix multiplication and automatic differentiation—can be pushed down into the database layer rather than being post-processed in external numerical code.

Matrix Multiplication and Relational Algebra

Matrix multiplication (matmul) in a relational model is expressed as a combination of joins and aggregations. Specifically, matmul is implemented as SUM(val * val) ... JOIN .. GROUP BY in SQL.

This implementation has a direct parallel to einsum (Einstein summation) notation. The project reveals that common operations in geospatial and climate sciences, such as regridding, are essentially sparse matrix-vector products that can be executed using this relational logic.

Autograd and Calculus in the Database

To move beyond linear algebra and implement neural networks, the system requires the ability to perform calculus. Xarray-SQL implements autograd (automatic differentiation) on top of the DataFusion visitor pattern, drawing inspiration from JAX's implementation.

In this simplified array model, the system focuses on partial differentiation on the diagonal of the Jacobian. This reduces grad(), jvp (Jacobian-vector product), and vjp (vector-Jacobian product) to row-wise operations, enabling the database to handle the gradients necessary for training neural networks.

The Case for Relational AI Infrastructure

Using SQL as a declarative language for neural networks offers a significant architectural advantage: the separation of the logical layer from the physical layer.

If a neural network is treated as a series of relations, the database's logical plan can be used to optimize dataflow. This potentially allows for more efficient distribution of work across massive GPU clusters (1,000+ GPUs) by utilizing a global logical plan rather than manually managing data movement between devices.

Technical Perspectives and Counterpoints

While the implementation is a proof-of-concept, the community has noted several theoretical and practical considerations regarding this approach:

Theoretical Equivalence

Some developers note that einsum and database joins are mathematically identical, differing only in their semirings (real numbers for einsum versus booleans for databases). This suggests that relational algebra is a viable intermediate representation (IR) for tensor programs, allowing database optimizers to reason about machine learning workloads.

Existing Precedents

Relational ML is not entirely new. Projects like Apache MADlib have previously integrated neural networks into SQL. Additionally, research into "Tensor Logic" and tools like TenSQL (an SQL database based on GraphBLAS) have explored similar intersections of relational algebra and AI.

Scaling Challenges

Critics point out that natural joins can suffer from combinatorial explosion, leading to scaling issues. Some suggest that limiting the expressiveness of the queries (e.g., avoiding triangular queries) or using frameworks like Differential Dataflow could provide better parallelization and scaling properties for hardware utilization.