Implementing Neural Networks in SQL with Xarray-SQL

Neural Networks as Relational Operations

Xarray-SQL demonstrates that neural networks can be implemented directly within a SQL database by mapping multidimensional arrays (Nd arrays) to a tabular model. In this architecture, orthogonal dimensions of the Nd array serve as primary keys in a relational table, allowing tensor operations to be expressed as standard SQL queries.

This approach transforms the training and execution of neural networks into a series of relational operations, potentially allowing the logical dataflow of a model to be decoupled from its physical execution layer. This decoupling could enable more efficient distribution of workloads across large-scale GPU clusters by utilizing a global logical plan.

Mapping Tensor Algebra to SQL

The core of this implementation relies on the mapping of linear algebra operations to relational algebra. Specifically, matrix multiplication (matmul) is implemented as a combination of JOIN, GROUP BY, and SUM(val * val) operations.

This relational approach to tensor math has several key characteristics:

• Einsum Parallelism: The SQL implementation of matmul directly parallels einsum notation, where specific dimensions are contracted via joins and aggregations.
• Sparse Matrix Support: Operations like regridding—a core requirement in geospatial and climate science—are revealed to be sparse matrix-vector products that can be executed efficiently in SQL.
• Relational IR: By using relational algebra as an Intermediate Representation (IR), the system allows a database optimizer to reason about and optimize tensor programs.

Implementing Autograd in the Database

To move beyond simple linear algebra and into neural network training, Xarray-SQL implements automatic differentiation (autograd) directly on top of the DataFusion visitor pattern, drawing inspiration from JAX's implementation.

In the simplified array model used by Xarray-SQL, the system focuses on partial differentiation on the diagonal of the Jacobian. This reduces complex gradient calculations to row-wise operations, specifically implementing grad(), jvp (Jacobian-vector product), and vjp (vector-Jacobian product) as relational transformations.

Applications in Geospatial and Climate Science

The project originated from the need to perform complex physics calculations on massive datasets (100 TB range) common in climate and geospatial sciences. By pushing these calculations down into the database, Xarray-SQL avoids the need to post-process simulations in external numerical code.

Key findings from this application include:

• Relational Nature of Geo-Queries: Common operations in geospatial and climate science are essentially relational operations when the correct data model is applied.
• Database Push-down: Physics calculations that require gradients can be be pushed directly into the database engine, reducing data movement and increasing efficiency.

Technical Perspectives and Counterpoints

Community discussion highlights both the theoretical and practical implications of this approach. Some developers note that the mathematical relationship between einsum and database joins is well-established, noting that they operate over different semirings (real numbers for einsum and booleans for databases).

Other perspectives include:

"Initially rolled my eyes at 'neural networks in SQL,' but after reading the code I came away impressed basically it comes down to using relational algebra as the IR, letting a database optimizer reason about tensor programs."

Critics and observers have questioned the distinction between this approach and previous attempts to integrate ML into databases (such as Apache MADlib). The primary distinction here is the use of SQL as a declarative structured query language that can be optimized into a distributed directed acyclic graph (DAG) of processing, rather than simply adding ML functions as extensions to a relational engine.