sonnet: a composable neural network abstraction library for TensorFlow 2

What it solves

Sonnet is a library for machine learning research that simplifies the construction of neural networks. It provides a set of composable abstractions that allow researchers to build complex models without being forced into a specific training framework or opinionated workflow.

How it works

The library is built on top of TensorFlow 2 and centers around the snt.Module concept. Modules are self-contained, decoupled units that can hold parameters, other modules, and methods for processing input. Users can use predefined modules like snt.Linear or snt.nets.MLP or create their own by subclassing snt.Module.

Who it’s for

It is designed for machine learning researchers who need a high degree of control over their models and training processes, particularly those working within the TensorFlow ecosystem.

Highlights

• Unopinionated Design: Does not include a training framework, leaving the design of the training loop and optimizer choice to the user.
• Composable Abstractions: Uses a snt.Module system to allow for easy nesting and nesting of network components.
• Flexible Serialization: Supports Python's pickle, TensorFlow Checkpointing for saving training progress, and TensorFlow Saved Model for exporting models decoupled from Python source code.
• Distributed Training Support: Provides tools for distributed training via custom TensorFlow distribution strategies, while maintaining user control over gradient averaging and batch normalization stats.