apache/systemds
An open source ML system for the end-to-end data science lifecycle
What it solves
Apache SystemDS is designed to handle the entire data science lifecycle, covering everything from initial data preparation and cleaning to model training, debugging, and serving.
How it works
Users specify ML algorithms or pipelines using a high-level language with R-like syntax, or through Python and Java APIs. The system then automatically generates hybrid runtime plans that combine local in-memory operations with distributed operations on Apache Spark. It also supports additional backends for GPUs and federated learning.
Who it’s for
It is built for data scientists and developers who need a scalable system for end-to-end machine learning pipelines.
Highlights
- End-to-end data science lifecycle support
- Hybrid runtime plans (local and Apache Spark distributed)
- R-like syntax and Python/Java APIs
- Support for GPU and federated learning backends
Related
- Project
- Project
- Project
- Project
- Project