tensorflow/cloud

⛔️ DEPRECATED - The TensorFlow Cloud repository provides APIs that will allow to easily go from debugging and training your Keras and TensorFlow code in a local environment to distributed training in the cloud.

What it solves

TensorFlow Cloud simplifies the transition from local debugging and development of Keras and TensorFlow models to distributed training and tuning at scale on Google Cloud Platform (GCP). It removes the manual overhead of configuring cloud infrastructure, dockerizing applications, and setting up distribution strategies.

How it works

The project provides a run API that automates the deployment pipeline. When a user calls tfc.run(), the tool performs the following steps:

  1. Preparation: It makes the Keras script or notebook "cloud-ready."
  2. Containerization: It packages the code and dependencies into a Docker container, using either a local Docker daemon or Google Cloud Build.
  3. Deployment: It deploys the container to the Google AI Platform for execution.
  4. Monitoring: It handles log streaming and integrates with hosted TensorBoard for monitoring and checkpoint storage.

It automatically wraps model code in TensorFlow distribution strategies (such as OneDeviceStrategy, MirroredStrategy, MultiWorkerMirroredStrategy, or TPUStrategy) based on the provided machine configuration.

Who it’s for

Data scientists and ML engineers who use TensorFlow and Keras and want to scale their training from a local machine or notebook to GCP without manually managing infrastructure or Docker configurations.

Highlights

  • Seamless Scaling: Move from local to cloud training with a single API call.
  • Flexible Entry Points: Supports Python scripts, Jupyter notebooks, or calling run directly within the training script.
  • Automated Distribution: Automatically configures distribution strategies based on the hardware (CPUs, GPUs, TPUs) requested.
  • Integrated Tooling: Built-in support for Google Cloud Build and Google AI Platform.

Related

  • Project
    tensorflow/gnn

    A TensorFlow library for building and scaling Graph Neural Networks, featuring tools for heterogeneous graph representation and distributed graph sampling.

  • Project
    tensorflow/tfjs

    A hardware-accelerated JavaScript library for training and deploying machine learning models in the browser and Node.js.

  • Project
    tensorflow/tensorflow

    An end-to-end open source platform for machine learning that provides a flexible ecosystem of tools and libraries for building and deploying ML-powered applications.

  • Project
    tensorflow/models

    A collection of state-of-the-art model implementations and best practices for TensorFlow users to accelerate research and product development.

  • Project
    horovod/horovod

    A distributed deep learning training framework that makes it easy to scale single-GPU training scripts across multiple GPUs and hosts for TensorFlow, PyTorch, Keras, and MXNet.