py-mesh-tensorflow

v 0.1.18.20210305

Mesh TensorFlow - Model Parallelism Made Easier

Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying a broad class of distributed tensor computations. The purpose of Mesh TensorFlow is to formalize and implement distribution strategies for your computation graph over your hardware/processors. For example: Split the batch over rows of processors and split the units in the hidden layer across columns of processors. Mesh TensorFlow is implemented as a layer over TensorFlow.

Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying a broad class of distributed tensor computations. The purpose of Mesh TensorFlow is to formalize and implement distribution strategies for your computation graph over your hardware/processors. For example: Split the batch over rows of processors and split the units in the hidden layer across columns of processors. Mesh TensorFlow is implemented as a layer over TensorFlow.

+ Show more

https://github.com/tensorflow/mesh

Installable ports: