Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying a broad class of distributed tensor computations. The purpose of Mesh TensorFlow is to formalize and implement distribution strategies for your computation graph over your hardware/processors. For example: Split the batch over rows of processors and split the units in the hidden layer across columns of processors. Mesh TensorFlow is implemented as a layer over TensorFlow.
Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying a broad class of distributed tensor computations. The purpose of Mesh TensorFlow is to formalize and implement distribution strategies for your computation graph over your hardware/processors. For example: Split the batch over rows of processors and split the units in the hidden layer across columns of processors. Mesh TensorFlow is implemented as a layer over TensorFlow.