Model Parallelism
=================

DeepSpeed-MII supports model parallelism via tensor parallelism for splitting models across multiple GPUs.

For model parallelism with :doc:`pipeline`, please see :ref:`Pipeline Model
Parallelism <pipeline_model_parallelism>`.

For model parallelism with :doc:`deployment`, please see :ref:`Persistent
Deployment Model Parallelism <deployment_model_parallelism>`.