FastGen Quick Start Guide
Installation
API
Non-Persistent Pipelines
Persistent Deployments
Response Objects
Configuration
RESTful API
Model Parallelism
Model Replicas

DeepSpeed-MII

Model Replicas
View page source

Model Replicas

DeepSpeed-MII supports creating multiple replicas of a model with Persistent Deployments. Please see Persistent Deployment Model Replicas.

Previous

© Copyright 2023, Microsoft.

Built with Sphinx using a theme provided by Read the Docs.