No components marked as affected
Resolved
Model setup failures have dropped to normal levels since 10:40 UTC. Thank you for your patience!
Monitoring
We are seeing download issues from HuggingFace for a handful of T4 models, stopping new instances from starting. Requests for those models may be delayed as new capacity fails to come online. We will continue to monitor the situation.
Investigating
The problem seems to be isolated to models downloading weights from HuggingFace.
Investigating
We are seeing increased setup failure rates on models using T4 GPUs.
