Increased setup failures for T4 models
Resolved

Model setup failures have dropped to normal levels since 10:40 UTC. Thank you for your patience!

Mon, Jan 26, 2026, 03:16 PM
(2 weeks ago)
·
Affected components

No components marked as affected

Updates

Resolved

Model setup failures have dropped to normal levels since 10:40 UTC. Thank you for your patience!

Mon, Jan 26, 2026, 03:16 PM

Monitoring

We are seeing download issues from HuggingFace for a handful of T4 models, stopping new instances from starting. Requests for those models may be delayed as new capacity fails to come online. We will continue to monitor the situation.

Mon, Jan 26, 2026, 10:10 AM(5 hours earlier)

Investigating

The problem seems to be isolated to models downloading weights from HuggingFace.

Mon, Jan 26, 2026, 08:53 AM(1 hour earlier)

Investigating

We are seeing increased setup failure rates on models using T4 GPUs.

Mon, Jan 26, 2026, 07:33 AM(1 hour earlier)