No components marked as affected
Monitoring
Still seeing signs of improvement, most models are able to spin up with a few still waiting their turn from HuggingFace. Models should spin up after enough tries if they are failing for HuggingFace related reasons.
Monitoring
We have seen most HuggingFace calls start succeeding, except for a small subset that are being downloaded through cas-bridge.xethub.hf.co. Most models are now succeeding to spin up
Monitoring
HuggingFace is reporting recovery, which we are also seeing on our end. However, we still see minor degradation, which we believe is due to thundering herd once the main issue was resolved. We are still keeping an eye out, but believe we are recovering
Investigating
We are aware of 504s being returned by models that reach out to HuggingFace during setup. We believe this is likely related to the disruption visible at https://status.huggingface.co/. We are monitoring and will update when we can identify the HuggingFace CDN is back up/