[Identified] There is an issue/bug with suspending GPU instances with the version of libvirt Jetstream2 is using for virtualization.
We will have to upgrade the compute nodes to resolve it. This is on the near-term timeline but we do not have a precise date at this time.
In the meantime, please only use stop or shelve with GPU instances.
Jetstream2 Research and Education Cloud
JS2 Docs are here: https://docs.jetstream-cloud.org/
Primary Data Center, ASU Data Center, Cornell Data Center, Hawaii Data Center, TACC Data Center
September 21, 2022 4:19PM EDT September 21, 2022 8:19PM UTC
[Resolved] Vendor issue with coolant system caused thermal shutdown on Jetstream2 compute nodes.
These have all been returned to service at this time. If your VM is not reachable, you may need to reboot it via the interface of your choice.
September 21, 2022 3:08PM EDT September 21, 2022 7:08PM UTC
[Monitoring] There was a coolant outage at the IU Data Center in Bloomington. Engineers are investigating this issue now.
Direct cooled nodes went into thermal shutdown. Jetstream2 engineers are working to bring these back online presently.