Failure of attached network storage causing high load average on some vServers

DE: Die betroffenen vServer laufen aktuell wieder stabil. Wir beobachten dennoch den Sachverhalt weiter. EN: The affected vServers are currently stable again. However, we are continuing to monitor the situation.

DE: Reboot wird durchgeführt. Einzelne vServer davon betroffen.

EN: Reboot in progress. Partial vServers are affected.

We were seeing an unusual amount of request on single servers. We are closely monitoring the issue and see already a decrease in load average on these servers.

We've now resolved the incident. All systems are up again. If you are facing issues, please report them to service@web-vision.de We are now monitoring the system over the next two days. On Monday, minor follow-up tasks like re-migrations of servers, will be coordinated with the affected customers.

Thanks for your patience. Please accept our apologies for the inconveniences caused.

We've fixed the core issue, and are waiting for things to recover. At the moment, the affected Chunk server is in releasing status and distributing the stuck data chunks to other chunk servers.

At the moment, we are seeing that some servers are facing a "flapping" state, changing between off- and online. This is caused by old, already migrated servers seem to take packages on different hardware. We are at them moment checking the options to avoid this behavior.

More and more vServers are recovering now.

We are now migrating the affected vServers inside the cluster to none affected areas.

A failure of an attached network storage is causing at the moment high load average for some vServer, which make them inaccessible from the outside. At the moment, we are working on removing the failed network storage.