Good day,
As a recipient of this e-mail, you are currently registered as a developer on Goo.im. I do wish this mail was in better tidings, however, it is not. During our maintenance yesterday, expansion of the Lustre FS that files are stored on, was on the todo list, and was done. However, during this, the old primary OST for Lustre died, causing a race condition where the new server was seen as primary, and the metadata server blanked itself, causing a critical loss of all files stored on the FS. Unfortunately, there is no recovery of files from this. At this time, all home directories have been re-created, and users re-linked to /dev/ folders on Goo.im at this time.
Full details can be found at:
Lustre, Pain, And Goo.
We're requesting that files be uploaded at your convenience, we're also beginning to work on a full backup system to avoid a critical loss like this again in the future.
Please feel free to contact us with any questions/concerns.
Thank you,
Alexander Blair
