Archive for the 'cloud' Category


Ooops, did I crash the cloud?

today we received an e-mail from Amazon Web Services:

Hello,

We have noticed that one or more of your instances are running on a host
degraded due to hardware failure.

   i-c4b111ad

The risk of your instances failing is increased at this point. We cannot
determine the health of any applications running on the instances. We
recommend that you launch replacement instances and start migrating
to them.

Feel free to terminate the instances with the ec2-terminate-instance API
when you are done with them.

Let us know if you have any questions.

Sincerely,

The Amazon EC2 Team

Reference: dd735587-a56f-4b29-a3a2-fc2c7e2ca228

 
 
and yes, the machine had failures:

end_request: I/O error, dev sda1, sector 17383424
Buffer I/O error on device sda1, logical block 2172928
lost page write due to I/O error on sda1
end_request: I/O error, dev sda1, sector 17383624
Buffer I/O error on device sda1, logical block 2172953
lost page write due to I/O error on sda1
Aborting journal on device sda1.
end_request: I/O error, dev sda1, sector 17383624
Buffer I/O error on device sda1, logical block 2172953
lost page write due to I/O error on sda1
EXT3-fs error (device sda1) in ext3_ordered_writepage: IO failure
end_request: I/O error, dev sda1, sector 0
Buffer I/O error on device sda1, logical block 0
lost page write due to I/O error on sda1
ext3_abort called.
EXT3-fs error (device sda1): ext3_journal_start_sb: Detected aborted journal
Remounting filesystem read-only
end_request: I/O error, dev sda1, sector 8920928
end_request: I/O error, dev sda1, sector 8920928
end_request: I/O error, dev sda1, sector 8920928
end_request: I/O error, dev sda1, sector 8920928
end_request: I/O error, dev sda1, sector 8920928

 
so: save early, safe often. and make backups!