Thomas (boggyb) wrote,

  • Music:
Since I'm sure talismancer will post even more snark if I just leave it at the two previous posts for today, have a not-even-slightly-epic snippet of computer fail.

There's an unexpected downside to reducing the amount of servers you need by consolidating several under-used ones into a bunch of virtual machines on one real server: when (not if) the server fails you lose all the virtual machines.

In this case it was a disk failure that took out the server. Now, disk failures had been planned for by creating a RAID-5 array, which uses MATHS to ensure that you can lose a disk in the array without losing any data.

So Murphy ensured that on the server in question two disks failed a few hours apart, at which point the RAID controller threw its hands up in the air and gave up. IT then had to spend several hours rebuilding the various VMs from backups.

I wonder how common double disk failures actually are? I do recall reading something about this a few years ago - someone had a look at the specified uncorrectable error rate (how likely the drive is to be unable to read a sector). They worked out that with high-capacity hard disks (on the order of 1TB) there's a scarily high chance of a RAID-5 rebuild failing due to an uncorrectable error on one of the remaining disks.
Tags: computing, nablopomo

  • Name that Song, round 11!

    Well, people have been pestering me for a blog post - so here, have another round of Name that Song! The rules are simple: Step 1: Put your…

  • End of the Year meme 2019

    Welcome all to a new decade! What did you do in 2019 that you'd never done before? I'm... not sure there was anything new in 2019. Does seeing…

  • Week of books meme: day 7

    "To post seven books I love, one book per day. No explanations, no reviews, just covers. Everyday I'll ask a friend to share the challenge. Let's…

  • Post a new comment


    default userpic
    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.
  • 1 comment