Troubles getting a software RAID 5 to work correctly

Nicholas Leippe nick at
Fri Nov 24 17:08:55 MST 2006

On Friday 24 November 2006 14:15, Kenneth Burgener wrote:
> I am working on building my home file server (as noted from a previous
> post about which IDE controller card to purchase), and I am running into
> a minor stumbling block.
> Hardware:
> 1 x IDE hard drive for OS
> 3 x 250GB IDE hard drives (RAID ARRAY #1 - level 5)  hdc hde hdg
> 3 x 200GB IDE hard drives (RAID ARRAY #2 - level 0)  hdd hdf hdh
> for roughly 1.1 terabytes of disk space.
> I am able to create the level 0 array with no problems.  I am even able
> to create the level 5 array, which appears to work, until I attempt to
> simulate a failure.
> I get the following error when I fail/remove hdc and attempt to
> stop/restart the array, "assembled from 1 drive and 1 spare - not enough
> to start the array."

[UU_] indicates that the array is degraded.  Either a device has failed (or is 
in failure mode because you told it so), or it has not completed 
reconstruction (it remains degraded until this is done).

When you first create a raid5 array, it computes the parity across the entire 
volume. (I know, kinda dumb when the data is all junk.)

from man mdadm(8):
When creating a RAID5 array, mdadm will automatically create a degraded array 
with an extra spare drive.  This is because building the spare into a 
degraded array is in general faster than resyncing the parity on a 
non-degraded, but not clean, array.  This feature can be over-ridden with 
the --force option.

Until it finishes adding the third drive into the array, removing either of 
the others fails the array completely--IOW all of the data is now lost.

I don't recall if you need to manually tell it to use the spare (I usually 
use --force on a new array).  If /proc/mdadm does not show that 
reconstruction is in progress, then you'll have to tell it to do it.

> Any thoughts as to why I am unable to remove any single disk, and not
> run in "degraded" mode?

You're already in degrade mode.  You can safely fail the drive being added 
into the array, just none of the remaining drives while it is incomplete.

> As soon as I can get this and other things stable I intend to install
> the 64bit edition, but if I can't get simple items like this working, I
> dare not play with 64bit.

In my experience, 64bit vs 32bit makes no difference for the raid code--both 
are rock stable.

Also, with such large arrays, I recommend using the bitmap feature in case 
reconstruction gets interrupted for whatever reason.  (power outage, or 

> The other concern is what happens if I reinstall the OS?  Will I be able
> to rebuild the array, or will all of my data be lost?

The linux software raid format is stable, and is binary compatible across 
architectures (both endianness and bit width).  So, yes, you'll still have 
your data.

More information about the PLUG mailing list