Linux MD problem
shane at hathawaymix.org
Tue Jul 7 22:55:20 MDT 2009
Mike Lovell wrote:
> I have a machine that has 4 disks in a raid 10 using md. The machine
> went through an unclean shutdown yesterday and when the box came up, I
> saw errors like the following in the kernel log and the array no longer
> [ 28.575149] md: raid10 personality registered for level 10
> [ 28.610827] md: md0 stopped.
> [ 28.688678] md: bind<sdu1>
> [ 28.688981] md: bind<sdv1>
> [ 28.689269] md: bind<sdw1>
> [ 28.689566] md: bind<sdx1>
> [ 28.689748] md: kicking non-fresh sdw1 from array!
> [ 28.689890] md: unbind<sdw1>
> [ 28.690036] md: export_rdev(sdw1)
> [ 28.690175] md: kicking non-fresh sdv1 from array!
> [ 28.690316] md: unbind<sdv1>
> [ 28.690452] md: export_rdev(sdv1)
> [ 28.690589] md: kicking non-fresh sdu1 from array!
> [ 28.690730] md: unbind<sdu1>
> [ 28.690866] md: export_rdev(sdu1)
> [ 28.704706] raid10: not enough operational mirrors for md0
> [ 28.704706] md: pers->run() failed ...
> Anyone have some suggestions on what can be done to get the array
> working again? There are a lot of virtual machines that are on the array
> that I really don't want to have to go through and rebuild. Thanks in
> advance for any advice you guys have.
I would try assembling the array without sdx1:
mdadm --assemble /dev/md0 /dev/sd[uvw]1 missing
If that works, I would check the filesystem for errors *without
repairing anything yet*, since I don't want to change any bits at all yet:
e2fsck -n -f /dev/md0
If that shows no errors, or a small number of easily repaired errors,
then I would try to re-add sdx1:
mdadm --manage --re-add /dev/md0 /dev/sdx1
If that second command fails, I would use "--add" instead of "--re-add",
causing sdx1 to be rebuilt from scratch.
Then I would perform the real filesystem repair:
e2fsck -p -f /dev/md0
If the first command I gave doesn't work, I would use "dd" to copy the
partitions to backup drives before doing anything else, then I would do
more aggressive things like "mdadm --assemble --force --update resync".
More information about the PLUG