RAID Rebuild Failed or Stuck? Do This Before Retrying

A RAID rebuild reads every surviving disk from start to finish in order to recalculate the missing data. That is a heavy workload for disks that are already ageing, and it is exactly when a second problem tends to appear: a weak disk that hits read errors, a controller that drops a member, or a rebuild that stops at a fixed percentage. What happens in the next few minutes decides whether the array can still be recovered.

Each retry puts the same load on the same disks and can push a marginal disk over the edge. If the array is still readable, copy the most important data off it first, to storage that is not part of the array. If it has gone offline, leave it offline. Powering disks on and off repeatedly is also worth avoiding on a disk that is already struggling.

Before anything is moved, note which disk sits in which slot, and label them. Export or photograph the controller's configuration: the RAID level, the stripe size, the disk order and the parity layout. This information is what allows the array to be reassembled correctly later, and it is easily lost if disks are shuffled or the controller configuration is cleared.

Do not initialize the array, do not create a new array over the same disks, and do not accept a prompt to clear or import a foreign configuration without understanding it, because some of these operations write new metadata or wipe existing structures. Avoid running file system repair tools such as chkdsk or fsck on a degraded or offline volume, because they write to the disks on the assumption that the underlying array is healthy.

Replacing several disks at once, or swapping in a disk of a different size, can also make a bad situation harder to untangle. If a spare is going to be used, it should be one change at a time and with the array's state recorded first.

Many controllers allow a member that was marked as failed to be forced back online. That can bring an array up, but if the disk missed writes while it was out, its data no longer matches the rest, and using it can corrupt the array. It is a decision to make deliberately, after copying the disks, and not as a reflex.

The safest approach is to make a full image of each disk, using a tool that handles read errors gracefully, and to work on the images rather than on the originals. That protects the original data from further reads on failing disks, and it allows the array to be reconstructed in software with the configuration noted earlier.

If more than one disk has failed, if the array will not come online, or if the configuration is unclear, the outcome depends on the state of each disk and the correctness of the reconstruction. We have solved many cases like this, but each additional write to the original disks reduces the chances.