mdadm

Use mdadm to create RAID levels.

Modes (from the man page):

  • Create - Create a new array with per-device metadata (superblocks).
  • Assemble - Assemble the components of a previously created array into an active array, either by searching or specifying devices directly (see Create mode).
  • Build - Build an array that doesn't have per-device metadata (superblocks); probably not going to use, see man page.
  • Manage - This is for doing things to specific components of an array such as adding new spares and removing faulty devices.
  • Misc - This is an 'everything else' mode that supports operations on active arrays, operations on component devices such as erasing old superblocks, and information gathering operations.
  • Grow - Grow (or shrink) an existing array, or otherwise reshape it in some way; for RAID1, RAID4, RAID5, RAID6 or RAID10.
  • Incremental Assembly - Add a single device to an existing array.
  • Follow or Monitor - Monitor one or more md devices and act on any state changes.
  • Auto-detect - This mode does not act on a specific device or array, but rather it requests the Linux kernel to activate any auto-detected arrays.

Options:

mdadm --assemble md-device ...
  • -A or –assemble md-device - assemble a pre-existing array
  • -B or –build md-device - build a legacy array without superblocks
  • -C or –create md-device - create a new array
  • -F or –follow or –monitor md-device - select Monitor mode
  • -G or –grow md-device - change to size or shape of an active array
  • -I or –incremental md-device - add or remove a single device to or from an array, and possibly restart it
  • –auto-detect - request that the kernel starts any auto-detected arrays – md driver has to be in-kernel, and not loaded as a module
  • -c filename - specify the config file to use
  • -s - scan config file or /proc/mdstat for missing information

For create, build, or grow:

mdadm --build md-device ...
mdadm --create md-device ...
  • -n or –raid-devices=n devices - specify the number of active devices in the array
  • -x or –spare-devices= - specify the number of spare devices in the initial array; spares can also be added and removed later
  • -l or –level= - set RAID level (raid1, raid4, raid5, raid6, raid10, etc.)

For assemble:

mdadm --assemble md-device ...
  • -u or –uuid - uuid of array to assemble
  • -R or –run - attempt to start the array even if fewer drives were given then the last time it was present

For manage mode:

mdadm --md-device ...
  • –re-add device - re-add a device that was previously removed from the array
  • -r or –remove device [device] - remove listed devices – they must not be active, as in they should be flagged as failed or spare devices
  • -f or –fail device [detached] [device [detached]] - mark listed devices as faulty; detach will cause a device that was been detached from the system to be marked as failed (?)

For misc. mode:

mdadm md-device options... devices...
  • -Q or –query device - examine a device to see if it an md device and if it is a component of an md array
  • -D or –detail print details of one or more md devices
  • –detail-platform - print details of the platform's RAID capabilities (firmware / hardware topology) for a given metadata format
  • -Y or –export - when used with –detail or –examine, output will be formatted as key=value pairs
  • -R or –run - start a partially assembled array
  • -S or –stop - deactivate array, releasing all resources
  • -o or –readonly - mark array as readonly
  • -w or –readwrite - mark array as readwrite
  • –zero-superblock - if the device contains a valid md superblock, it is overwritten with zeroes; IOW make this device not recognizable as a RAID capable one
  • -W or –wait - for each md device given, wait for any resync, recovery, or reshape activity to finish before returning
  • –wait-clean - for each md device given, or each device in /proc/mdstat if –scan is given, arrange for the array to be marked clean as soon as possible

For monitor mode:

  • -y or –syslog - cause all events to be reported through syslog
  • -r or –increment - give a percentage increment

Setup RAID

  • Partition the new disks
  • Leave the size for each disk all space minus 100 MB so new HDDs with slightly more or less space can still work.
  • Use non-fs data type for the partition that RAID will access, instead of the autodetect. The autodetect might be read / accessed wrong by other tools, say, a livecd.

RAID1

RAID1 is for mirroring data across disks. If one drive fails, the data is still preserved.

Create the RAID1 array using the two harddrives:

mdadm --create /dev/md0 -l raid1 -n 2 /dev/sdb1 /dev/sdc1

Monitor the creation of the RAID1 drive:

mdadm --monitor

Create a filesystem:

mkfs.ext3 /dev/md0

Stop the array:

mdadm --stop /dev/md0

Start the array:

mdadm --assemble /dev/md0 /dev/sdb1 /dev/sdc1

RAID5

RAID5 is for mirroring and striping. It will combine two disks and use the third for parity checks. If two drives fail, all the data is lost. At least three devices are needed.

Create a RAID5 array
mdadm --create /dev/md0 --level=5 -n 3 /dev/sda1 /dev/sdb1 /dev/sdc1

Howto

Add a drive to an existing RAID setup

In this scenario, /dev/sda1 and /dev/sdb1 already exist as block devices, and we are adding /dev/sdc1 to the array.

mdadm --add /dev/md0 /dev/sdc1
mdadm --grow --raid-devices=3 /dev/md0

See here for more instructions before doing this.

Gotchas

removed block devices

When mdadm is started building / rebuilding an array, it will create /dev/mdX entries, and remove the block devices that are part of the RAID (/dev/sda1, /dev/sdb1, fex). After stopping mdadm, run partprobe to load the device nodes. It will restart mdadm, but stopping it the second time will leave the device blocks there.

Examples

From the man page.

Query the device to see if it's a RAID array, or part of one, etc.:

mdadm --query device

Scan the config file and assemble and start all arrays:

mdadm --assemble --scan

Shut down all arrays that can be shut down; not currently in use:

mdadm --stop --scan

Check devices that have RAID superblocks and print out results:

mdadm --examine --brief --scan --config=partitions