Labels

R (15) Admin (12) programming (11) Rant (6) personal (6) parallelism (4) HPC (3) git (3) linux (3) rstudio (3) spectrum (3) C++ (2) Modeling (2) Rcpp (2) SQL (2) amazon (2) cloud (2) frequency (2) math (2) performance (2) plotting (2) postgresql (2) DNS (1) Egypt (1) Future (1) Knoxville (1) LVM (1) Music (1) Politics (1) Python (1) RAID (1) Reproducible Research (1) animation (1) audio (1) aws (1) data (1) economics (1) graphing (1) hardware (1)

11 June 2013

Learning new fileserver tricks: RAID + LVM

I've finally gotten comfortable with linux's software raid, aka mdadm. I've been hearing about LVM, and I finally took the plunge and figured out how to get the two to play together. Of course, a benefit of RAID is data security. The big benefit I see from LVM is getting to add/remove disk space without repartitioning. Once RAID is working, stacking LVM on top was easy enough, especially for my use case of a single-big-filesystem. I was able to move all my data onto one RAID array, built a new filesystem on top of a logical volume, move data to the new filesystem, and then add the final RAID array to the logical volume and resize the filesystem. Thus, I end up with 3 separate RAID arrays glommed together into a single, large filesystem.

## Tell LVM about RAID arrays 
sudo pvcreate /dev/md2
sudo pvcreate /dev/md3

## Create a volume group from empty RAID arrays
sudo vgcreate VolGroupArray /dev/md2 /dev/md3

## Create a logical volume named "archive", using all available space 
sudo lvcreate -l +100%FREE VolGroupArray -n archive
sudo lvdisplay 
## and create a filesystem on the new logical volume 
sudo mkfs.ext4 /dev/VolGroupArray/archive

## mount the new filesystem
## and move files from the mount-point of /dev/md1 to /dev/VolGroupArray/archive
## then unmount /dev/md1

## Add the last RAID array to the volume group
sudo pvcreate /dev/md1
sudo vgextend VolGroupArray /dev/md1

## Update the logical volume to use all available space 
sudo lvresize -l +100%FREE /dev/VolGroupArray/archive
## And resize the filesystem -- rather slow, maybe faster to unmount it first...
sudo resize2fs /dev/VolGroupArray/archive

## Finally, get blkid and update /etc/fstab with UUID and mount options (here, just noatime)
sudo blkid

I probably should have made backups before I did this, but everything went smoothly...
Also, I discovered this python tool to do conversions in-place. Again, this appears non-destructive, but back-ups never hurt. Also of interest for a file server is Smartmontools to monitor for hardware/disk failures: a nice review is here.

[REFS]
* http://home.gagme.com/greg/linux/raid-lvm.php
* https://wiki.archlinux.org/index.php/Software_RAID_and_LVM
* http://webworxshop.com/2009/10/10/online-filesystem-resizing-with-lvm

No comments:

Post a Comment