Replace a failing drive in a ZFS Zpool

I had a hard drive start throwing some sense key errors on one of our big Sun servers at work today. Besides dealing with problems from a major power outage in Southern California recently this was just one more thing to deal with. Good news is it was part of a ZFS zpool and after some refreshing of the commands I needed to use I was able to replace the drive in a matter of minutes and there was no downtime for any of the users on the system. We have about 1,000 zfs file systems on this particular zpool, mostly all being very active user home directories. It is also a print server for about 1,000 users.

ZFS is amazing.

If you need to replace a failing drive in a ZFS zpool, maybe seeing what I needed to do will help you. Follow along to see what was needed.

Here you can see the sense key errors in the messages file

Running the zpool status command shows me only a single read error. This is enough for me to swap out the drive, this would be the c1t5d0 device.

The Sun server has many drives in it and I wanted to make sure I pull the correct drive out so I use the luxadm command to flash the amber light next to the drive in the system. These are fiber channel drives so luxadm is used. Next I tell the zpool system to take the drive offline so I can hot swap it out of the system. I run the the zpool status command to verify the drive is offline. Now I need to run to the data center, find the flashing light on the disk drive cage and swap it out with a new one.

Once the drive is replaced I run the devfsadm -C command to tell the OS to scan the system and add the new drive so it is usable to the zpool system. Now tell zpool to replace the failing drive with the new one. Since I was doing a direct replacement from the same slot in the system the command was very simple, "zpool replace zdata c1t5d0", "zdata" being the name of the zpool itself. Now running the zpool status command you can see it has already begun to rebuild the new drive back into the zpool volume.

With just a few simple commands I was able to hot swap and replace a failing disk drive in a very busy Sun server with no downtime at all. ZFS is an amazing piece of technology.

I hope this post might be helpful for you if you need to replace a failing hard drive using ZFS.

ImageMagick - CLI image manipulation

I think ImageMagick is my new BF today. I take pictures with my phone and it embeds a lot of information into the meta-data of the actual image. Data like camera model, date/time and GPS location. Sometimes when I publish photos on the web I don't want that information to be included with the image. I started looking for a tool that would allow me to strip out the meta-data from large quantities of images.

ImageMagick to the rescue.

In a simple one line command I can remove the metadata from all of the photos in the current dir:

prompt# mogrify -strip *.jpg

I can also easily embed a watermark or other text right onto the images. This command is slightly more complicated but not too long:

prompt# for f in `ls -1`; do convert -size 500x14 xc:none -gravity center -stroke black -strokewidth 2 -annotate 0 'Indian Valley ABDSP 2010 - www.sefcik.com' -background none -shadow 100x3+0+0 +repage -stroke none -fill white -annotate 0 'Indian Valley ABDSP 2010 - www.sefcik.com' $f +swap -gravity south -geometry +0-3 -composite wm_${f}; done

This basically loops thru all of the images in the current dir, creates a new image and adds a watermark with the location of the picture and my website url to the new image. Both of these commands took seconds to run on 170 images total.

You can see the result on this image, it placed the text on the bottom center (you may need to ckick the image to enalrge it for a better view).

ImageMagick is extremely powerful for both web developers and the average user, more info about ImageMagick can be found on their website.

--