I received some new drives recently to expand my NAS. Here I document the process of setting up a new drive, mainly for my future reference, to get everything in one place so I don’t need to look at a load of different articles to figure out what to do.
Write down the details
When I first receive a drive the first thing I do is update a spreadsheet with the details of it. This consists of:
- Serial number (double check this is correct!)
- Warranty expiry date
- A unique identifier in the format
, e.g. 001wd3
- The machine and bay it is mounted in
Once you’ve inserted it into your machine you need to figure out the device identifier, which is something like
sdX – where X is a letter (it may also be a different format, depending on the drive interface). This probably won’t map to physical bank or slot, so you need to figure out which drive is which. In my case I’m adding two new drives which are the same size and model. The
lsblk command is the first step to figuring it out:
$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 1 2.7T 0 disk └─vg001wd3-pool0 254:0 0 1.5T 0 lvm /pool sdb 8:16 1 2.7T 0 disk sdc 8:32 1 2.7T 0 disk sdd 8:48 1 58.7G 0 disk └─sdd1 8:49 1 1M 0 part
You need to look for the drives which don’t have any partitions associated (assuming you drive is brand new), so in this case
sdc. You can then use
smartctl to check the serial number:
$ smartctl -i /dev/sdb smartctl 6.4 2014-10-07 r4002 [x86_64-linux-4.6.0-0.bpo.1-amd64] (local build) Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate NAS HDD Device Model: ST3000VN000-1HJ166 Serial Number: W6A1JKW8 LU WWN Device Id: 5 000c50 09bbbe2bf Firmware Version: SC60 User Capacity: 3,000,592,982,016 bytes [3.00 TB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5900 rpm Form Factor: 3.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 3.0 Gb/s) Local Time is: Sat Jul 9 07:54:51 2016 EEST SMART support is: Available - device has SMART capability. SMART support is: Enabled
Drive Burn In
Before putting the drive in production you should perform a series of tests to ensure that it is working. This consists of running the passive SMART tests, and active badblocks test. If the drive fails this ask for a replacement from your retailer (don’t RMA it, as you’ll get a refurbished drive).
First run a couple of quick SMART tests, you’ll need to wait for the test to finish on a drive before running the next test, but you can test multiple drives at once. After running each command it’ll tell you how long the test will take (only a few minutes), and if you try to run multiple tests at once it’ll tell you that one is in progress.
$ smartctl -t short /dev/sdb $ smartctl -t conveyance /dev/sdb
Next up is the badblocks test. The SMART tests are passive, so they don’t usually report anything until the drive has attempted to be used. The badblocks test will write a set pattern to the whole drive, then read it back and check it is correct. This will wipe the drive, so don’t do this if you want to keep the data on it!
$ badblocks -b 4096 -c 98304 -ws /dev/sdb
-b 4096 parameter sets the block size of the drive (which is 4096 for modern hard disks), and the
-c 98304 parameter says to read 98304 blocks at once which is faster – this doesn’t affect the integrity of the test but will use more memory. In my case each instance of badblocks (I was setting up two new drives at once) used around 2GB of RAM. Note that this test will take a while, so run it in
tmux. In my case it took 48 hours on a 3TB drive.
Once that is complete, run a long SMART test which will take another few hours:
$ smartctl -t long /dev/sdb
Once complete, you can then check the SMART parameter of the drive.
$ smartctl -a /dev/sdb
The important bits to check are:
- The test log should show no errors (“Completed without error”)
- The “Current_Pending_Sector” and “Offline_Uncorrectable” parameters should be zero
Depending on who you ask the SMART parameters can mean a million things (most not important), so I’ll leave this as an exercise for the reader to figure out.
Full Disk Encryption
If you are going to setup full disk encryption now is the time. In my case I won’t be storing anything I need to keep secure (e.g. business documents) on this, but I’d like to use it as if I ever need to sell or RMA a drive I won’t have to worry about the data being read (I don’t really want some random person getting hold of my family photos). Unfortunately the CPU on my NAS is too slow, so I am not using it.
You can run the following command to test the performance of decryption operations on your CPU:
$ cryptsetup benchmark # Tests are approximate using memory only (no storage IO). PBKDF2-sha1 346751 iterations per second PBKDF2-sha256 193893 iterations per second PBKDF2-sha512 153121 iterations per second PBKDF2-ripemd160 230760 iterations per second PBKDF2-whirlpool 73801 iterations per second # Algorithm | Key | Encryption | Decryption aes-cbc 128b 74.7 MiB/s 87.2 MiB/s serpent-cbc 128b 35.8 MiB/s 97.8 MiB/s twofish-cbc 128b 81.5 MiB/s 107.3 MiB/s aes-cbc 256b 58.5 MiB/s 65.6 MiB/s serpent-cbc 256b 39.7 MiB/s 97.5 MiB/s twofish-cbc 256b 86.1 MiB/s 106.5 MiB/s aes-xts 256b 81.2 MiB/s 83.5 MiB/s serpent-xts 256b 90.4 MiB/s 92.4 MiB/s twofish-xts 256b 99.2 MiB/s 98.2 MiB/s aes-xts 512b 62.1 MiB/s 64.1 MiB/s serpent-xts 512b 91.0 MiB/s 92.5 MiB/s twofish-xts 512b 99.3 MiB/s 98.3 MiB/s
The drives I have can each read at 150MB/s – 200MB/s, so with four in the NAS, it can potentially read at 800MB/s. It only has gigabit ethernet, so in reality I’ll only be doing reads at around 125MB/s. However, housekeeping operations like scrubs will be faster with higher read speeds.
In reality I’d like to get at least 200MB/s decryption speeds for full disk encryption to be worth it for me, but as you can see the CPU can barely achieve half that (while idle). At this speed it’ll just be too slow, as the CPU is old (6 years) and doesn’t have native encryption instructions. For comparison my laptop, which has a modern Intel Core i5 processor, achieves 350MiB/s encryption and 1900MiB/s decryption.
As I’m not using it, I won’t explain how to setup full disk encryption, but the Arch wiki has good instructions (even if you are using another Linux distribution).
On top of the physical device, or encrypted device, you should create a LVM volume:
$ pvcreate /dev/sdb $ vgcreate 002sg3 /dev/sdb $ lvcreate -l 100%FREE 002sg3 -n pool0
In this case
002sg3 is the identifier I am using for this drive, and
pool0 is the identifier I’m using for the device pool. By using LVM it means if there is ever an issue with a drive, the filesystem tools will show your identifier instead of the system’s identifier – so you can easily figure out what drive is broken.
In this case we aren’t really using any features of LVM, but there is basically no performance impact, and it’s worth it for future expandability (and being able to easily identify a drive when it fails!). You could use it to, for example, split a drive amongst two pools, one in RAID1 and another in RAID6.
Next up create or add the drive to your filesystem pool. I’m using BTRFS, so to create a new pool it’s this:
$ mkfs.btrfs /dev/002sg3/pool0
And to add it to the existing pool (there is currently a single drive, I want to turn it into RAID1), it’s this:
$ btrfs device add /dev/002sg3/pool0 /pool $ btrfs balance start -dconvert=raid1 -mconvert=raid1 /pool
/pool in this case is the mount point of the filesystem – BTRFS can perform these operations while the filesystem is live!)