This is a quick article about using Adaptec RAID controllers with OpenIndiana. In this case, I am using an Adaptec 5805. The driver appears to handle hardware based raid as well as JBOD. This is an improvement over the Oracle shipped driver in Solaris, or OpenSolaris. OpenIndiana does not ship with the StorMan package, which provides arcconf, so you will have to fetch StorMan from Adaptec yourself and install it.
Like most RAID command line tools, arcconf has a myriad of options to support your ZFS NAS server. We are going to cover these typical operations. It is my hope that by seeing these examples it will give readers a feel for how to use arcconf.
- How to destroy all JBODS
- How to create a hardware RAID10 logical disk
- How to check status of a logical disk
- Using iostat to check existing a logical drive
- How to lay down a UFS file system on a logical disk
- How to destroy a logical disk
- How to get a list of all physical drives
- How to turn all physical drives into JBODs suitable for ZFS
- How to create a hardware RAID6 logical disk
- How to verify the status of a logical disk
- How to start a verify_fix operation How to extract the event log from the RAID controller in XML format
- How to extract the event log from the RAID controller in tabular format
- How to extract the device log from the RAID controller in XML format
- How to extract the device log from the RAID controller in tabular format
- How to convert time stamps in the RAID controller to real time
- How to rescan the Bus for new or removed drives
- How to convert a RAID10 volume to RAID5 (I never recommend using RAID5)
- Replace a failed JBOD disk
Destroy all JBODS
root@vs-lm1577:/opt/StorMan# ./arcconf delete 1 JBOD ALL noprompt
Controllers found: 1
All data in JBOD 0,8 will be lost.
Deleting: JBOD 0,8
All data in JBOD 0,9 will be lost.
Deleting: JBOD 0,9
All data in JBOD 0,10 will be lost.
Deleting: JBOD 0,10
All data in JBOD 0,11 will be lost.
Deleting: JBOD 0,11
// there are 24 disks on box, we'll stop here as by now you get the idea
Command completed successfully.
Create RAID10 Volume
I recommend using ZFS for your mirrors instead of the RAID controller. I am sure you have your reasons for wanting to do this, but ZFS is better than hardware raid.
root@vs-lm1577:/opt/StorMan# ./arcconf create 1 LOGICALDRIVE \
Name TestVolume Max 10 0,8 0,9 0,10 0,11 noprompt
Controllers found: 1
Creating logical device: TestVolume
Command completed successfully.
Check Status of Existing Volume
Check status of raid volume 0
root@vs-lm1577:/opt/StorMan# ./arcconf getconfig 1 LD 0
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical device number 0
Logical device name : TestVolume
RAID level : 10
Status of logical device : Optimal
Size : 1906678 MB
Stripe-unit size : 256 KB
Read-cache mode : Enabled
MaxIQ preferred cache setting : Disabled
MaxIQ cache setting : Disabled
Write-cache mode : Enabled (write-back)
Write-cache setting : Enabled (write-back) when protected by battery/ZMM
Partitioned : Yes
Protected by Hot-Spare : No
Bootable : Yes
Failed stripes : No
Power settings : Disabled
--------------------------------------------------------
Logical device segment information
--------------------------------------------------------
Group 0, Segment 0 : Present (0,8) WD-WMATV4097675
Group 0, Segment 1 : Present (0,9) WD-WMATV4111023
Group 1, Segment 0 : Present (0,10) WD-WMATV4095249
Group 1, Segment 1 : Present (0,11) WD-WMATV4109136
Using iostat to Check Existence of a Logical Drive
In OpenIndiana, iostat -En sees the RAID10 volume
root@vs-lm1577:/opt/StorMan# iostat -En
c1t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: Adaptec Product: RAID 5805 Revision: V1.0 Serial No:
Size: 1999.31GB <1999307276288 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
How to Lay Down a UFS File System on a Logical Disk
root@vs-lm1577:/opt/StorMan# newfs /dev/dsk/c1t0d0s0
newfs: construct a new file system /dev/rdsk/c1t0d0s0: (y/n)? y
Warning: 304 sector(s) in last cylinder unallocated
/dev/rdsk/c1t0d0s0: 3904880336 sectors in 635560 cylinders of 48 tracks, 128 sectors
1906679.9MB in 4445 cyl groups (143 c/g, 429.00MB/g, 448 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 878752, 1757472, 2636192, 3514912, 4393632, 5272352, 6151072, 7029792,
7908512,
Initializing cylinder groups:
...............................................................................
.........
super-block backups for last 10 cylinder groups at:
3896557984, 3897436704, 3898315424, 3899194144, 3900072864, 3900951584,
3901830304, 3902709024, 3903587744, 3904466464
root@vs-lm1577:/opt/StorMan# mkdir /tmp/a
root@vs-lm1577:/opt/StorMan# mount /dev/dsk/c1t0d0s0 /tmp/a
root@vs-lm1577:/opt/StorMan# df -h /tmp/a
Filesystem Size Used Avail Use% Mounted on
/dev/dsk/c1t0d0s0 1.9T 257M 1.8T 1% /tmp/a
Destroy a Logical Disk
root@vs-lm1577:/opt/StorMan# ./arcconf delete 1 logicaldrive 0 noprompt
Controllers found: 1
WARNING: logical device 0 may contain a partition.
All data in logical device 0 will be lost.
Deleting: logical device 0 ("TestVolume")
Command completed successfully.
Get list of all physical drives
root@vs-lm1577:/opt/StorMan# ./arcconf getconfig 1 PD | \
grep "Reported Channel,Device(T:L)" |grep -v 0:0 | \
cut -d: -f 3 |cut -d'(' -f 1
0,8
0,9
0,11
0,12
0,13
0,14
0,15
0,16
0,17
0,18
0,19
0,21
0,22
0,23
0,24
0,25
0,26
0,27
0,28
0,29
0,31
How to turn all physical drives into JBODs suitable for ZFS
root@vs-lm1577:/opt/StorMan# list=`./arcconf getconfig 1 PD |\
grep "Reported Channel,Device(T:L)" |grep -v 0:0 |cut -d: -f 3 | \
cut -d'(' -f 1 | sed 's/,/ /g'` && ./arcconf create 1 jbod $list
Controllers found: 1
Created JBOD: 0,8
Created JBOD: 0,9
Created JBOD: 0,11
Created JBOD: 0,12
Created JBOD: 0,13
Created JBOD: 0,14
Created JBOD: 0,15
Created JBOD: 0,16
Created JBOD: 0,17
Created JBOD: 0,18
Created JBOD: 0,19
Created JBOD: 0,21
Created JBOD: 0,22
Created JBOD: 0,23
Created JBOD: 0,24
Created JBOD: 0,25
Created JBOD: 0,26
Created JBOD: 0,27
Created JBOD: 0,28
Created JBOD: 0,29
Created JBOD: 0,31
Command completed successfully.
How to create a hardware RAID6 logical disk
root@vs-lm1577:/opt/StorMan# ./arcconf create 1 logicaldrive max 6 $list
Controllers found: 1
Do you want to add a logical device to the configuration?
Press y, then ENTER to continue or press ENTER to abort: y
Creating logical device: Device 0
Command completed successfully.
How to start a verify_fix operation
What to do when your array has an ‘impacted’ status. Basically this means the array build process has not completed. You can verify this with the following command:
root@vs-lm1577:/opt/StorMan# ./arcconf getstatus 1
Controllers found: 1
Logical device Task:
Logical device : 0
Task ID : 100
Current operation : Build/Verify
Status : In Progress
Priority : High
Percentage complete : 0
Here we see that while we created the giant RAID6 volume, there is still background initialization going on. The volume is healthy, but performance is degraded while it completes the initialization.
This is how you start a build/verify check on your volume. This fails in my example as the job is already running.
root@vs-lm1577:/opt/StorMan# ./arcconf task start 1 logicaldrive 0 verify_fix noprompt
Controllers found: 1
Task 'Build/Verify' is already running on this device. Aborting
Command aborted.
How to extract the event log from the RAID controller in XML format
root@vs-lm1577:/opt/StorMan# ./arcconf getlogs 1 event
Controllers found: 1
Command completed successfully.
How to extract the event log from the RAID controller in tabular format
root@vs-lm1577:/opt/StorMan# ./arcconf getlogs 1 device tabular
Controllers found: 1
ControllerLog
controllerID ..................... 0
type ............................. 0
time ............................. 1308362117
version .......................... 3
tableFull ........................ false
How to convert time stamps in the RAID controller to real time
root@vs-lm1577:/opt/StorMan# perl -e 'print scalar localtime(shift),"\n";' 1308362117
Fri Jun 17 18:55:17 2011
How to rescan the Bus for new or removed drives
root@vs-lm1577:/opt/StorMan# ./arcconf rescan 1
Controllers found: 1
Command completed successfully.
How to convert a RAID10 volume to RAID5. He we are using the above RAID10.
First, let’s view our RAID10 volume:
root@vs-lm1577:/opt/StorMan# ./arcconf getconfig 1 ld 0
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical device number 0
Logical device name : TestVolume
RAID level : 10
Status of logical device : Optimal
Size : 1906678 MB
Stripe-unit size : 256 KB
Read-cache mode : Enabled
MaxIQ preferred cache setting : Disabled
MaxIQ cache setting : Disabled
Write-cache mode : Enabled (write-back)
Write-cache setting : Enabled (write-back) when protected by battery/ZMM
Partitioned : No
Protected by Hot-Spare : No
Bootable : Yes
Failed stripes : No
Power settings : Disabled
--------------------------------------------------------
Logical device segment information
--------------------------------------------------------
Group 0, Segment 0 : Present (0,8) WD-WMATV4097675
Group 0, Segment 1 : Present (0,9) WD-WMATV4111023
Group 1, Segment 0 : Present (0,10) WD-WMATV4095249
Group 1, Segment 1 : Present (0,11) WD-WMATV4109136
yep, it is still there.
Now, let’s convert it to RAID5
root@vs-lm1577:/opt/StorMan# ./arcconf modify 1 from 0 to max \
5 0 8 0 9 0 10 0 11 noprompt
Controllers found: 1
Reconfiguring logical device: TestVolume
Command completed successfully.
and now verify…
root@vs-lm1577:/opt/StorMan# ./arcconf getconfig 1 ld 0
Controllers found: 1
----------------------------------------------------------------------
Logical device information
----------------------------------------------------------------------
Logical device number 0
Logical device name : TestVolume
RAID level : 5
Status of logical device : Logical device Reconfiguring
Size : 2860022 MB
Stripe-unit size : 256 KB
Read-cache mode : Enabled
MaxIQ preferred cache setting : Disabled
MaxIQ cache setting : Disabled
Write-cache mode : Enabled (write-back)
Write-cache setting : Enabled (write-back) when protected by battery/ZMM
Partitioned : No
Protected by Hot-Spare : No
Bootable : Yes
Failed stripes : No
Power settings : Disabled
--------------------------------------------------------
Logical device segment information
--------------------------------------------------------
Segment 0 : Present (0,8) WD-WMATV4097675
Segment 1 : Present (0,9) WD-WMATV4111023
Segment 2 : Present (0,10) WD-WMATV4095249
Segment 3 : Present (0,11) WD-WMATV4109136
Yep, it worked as advertised.
Replace a Failed JBOD Disk
See my previous article on how to replace a failed JBOD
