I'm are having trouble making RHEL 5 and IBM DS4800 interact: Using RHEL's multipathing, I've managed to coalesce devices from different fibre paths in to mpath-devices in /dev/mpath. However, when I use the SAN management software to change the preferred path of a logical drive, the server doesn't seem to respect it. This generates an alarm in the SAN, saying Logical Drive Not On Preferred Path.
Hardware: Dell PowerEdge 2950 with a dual-channel QLogic QLE2462 (AKA 'ISP2432') based 4Gb Fibre Channel to PCI Express HBA.
Software: RHEL 5.2 x86_64. Same trouble was seen on the same server running RHEL 5.1 x86_64.
No 3rd party (such as IBM-, LSI- or QLogic-provided) drivers installed.
/etc/modprobe.conf:
alias eth1 bnx2
alias scsi_hostadapter megaraid_sas
alias scsi_hostadapter1 qla2xxx
alias net-pf-10 off
alias ipv6 off
IBM DS4800 with two controllers (firmware version 06.60.02.00). Between the DS4800 unit and the server, there are redundant FC switches.
Note that DS4800 doesn't support load balancing; i.e. only fail-over redundancy is supported.
multipath -ll' outputAs the DS4800 box is the only SAN storage unit, multipath.conf's defaults section is used instead of individual device specifications.
In the output below, the WWNs are fictional.
blacklist {
devnode "a"
}
defaults {
udev_dir /dev
polling_inte 10
selector "round-robin 0"
path_groupinlicy group_by_serial
getuid_callo "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_alua /dev/%n"
rr_min_io 100
failback immediate
path_checker tur
rr_weight priorities
user_friendlmes no
}
multipath -ll:/sbin/mpath_prio_tpc /dev/sdX by hand reveals that the command isn't supported for the involved devices.
blacklist {
devnode "a"
}
defaults {
udev_dir /dev
polling_inte 10
selector "round-robin 0"
path_groupinlicy group_by_serial
getuid_callo "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_tpc /dev/%n"
rr_min_io 100
failback immediate
path_checker tur
rr_weight priorities
user_friendlmes no
}
multipath -ll before having changed preferred path for the 140G logical drive in the SAN:3600d1b700011656c00004802434d4429dm-3 IBM,1815 FAStT [size=30G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:0 sdb 8:16 [active][ready] \_ 2:0:0:0 sdg 8:96 [active][ready] 3600d1b70001174ee0000ac7b483fcfb6dm-7 IBM,1815 FAStT [size=30G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:5 sdf 8:80 [active][ready] \_ 2:0:0:5 sdk 8:160 [active][ready] 3600d1b70001174ee00009e6e373ad726dm-5 IBM,1815 FAStT [size=50G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:2 sdd 8:48 [active][ready] \_ 2:0:0:2 sdi 8:128 [active][ready] 3600d1b700011656c0000552e47e88df7dm-6 IBM,1815 FAStT [size=40G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:4 sde 8:64 [active][ready] \_ 2:0:0:4 sdj 8:144 [active][ready] 3600d1b70001174ee0000a50252abfe88dm-4 IBM,1815 FAStT [size=140G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:1 sdc 8:32 [active][ready] \_ 2:0:0:1 sdh 8:112 [active][ready]
multipath -ll after having changed preferred path for the 140G logical drive in the SAN (note: prio=2 instead of prio=6, for whatever that's worth):3600d1b700011656c00004802434d4429dm-3 IBM,1815 FAStT [size=30G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:0 sdb 8:16 [active][ready] \_ 2:0:0:0 sdg 8:96 [active][ready] 3600d1b70001174ee0000ac7b483fcfb6dm-7 IBM,1815 FAStT [size=30G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:5 sdf 8:80 [active][ready] \_ 2:0:0:5 sdk 8:160 [active][ready] 3600d1b70001174ee00009e6e373ad726dm-5 IBM,1815 FAStT [size=50G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:2 sdd 8:48 [active][ready] \_ 2:0:0:2 sdi 8:128 [active][ready] 3600d1b700011656c0000552e47e88df7dm-6 IBM,1815 FAStT [size=40G][features=0][hwhandler=0] \_ round-robin 0 [prio=6][active] \_ 1:0:0:4 sde 8:64 [active][ready] \_ 2:0:0:4 sdj 8:144 [active][ready] 3600d1b70001174ee0000a50252abfe88dm-4 IBM,1815 FAStT [size=140G][features=0][hwhandler=0] \_ round-robin 0 [prio=2][active] \_ 1:0:0:1 sdc 8:32 [active][ready] \_ 2:0:0:1 sdh 8:112 [active][ready]
blacklist {
devnode "sda"
}
defaults {
failback immediate
no_path_retry queue
user_friendly_names no
hardware_handler "1 rdac"
path_checker rdac
path_grouping_policy failover
prio_callout "/sbin/mpath_prio_rdac /dev/%n"
}
multipath -ll before having changed preferred path for the 140G logical drive in the SAN:3600d1b700011656c00004802434d4429dm-3 IBM,1815 FAStT [size=30G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=100][active] \_ 1:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:0 sdd 8:48 [active][ready] 3600d1b70001174ee0000ac7b483fcfb6dm-7 IBM,1815 FAStT [size=30G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=100][active] \_ 2:0:0:5 sdj 8:144 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 1:0:0:5 sdk 8:160 [active][ready] 3600d1b70001174ee00009e6e373ad726dm-5 IBM,1815 FAStT [size=50G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=100][active] \_ 1:0:0:2 sdf 8:80 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:2 sdg 8:96 [active][ready] 3600d1b700011656c0000552e47e88df7dm-6 IBM,1815 FAStT [size=40G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=100][active] \_ 1:0:0:4 sdh 8:112 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:4 sdi 8:128 [active][ready] 3600d1b70001174ee0000a50252abfe88dm-4 IBM,1815 FAStT [size=140G][features=1 queue_if_no_path][hwhandler=0] \_ round-robin 0 [prio=0][active] \_ 1:0:0:1 sdc 8:32 [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 2:0:0:1 sde 8:64 [active][ready]
In the previous example, hwhandler=0, even though the hardware handler had been specified as "1 rdac". So I tried adding a file /etc/sysconfig/modules:
#!/bin/sh modprobe dm_rdac >/dev/null 2>&1
blacklist {
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^sda"
device {
vendor "Maxtor*"
product "OneTouch*"
}
}
blacklist_exceptions {
device {
vendor "IBM"
product "1815*"
}
}
defaults {
failback immediate
no_path_retry queue
user_friendly_names no
path_grouping_policy failover
}
devices {
device {
vendor "IBM"
product "1815*"
failback manual
hardware_handler "1 rdac"
path_checker rdac
prio_callout "/sbin/mpath_prio_rdac /dev/%n"
}
}
multipath -ll:3600d1b700011656c00004802434d4429dm-3 IBM,1815 FAStT [size=30G][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=100][active] \_ 1:0:0:0 sdb 8:16 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:0 sdg 8:96 [active][ready] 3600a0b80001174ee0000ac7a480fcfb6dm-6 IBM,1815 FAStT [size=30G][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=100][active] \_ 1:0:0:5 sdf 8:80 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:5 sdk 8:160 [active][ready] 3600d1b70001174ee00009e6e373ad726dm-5 IBM,1815 FAStT [size=50G][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=100][active] \_ 1:0:0:2 sdd 8:48 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:2 sdi 8:128 [active][ready] 3600d1b700011656c0000552e47e88df7dm-6 IBM,1815 FAStT [size=40G][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=100][active] \_ 1:0:0:4 sde 8:64 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:4 sdj 8:144 [active][ready] 3600d1b70001174ee0000a50252abfe88dm-4 IBM,1815 FAStT [size=140G][features=1 queue_if_no_path][hwhandler=1 rdac] \_ round-robin 0 [prio=100][active] \_ 1:0:0:1 sdc 8:32 [active][ready] \_ round-robin 0 [prio=100][enabled] \_ 2:0:0:1 sdh 8:112 [active][ready]