Monday, April 14, 2008

SAN Pathing on an Active/Passive Array

So the best practice pathing on an Active/Passive array is MRU (Most Recently Used). This keeps the host from trying to use more than one SP and only failing The command to observe the startup behavior of paths in ESX 3.0.x is "esxcfg-mpath -l" /dev/sda is the local VMFS and the other path is the CDROM. Next you will see the 4 LUNS. These are on two separate arrays. One CX600 and one CX3/80. [root@xxxxx35 root]# esxcfg-mpath -l Disk vmhba0:0:0 /dev/sda (139392MB) has 1 paths and policy of Fixed Local 2:14.0 vmhba0:0:0 On active preferred Enclosure vmhba0:264:0 (0MB) has 1 paths and policy of Fixed Local 2:14.0 vmhba0:264:0 On active preferred Disk vmhba1:0:0 /dev/sdb (256000MB) has 4 paths and policy of Most Recently Used FC 15:0.0 2100001b3209abad<->500601601060225b vmhba1:0:0 On active preferred FC 15:0.0 2100001b3209abad<->5006016a1060225b vmhba1:1:0 Standby FC 17:0.0 2100001b32091db5<->500601681060225b vmhba2:0:0 Standby FC 17:0.0 2100001b32091db5<->500601621060225b vmhba2:1:0 On Disk vmhba1:0:1 /dev/sdc (256000MB) has 4 paths and policy of Most Recently Used FC 15:0.0 2100001b3209abad<->500601601060225b vmhba1:0:1 On active preferred FC 15:0.0 2100001b3209abad<->5006016a1060225b vmhba1:1:1 Standby FC 17:0.0 2100001b32091db5<->500601681060225b vmhba2:0:1 Standby FC 17:0.0 2100001b32091db5<->500601621060225b vmhba2:1:1 On Disk vmhba1:2:0 /dev/sdd (256000MB) has 4 paths and policy of Most Recently Used FC 15:0.0 2100001b3209abad<->5006016a39a02964 vmhba1:2:0 On active preferred FC 15:0.0 2100001b3209abad<->5006016339a02964 vmhba1:3:0 Standby FC 17:0.0 2100001b32091db5<->5006016239a02964 vmhba2:2:0 Standby FC 17:0.0 2100001b32091db5<->5006016b39a02964 vmhba2:3:0 On Disk vmhba1:2:1 /dev/sde (256000MB) has 4 paths and policy of Most Recently Used FC 15:0.0 2100001b3209abad<->5006016a39a02964 vmhba1:2:1 On active preferred FC 15:0.0 2100001b3209abad<->5006016339a02964 vmhba1:3:1 Standby FC 17:0.0 2100001b32091db5<->5006016239a02964 vmhba2:2:1 Standby FC 17:0.0 2100001b32091db5<->5006016b39a02964 vmhba2:3:1 On [root@xxxxxx35 root]# You can see how ESX loads all the paths to the first hba, in this case vmhba1 (On Active). The array has a preferred SP so the array picks which Storage Processor, and the ESX host picks which hba. This will eventually create a bottleneck on vmhba1(or first HBA). To solve this, I'm working with VMWare on the best way to balance the paths. There will be no intelligence to the balancing based on load, but I would like to put half of the LUNs onto the second HBA. To do this I'm looking at a startup script that would disable the standby path on the first HBA and then disable the On Active path on the first HBA. This would force the path to fail to the second HBA, and allow the path to stay on the preferred Storage Processor. If the On Active path is failed first, this could cause a LUN trespass and move the LUN (and all other ESX hosts connected to the LUN) to the second Storage Processor. After a pause both paths would be re-enabled. I have proposed this to our VMWare/EMC TAM's and am waiting on a responses.

0 comments: