1. Update the adapter settings on both vios (if not done already - normally this should be already set if you have disks attached to clients):
# chdev -l fscsi0 -a fc_err_recov=fast_fail # chdev -l fscsi0 -a dyntrk=yes
2. Make sure that SAN lun is available on both VIOS.
Also, in order to be safe and know exactly which disk you have to assign, do the following:
- discover the disk on one vio and add pvid to it:
gzvio1 $ cfgdev gzvio1 $ chdev -l hdisk69 -a pv=yes
gzvio2 $ cfgdev
3. Set reserve_lock as no to disks on both vios:
gzvioX $ chdev -l hdiskX -a reserve_lock=no
4. Set the reserve_policy to no_reserve on both vios:
gzvioX $ chdev -dev hdiskX -attr reserve_policy=no_reserve
5. Not required if your disks are newly added but it's always good to check if the disk(s) is(are) not already used by another client or even by vio itself:
gzvioX # oem_setup_env gzvioX # lqueryvg -Atp hdisk69
You should get this if the disk is empty:
0516-320 lqueryvg: Physical volume hdisk69 is not asigned to a volume group. 0516-066 lqueryvg: Physical volume is not a volume group member.
6. Map the disk to the client from both vios:
PS: I name virtual devices as VIO client hostname + disk size + Sequence number where sequence number is the next available one (if the client have already 5 disks, the next number would be 6).
gzvioX $ mkvdev -vdev hdiskX -vadapter vhost1 -dev server_size_number
7. Run cfgmgr on client and list the disks. The new ones normally are not assigned to any VG (None) and since you have already set ip the PVID on vio level, you should have the same here too. In our example it's hdisk5:
gzaix:~# cfgmgr gzaix:~# lspv hdisk0 00cf5e9f0aa3aa13 rootvg active hdisk1 00cf5e9f2d5fbe03 datavg active hdisk2 00cf5e9fbe1e95ff datavg active hdisk3 00cf5e9fbe1eb2e0 datavg active hdisk4 00cf5e9ff7560088 dsmonvg active [color=green]hdisk5 00cf5e9ff75600eb none[/color]
8. Check if you have both paths available:
gzaix:~# lspath -l hdisk5 Enabled hdisk5 vscsi0 Enabled hdisk5 vscsi1
9. At this point you may think you are free to include the disk into an existing volume group or create another one. Not just yet.
There are few steps to be done before, because once is added to the VG, you cannot do them unless you take the disk offline again.
a. Set up vscsi priority according to your design. The goal is to balance the CPU/Memory load across the vios. As idea, my standard is to direct path priority to vscsi0 for all even numbers and to vscsi1 for all odd numbers. It is up to you how you want to do it. If you do not know, ask the leader :).
gzaix:~# lspath -AE -l hdisk5 -p vscsi0 priority 1 Priority True gzaix:~# lspath -AE -l hdisk5 -p vscsi1 priority 1 Priority True gzaix:~# chpath -l hdisk5 -a priority=2 -p vscsi0 gzaix:~# lspath -AE -l hdisk5 -p vscsi1 priority 2 Priority True
b. Set up queue_depth to match the value on vio (the value is 20 if you have SVC disks on SDDPCM).
More info about queue depth here: http://www-01.ibm.com/support/docview.wss?uid=isg3T1012636
gzaix:~# chdev -l hdisk5 -a queue_depth=20 gzaix:~# lsattr -El hdisk5 -a queue_depth queue_depth 20 Queue DEPTH True
c. Set up hcheck_interval (usually to 60 seconds) to avoid enabling manually the paths when you take down the vios. If you have many disks/luns, this could be pain (you know where) so you must set-up this from the begining.
gzaix:~# chdev -l hdisk0 -a hcheck_interval=60 gzaix:~# lsattr -El hdisk5 -a hcheck_interval hcheck_interval 60 Health Check Interval True
On both chdev commands above, if you have your disks in use and you cannot put them offline, you add "-P" at the end and the modifications will be done in the next reboot.
Do not forget to reboot before taking off any vio.
If you ignore the step 9 and you put your disk in use, this is you will have to do:
- unmount the file systems after stopping any processes using them;
- deactivate any paging spaces in that volume group (swapoff or just use SMIT);
- deactivate the volume group using varyoffvg then run the chdev command without the "-P" flag;
- activate the volume group using varyonvg;
- mount the file systems, reactivate paging space if applicable.