구성환경
OS: AIX 6.1(6100-08-02-1316)
SAN: CISCO
Storage: EMC CX4-140
EMC SAN s/w SERVER
SPA------+----------+----HBA0
|
SPB------+
1. 사전작업
AIX MPIO 파일셋이 설치되어 있는지 확인하고, 설치되어 있으면 제거한다.
disk 확인
# lsdev -Cc disk hdisk0 Available 03-08-00-4,0 16 Bit LVD SCSI Disk Drive hdisk1 Available 0A-08-00-5,0 16 Bit LVD SCSI Disk Drive hdisk2 Available 0C-08-01 MPIO Other FC SCSI Disk Drive hdisk3 Available 0C-08-01 MPIO Other FC SCSI Disk Drive hdisk4 Available 0C-08-01 MPIO Other FC SCSI Disk Drive hdisk5 Available 0C-08-01 MPIO Other FC SCSI Disk Drive hdisk6 Available 0C-08-01 MPIO Other FC SCSI Disk Drive
storage 관련 디스크를 삭제한다.
# lsdev -Ctmpio* -Fname hdisk2 hdisk3 hdisk4 hdisk5 hdisk6 # lsdev -Ctmpio* -Fname | xargs -n1 rmdev -dl hdisk2 deleted hdisk3 deleted hdisk4 deleted hdisk5 deleted hdisk6 deleted
화일셋 확인
# lslpp -l | grep -i emc EMC.CELERRA.aix.rte 5.3.0.8 COMMITTED EMC CELERRA AIX Support EMC.CELERRA.iscsi.rte 5.3.0.8 COMMITTED EMC CELERRA iSCSI Support EMC.CLARiiON.aix.rte 5.3.0.8 COMMITTED EMC CLARiiON AIX Support EMC.CLARiiON.iscsi.rte 5.3.0.8 COMMITTED EMC CLARiiON iSCSI Support EMC.INVISTA.aix.rte 5.3.0.8 COMMITTED EMC INVISTA AIX Support EMC.Symmetrix.aix.rte 5.3.0.8 COMMITTED EMC Symmetrix AIX Support EMC.Symmetrix.ha.rte 5.3.0.8 BROKEN EMC Symmetrix HA Concurrent EMC.Symmetrix.iscsi.rte 5.3.0.8 COMMITTED EMC Symmetrix iSCSI Support EMC.CELERRA.aix.rte 5.3.0.8 COMMITTED EMC CELERRA AIX Support EMC.CELERRA.iscsi.rte 5.3.0.8 COMMITTED EMC CELERRA iSCSI Support EMC.CLARiiON.aix.rte 5.3.0.8 COMMITTED EMC CLARiiON AIX Support EMC.CLARiiON.iscsi.rte 5.3.0.8 COMMITTED EMC CLARiiON iSCSI Support EMC.INVISTA.aix.rte 5.3.0.8 COMMITTED EMC INVISTA AIX Support EMC.Symmetrix.aix.rte 5.3.0.8 COMMITTED EMC Symmetrix AIX Support EMC.Symmetrix.iscsi.rte 5.3.0.8 COMMITTED EMC Symmetrix iSCSI Support devices.common.IBM.modemcfg.data
위의 화일셋을 모두 삭제하고 다시 확인
# lslpp -l | grep -i emc devices.common.IBM.modemcfg.data
2. powerpath 설치
화일셋 위치: /file/powerpath
# cd /file/powerpath # installp -agXd . EMCpower ... Installation Summary -------------------- Name Level Part Event Result ------------------------------------------------------------------------------- EMCpower.base 5.3.1.0 USR APPLY SUCCESS EMCpower.mpx 5.3.1.0 USR APPLY SUCCESS EMCpower.migration_enabler 5.3.1.0 USR APPLY SUCCESS EMCpower.encryption 5.3.1.0 USR APPLY SUCCESS EMCpower.consistency_grp 5.3.0.0 USR APPLY SUCCESS #
3. 등록
# emcpreg -install =========== EMC PowerPath Registration =========== Do you have a new registration key or keys to enter?[n] XIPV-YFP3-H3RM-YTVG-AP9P-VXQP^C# emcpreg -install =========== EMC PowerPath Registration =========== Do you have a new registration key or keys to enter?[n] y Enter the registration keys(s) for your product(s), one per line, pressing Enter after each key. After typing all keys, press Enter again. Key (Enter if done): XXXX-xxxx-xxxx-xxxx-xxxx-xxxx 1 key(s) successfully added. Key successfully installed. Key (Enter if done): 1 key(s) successfully registered.
4. 설정
# powermt config
Method error (/etc/methods/cfgpower -l powerpath0 ):
0514-040 Error initializing a device into the kernel.
Unexpected error occured.
error 가 발생한다.
커널에 드라이버가 제대로 로드 되었는지 확인해보니, 드라이버가 제대로 로드되지 않았다.
# genkex | grep -i power
5036000 89000 /usr/lib/drivers/powerdd
# lsdev -Cc driver powerpath0 Defined PowerPath Control Device
재부팅해도 동일한 결과.
———————
최종적으로, 아래와 같은 화일셋만 설치하고 재부팅 하면 정상작동함을 확인 할 수 있다.
smitty 커맨드로 아래와 같은 화일셋만 다시 설치하였다.
# lslpp -l | grep -i emc EMC.CLARiiON.aix.rte 5.3.0.4 COMMITTED EMC CLARiiON AIX Support EMC.CLARiiON.fcp.rte 5.3.0.4 COMMITTED EMC CLARiiON FCP Support EMCpower.base 5.3.1.0 COMMITTED PowerPath Base Driver and EMCpower.encryption 5.3.1.0 COMMITTED PowerPath Encryption with RSA EMCpower.migration_enabler EMCpower.mpx 5.3.1.0 COMMITTED PowerPath Multi_Pathing EMC.CLARiiON.aix.rte 5.3.0.4 COMMITTED EMC CLARiiON AIX Support EMC.CLARiiON.fcp.rte 5.3.0.4 COMMITTED EMC CLARiiON FCP Support devices.common.IBM.modemcfg.data
커널에는 아래와 같은 모듈이 로드된다.
# genkex | grep -i power
4dc9000 7000 /usr/lib/drivers/powerdiskdd
4ccb000 89000 /usr/lib/drivers/powerdd
5. powermt 유틸리티 사용하기
powerpath 확인
# powermt display Symmetrix logical device count=0 CLARiiON logical device count=5 Hitachi logical device count=0 Invista logical device count=0 HP xp logical device count=0 Ess logical device count=0 HP HSx logical device count=0 ============================================================================== ----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------ ### HW Path Summary Total Dead IO/Sec Q-IOs Errors ============================================================================== 0 fscsi0 optimal 10 0 - 0 0
아래를 보면, hdisk2와 hdisk7이 powerdisk0 로 표시되며, 스토리지의 LUN 14번 임을 알 수 있다.
# powermt display dev=all Pseudo name=hdiskpower0 CLARiiON ID=APM00063706620 [aix1_192.168.0.210] Logical device ID=600601607F121A00126BFF6AB900E411 [LUN 14] state=alive; policy=CLAROpt; priority=0; queued-IOs=0 Owner: default=SP A, current=SP A Array failover mode: 1 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 0 fscsi0 hdisk2 SP A1 active alive 0 0 0 fscsi0 hdisk7 SP B1 active alive 0 0 ...
장애 테스트
여기서 실제로 서버에는 하나의 hba만 존재한다. 따라서, 스토리지 와 SAN 사이에 연결된 두개의 케이블중 하나를 제거 해 보면,
상태가 degraded로 바뀐 것을 볼 수 있다.
# powermt display Symmetrix logical device count=0 CLARiiON logical device count=5 Hitachi logical device count=0 Invista logical device count=0 HP xp logical device count=0 Ess logical device count=0 HP HSx logical device count=0 ============================================================================== ----- Host Bus Adapters --------- ------ I/O Paths ----- ------ Stats ------ ### HW Path Summary Total Dead IO/Sec Q-IOs Errors ============================================================================== 0 fscsi0 degraded 10 5 - 0 5
좀더 자세하게 보면, 스토리지의 SP B 의 상태가 dead 임을 볼 수 있다.
# powermt display dev=all Pseudo name=hdiskpower0 CLARiiON ID=APM00063706620 [aix1_192.168.0.210] Logical device ID=600601607F121A00126BFF6AB900E411 [LUN 14] state=alive; policy=CLAROpt; priority=0; queued-IOs=0 Owner: default=SP A, current=SP A Array failover mode: 1 ============================================================================== ---------------- Host --------------- - Stor - -- I/O Path - -- Stats --- ### HW Path I/O Paths Interf. Mode State Q-IOs Errors ============================================================================== 0 fscsi0 hdisk2 SP A1 active alive 0 0 0 fscsi0 hdisk7 SP B1 active dead 0 1
다시 제거했던 케이블을 다시 연결하면, fail-over 된 것을 확인 할 수 있다.
기타 커맨드들…
# powermt display ports
Storage class = CLARiiON
==============================================================================
----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Errors
==============================================================================
APM00063706620 SP B1 256 5 0 0 5
APM00063706620 SP A1 256 5 0 0 0
# powermt display port_mode
Storage class = Symmetrix
==============================================================================
----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Mode
==============================================================================
Storage class = CLARiiON
==============================================================================
----------- Storage System --------------- -- I/O Paths -- --- Stats ---
ID Interface Wt_Q Total Dead Q-IOs Mode
==============================================================================
APM00063706620 SP B1 256 5 0 0 Enabled
APM00063706620 SP A1 256 5 0 0 Enabled
# powermt check_registration
Key xxxx-xxxx-xxxx-xxxx-xxxx-xxxx
Product: PowerPath
Capabilities: All
참고문서:
http://unixadminguide.blogspot.kr/2012/07/steps-to-remove-powerpath-software.html
http://ke-liu.blogspot.kr/2011/01/emc-powerpath.html