리눅스 LVM, LVM mirror.

리눅스에서 LVM 사용하기. lvm mirror는, sdb, sdc 두개의 디스크를 이용하였으며, sdb를 lvm으로 먼저 구성 한 이후, sdc를 기존 볼륨에 추가하여 mirror를 걸었다. OS 부트 영역(/boot)을 lvm으로 구성하여 부팅하는 방법은 좀 더 알아보아야 겠다.

mdadm으로 미러거는 방법은 solaris md와 비슷, lvm은 hpux lvm과 비슷함(사견임).

1. 기존 디스크 정보

sda(OS), sdb(linux raid), sdc(linux raid), sdd(data disk)

root@fox:~# fdisk -l

Disk /dev/sda: 146.8 GB, 146815737856 bytes
255 heads, 63 sectors/track, 17849 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000483ed

Device Boot Start End Blocks Id System
/dev/sda1 * 1 32 248832 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 32 17850 143123457 5 Extended
/dev/sda5 32 17850 143123456 8e Linux LVM

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406

Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 fd Linux raid autodetect

Disk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000

Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 fd Linux raid autodetect

Disk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x123d28c0

Device Boot Start End Blocks Id System
/dev/sdd1 1 60801 488384001 fd Linux raid autodetect
root@fox:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/fox-root 135113960 37217672 91032828 30% /
none 4092764 248 4092516 1% /dev
none 4097748 0 4097748 0% /dev/shm
none 4097748 108 4097640 1% /var/run
none 4097748 0 4097748 0% /var/lock
none 4097748 0 4097748 0% /lib/init/rw
/dev/sda1 233191 17475 203275 8% /boot
/dev/sdd1 480719056 260051128 196248728 57% /home2

2. sdb를 LVM으로  만들기.

2.1 fdisk /dev/sdb  실행하여, 파티션 타입을 Linux LVM으로 변경한다.

Command (m for help): d
Selected partition 1

Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-60801, default 60801):
Using default value 60801

Command (m for help): p

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406

Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 83 Linux

Command (m for help): t
Selected partition 1
Hex code (type L to list codes): L

Hex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)

Command (m for help): p

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406

Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 8e Linux LVM

Command (m for help): w
The partition table has been altered!

2.2 physical volume 생성

root@fox:~# pvcreate /dev/sdb1
Can’t open /dev/sdb1 exclusively. Mounted filesystem?

mdadm 으로 미러 걸렸던 상황이라 pv가 생성되지 않는것 같다. 일단 아래 커맨드로 md를 정지 시킴.

root@fox:~# ls -al /dev/md0
brw-rw—- 1 root disk 9, 0 2012-07-31 13:46 /dev/md0
root@fox:~# mdadm –zero -superblock /dev/md0
mdadm: option -u not valid in misc mode
root@fox:~# mdadm –zero-superblock /dev/md0
mdadm: Unrecognised md component device – /dev/md0
root@fox:~# mdadm –manage /dev/md0 –fail /dev/sdb1
mdadm: cannot get array info for /dev/md0
root@fox:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: cannot get array info for /dev/md0
root@fox:~# mdadm –manage –stop /dev/md0
mdadm: stopped /dev/md0
root@fox:~# ls -al /dev/md0

다시 pvcreate

root@fox:~# pvcreate /dev/sdb1
Wiping software RAID md superblock on /dev/sdb1
Physical volume “/dev/sdb1” successfully created

2.3 volume group의 생성 (여기서는 vg01 이라는 이름으로 생성했다)

root@fox:~# vgcreate vg01 /dev/sdb1
Volume group “vg01” successfully created

root@fox:~# vgdisplay
— Volume group —
VG Name vg01
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 465.76 GiB
PE Size 4.00 MiB
Total PE 119234
Alloc PE / Size 0 / 0
Free PE / Size 119234 / 465.76 GiB
VG UUID PYgBHB-oXKi-u1ss-9Hrd-fFuX-lp9u-Ntilev

2.4 logical volume을 새성(lvol1이라는 이름의 logical volume, 크기는 465.75GB로 생성).

root@fox:~# lvcreate -L465.76G -nlvol1 vg01
Rounding up size to full physical extent 465.76 GiB
Insufficient free extents (119234) in volume group vg01: 119235 required

root@fox:~# lvcreate -L465.75G -nlvol1 vg01
Logical volume “lvol1” created

root@fox:~# lvdisplay
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 465.75 GiB
Current LE 119232
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2

2.5 만들어진 lvol1을 포맷하여 사용한다. ext4 로 포맷 하였음.

root@fox:~# mkfs -t ext4 /dev/vg01/lvol1
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
30523392 inodes, 122093568 blocks
6104678 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
3726 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000

Writing inode tables: 739/3726

Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

root@fox:~# mount /dev/vg01/lvol1 /mnt
root@fox:~# df -h
Filesystem Size Used Avail Use% Mounted on
….. 생략…
/dev/mapper/vg01-lvol1  459G   198M   435G   1%   /mnt

2.6. LV 크기 조정(lvextend, lvreduce)

크기를 조정할때에는, 데이타가 삭제되므로, 데이타를 백업 받은 후 작업해야 함.

root@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 456.75 GiB
Current LE 116928
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2

lvreduce로 10GB를 줄여본다. 이때, 데이타는 안전하지 않게 된다.

root@fox:~# lvreduce -L-10G /dev/vg01/lvol1
WARNING: Reducing active logical volume to 446.75 GiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lvol1? [y/n]: y
Reducing logical volume lvol1 to 446.75 GiB
Logical volume lvol1 successfully resized
root@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 446.75 GiB
Current LE 114368
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2

root@fox:~# lvextend -L+10G /dev/vg01/lvol1
Extending logical volume lvol1 to 456.75 GiB
Logical volume lvol1 successfully resized

root@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 456.75 GiB
Current LE 116928
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2

크기 조정후 다시 화일시스템을 생성해야 한다.

3. lvm mirroring.

3.1 현재 lv상태 확인.

root@fox:/# pvs
PV VG Fmt Attr PSize PFree
/dev/sda5 fox lvm2 a- 136.49g 0
/dev/sdb1 vg01 lvm2 a- 465.76g 9.01g

root@fox:/# vgs
VG #PV #LV #SN Attr VSize VFree
fox 1 2 0 wz–n- 136.49g 0
vg01 1 1 0 wz–n- 465.76g 9.01g

root@fox:/# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 -wi-a- 456.75g /dev/sdb1(0)

3.2. vgextend로 새 디스크(sdc1)를 기존 볼륨그룹 vg01에 추가한다. 처음엔 vg01을 생성하는것으로 생각해서 에러가 발생 하였다.

root@fox:~# vgcreate vg01 /dev/sdc1
/dev/vg01: already exists in filesystem
New volume group name “vg01” is invalid
Run `vgcreate –help’ for more information.

root@fox:~# vgextend vg01 /dev/sdc1
Volume group “vg01” successfully extended

root@fox:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda5 fox lvm2 a- 136.49g 0
/dev/sdb1 vg01 lvm2 a- 465.76g 9.01g
/dev/sdc1 vg01 lvm2 a- 465.76g 465.76g

3.3.  lvconvert로 미러를 만든다.(꽤 오래 걸림…)

root@fox:~# lvconvert -m 1 vg01/lvol1
Insufficient suitable allocatable extents for logical volume : 116928 more required
Unable to allocate extents for mirror(s).

root@fox:~# lvconvert -m 1 –corelog vg01/lvol1
vg01/lvol1: Converted: 0.5%
vg01/lvol1: Converted: 0.9%
vg01/lvol1: Converted: 1.4%
… … …
vg01/lvol1: Converted: 99.6%
vg01/lvol1: Converted: 99.8%
vg01/lvol1: Converted: 100.0%
Logical volume lvol1 converted.

root@fox:~# lvs -a
LV                                   VG              Attr          LSize          Origin     Snap%    Move     Log    Copy%      Convert
root                                fox            -wi-ao        130.91g
swap_1                          fox            -wi-ao        5.58g
lvol1                               vg01         mwi-ao     456.75g                                                                 100.00
[lvol1_mimage_0]   vg01          iwi-ao      456.75g
[lvol1_mimage_1]   vg01           iwi-ao      456.75g

3.4.  Failover.

아래 커맨드로 에러 유발.

root@fox:/# dd if=/dev/zero of=/dev/sdb1 count=10

lv상태 확인

root@fox:/# lvs -a -o +devices
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 mwi-a- 456.75g 100.00 lvol1_mimage_0(0),lvol1_mimage_1(0)
[lvol1_mimage_0] vg01 iwi-ao 456.75g unknown device(0)
[lvol1_mimage_1] vg01 iwi-ao 456.75g /dev/sdc1(0)

 

/etc/lvm/archive 로 이동하여, 최근의 vg화일에서 uuid를 확인하거나, vgchange 커맨드로 uuid를 확인한다.

root@fox:/etc/lvm/archive# ls -al
total 52
drwx—— 2 root root 4096 2012-07-31 16:04 .
drwxr-xr-x 5 root root 4096 2012-07-31 14:37 ..
-rw——- 1 root root 805 2012-07-31 14:37 vg01_00000.vg
-rw——- 1 root root 813 2012-07-31 14:49 vg01_00001.vg
-rw——- 1 root root 1149 2012-07-31 14:59 vg01_00002.vg
-rw——- 1 root root 1150 2012-07-31 15:01 vg01_00003.vg
-rw——- 1 root root 1150 2012-07-31 15:05 vg01_00004.vg
-rw——- 1 root root 1143 2012-07-31 15:47 vg01_00005.vg
-rw——- 1 root root 1142 2012-07-31 15:58 vg01_00006.vg
-rw——- 1 root root 1388 2012-07-31 16:02 vg01_00007.vg
-rw——- 1 root root 1398 2012-07-31 16:04 vg01_00008.vg
-rw——- 1 root root 805 2012-07-31 15:41 vg02_00000.vg
-rw——- 1 root root 795 2012-07-31 15:48 vg02_00001.vg

root@fox:/# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
/dev/vg01/lvol1: stat failed: No such file or directory
Path /dev/vg01/lvol1 no longer valid for device(251,2)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilev3kQlAi2aAdcqyzQF7DzQ338JrbeXbWKz: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilev3kQlAi2aAdcqyzQF7DzQ338JrbeXbWKz no longer valid for device(251,2)
/dev/mapper/vg01-lvol1: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1 no longer valid for device(251,2)
/dev/disk/by-uuid/f0438f20-ed5b-4420-9a77-d135e095e0ae: stat failed: No such file or directory
Path /dev/disk/by-uuid/f0438f20-ed5b-4420-9a77-d135e095e0ae no longer valid for device(251,2)
/dev/block/251:2: stat failed: No such file or directory
Path /dev/block/251:2 no longer valid for device(251,2)
/dev/disk/by-id/dm-name-vg01-lvol1: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1 no longer valid for device(251,2)
/dev/mapper/vg01-lvol1_mimage_0: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1_mimage_0 no longer valid for device(251,3)
/dev/disk/by-id/dm-name-vg01-lvol1_mimage_0: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1_mimage_0 no longer valid for device(251,3)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevXPR0nFbDT2JhMlYskHgHd0A20nq6m6Ow: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevXPR0nFbDT2JhMlYskHgHd0A20nq6m6Ow no longer valid for device(251,3)
/dev/block/251:3: stat failed: No such file or directory
Path /dev/block/251:3 no longer valid for device(251,3)
/dev/mapper/vg01-lvol1_mimage_1: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1_mimage_1 no longer valid for device(251,4)
/dev/disk/by-id/dm-name-vg01-lvol1_mimage_1: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1_mimage_1 no longer valid for device(251,4)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevmvAccUwkO7r6M7encYPcLSuizu3qCdYt: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevmvAccUwkO7r6M7encYPcLSuizu3qCdYt no longer valid for device(251,4)
/dev/block/251:4: stat failed: No such file or directory
Path /dev/block/251:4 no longer valid for device(251,4)
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)

동일한 커맨드를 한번 더 내렸더니 아래와 같은 결과가…

root@fox:/# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV‘.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)

root@fox:/etc/lvm/archive# pvcreate –uuid “h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV” –restore /etc/lvm/archive/vg01_00007.vg /dev/sdb1
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Physical volume “/dev/sdb1” successfully created

저기를 vg01_0008로 해야하나?

root@fox:/etc/lvm/archive# ls -al
total 52
drwx—— 2 root root 4096 2012-07-31 16:04 .
drwxr-xr-x 5 root root 4096 2012-07-31 14:37 ..
-rw——- 1 root root 805 2012-07-31 14:37 vg01_00000.vg
-rw——- 1 root root 813 2012-07-31 14:49 vg01_00001.vg
-rw——- 1 root root 1149 2012-07-31 14:59 vg01_00002.vg
-rw——- 1 root root 1150 2012-07-31 15:01 vg01_00003.vg
-rw——- 1 root root 1150 2012-07-31 15:05 vg01_00004.vg
-rw——- 1 root root 1143 2012-07-31 15:47 vg01_00005.vg
-rw——- 1 root root 1142 2012-07-31 15:58 vg01_00006.vg
-rw——- 1 root root 1388 2012-07-31 16:02 vg01_00007.vg
-rw——- 1 root root 1398 2012-07-31 16:04 vg01_00008.vg
-rw——- 1 root root 805 2012-07-31 15:41 vg02_00000.vg
-rw——- 1 root root 795 2012-07-31 15:48 vg02_00001.vg

vg01의 메타데이타를 저장.

root@fox:/etc/lvm/archive# vgcfgrestore vg01
Restored volume group vg01

확인.

root@fox:/etc/lvm/archive# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 mwi-a- 456.75g 2.75 lvol1_mimage_0(0),lvol1_mimage_1(0)
[lvol1_mimage_0] vg01 Iwi-ao 456.75g /dev/sdb1(0)
[lvol1_mimage_1] vg01 Iwi-ao 456.75g /dev/sdc1(0)

lvs로 확인해보면 동기화가 진행되는것을 볼 수 있다.

root@fox:/etc/lvm/archive# lvs -a
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
root fox -wi-ao 130.91g
swap_1 fox -wi-ao 5.58g
lvol1 vg01 mwi-a- 456.75g                              8.28
[lvol1_mimage_0] vg01 Iwi-ao 456.75g
[lvol1_mimage_1] vg01 Iwi-ao 456.75g

root@fox:/etc/lvm/archive# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)

댓글 남기기

Your email address will not be published.

%d 블로거가 이것을 좋아합니다: