리눅스에서 LVM 사용하기. lvm mirror는, sdb, sdc 두개의 디스크를 이용하였으며, sdb를 lvm으로 먼저 구성 한 이후, sdc를 기존 볼륨에 추가하여 mirror를 걸었다. OS 부트 영역(/boot)을 lvm으로 구성하여 부팅하는 방법은 좀 더 알아보아야 겠다.
mdadm으로 미러거는 방법은 solaris md와 비슷, lvm은 hpux lvm과 비슷함(사견임).
1. 기존 디스크 정보
sda(OS), sdb(linux raid), sdc(linux raid), sdd(data disk)
root@fox:~# fdisk -l
Disk /dev/sda: 146.8 GB, 146815737856 bytes
255 heads, 63 sectors/track, 17849 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000483edDevice Boot Start End Blocks Id System
/dev/sda1 * 1 32 248832 83 Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2 32 17850 143123457 5 Extended
/dev/sda5 32 17850 143123456 8e Linux LVMDisk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 fd Linux raid autodetectDisk /dev/sdc: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000Device Boot Start End Blocks Id System
/dev/sdc1 1 60801 488384001 fd Linux raid autodetectDisk /dev/sdd: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x123d28c0Device Boot Start End Blocks Id System
/dev/sdd1 1 60801 488384001 fd Linux raid autodetect
root@fox:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/fox-root 135113960 37217672 91032828 30% /
none 4092764 248 4092516 1% /dev
none 4097748 0 4097748 0% /dev/shm
none 4097748 108 4097640 1% /var/run
none 4097748 0 4097748 0% /var/lock
none 4097748 0 4097748 0% /lib/init/rw
/dev/sda1 233191 17475 203275 8% /boot
/dev/sdd1 480719056 260051128 196248728 57% /home2
2. sdb를 LVM으로 만들기.
2.1 fdisk /dev/sdb 실행하여, 파티션 타입을 Linux LVM으로 변경한다.
Command (m for help): d
Selected partition 1Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-60801, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-60801, default 60801):
Using default value 60801Command (m for help): p
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 83 LinuxCommand (m for help): t
Selected partition 1
Hex code (type L to list codes): LHex code (type L to list codes): 8e
Changed system type of partition 1 to 8e (Linux LVM)Command (m for help): p
Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x04129406Device Boot Start End Blocks Id System
/dev/sdb1 1 60801 488384001 8e Linux LVMCommand (m for help): w
The partition table has been altered!
2.2 physical volume 생성
root@fox:~# pvcreate /dev/sdb1
Can’t open /dev/sdb1 exclusively. Mounted filesystem?
mdadm 으로 미러 걸렸던 상황이라 pv가 생성되지 않는것 같다. 일단 아래 커맨드로 md를 정지 시킴.
root@fox:~# ls -al /dev/md0
brw-rw—- 1 root disk 9, 0 2012-07-31 13:46 /dev/md0
root@fox:~# mdadm –zero -superblock /dev/md0
mdadm: option -u not valid in misc mode
root@fox:~# mdadm –zero-superblock /dev/md0
mdadm: Unrecognised md component device – /dev/md0
root@fox:~# mdadm –manage /dev/md0 –fail /dev/sdb1
mdadm: cannot get array info for /dev/md0
root@fox:~# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: cannot get array info for /dev/md0
root@fox:~# mdadm –manage –stop /dev/md0
mdadm: stopped /dev/md0
root@fox:~# ls -al /dev/md0
다시 pvcreate
root@fox:~# pvcreate /dev/sdb1
Wiping software RAID md superblock on /dev/sdb1
Physical volume “/dev/sdb1” successfully created
2.3 volume group의 생성 (여기서는 vg01 이라는 이름으로 생성했다)
root@fox:~# vgcreate vg01 /dev/sdb1
Volume group “vg01” successfully createdroot@fox:~# vgdisplay
— Volume group —
VG Name vg01
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 465.76 GiB
PE Size 4.00 MiB
Total PE 119234
Alloc PE / Size 0 / 0
Free PE / Size 119234 / 465.76 GiB
VG UUID PYgBHB-oXKi-u1ss-9Hrd-fFuX-lp9u-Ntilev
2.4 logical volume을 새성(lvol1이라는 이름의 logical volume, 크기는 465.75GB로 생성).
root@fox:~# lvcreate -L465.76G -nlvol1 vg01
Rounding up size to full physical extent 465.76 GiB
Insufficient free extents (119234) in volume group vg01: 119235 requiredroot@fox:~# lvcreate -L465.75G -nlvol1 vg01
Logical volume “lvol1” createdroot@fox:~# lvdisplay
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 465.75 GiB
Current LE 119232
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2
2.5 만들어진 lvol1을 포맷하여 사용한다. ext4 로 포맷 하였음.
root@fox:~# mkfs -t ext4 /dev/vg01/lvol1
mke2fs 1.41.11 (14-Mar-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
30523392 inodes, 122093568 blocks
6104678 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
3726 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000Writing inode tables: 739/3726
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 37 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.root@fox:~# mount /dev/vg01/lvol1 /mnt
root@fox:~# df -h
Filesystem Size Used Avail Use% Mounted on
….. 생략…
/dev/mapper/vg01-lvol1 459G 198M 435G 1% /mnt
2.6. LV 크기 조정(lvextend, lvreduce)
크기를 조정할때에는, 데이타가 삭제되므로, 데이타를 백업 받은 후 작업해야 함.
root@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 456.75 GiB
Current LE 116928
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2
lvreduce로 10GB를 줄여본다. 이때, 데이타는 안전하지 않게 된다.
root@fox:~# lvreduce -L-10G /dev/vg01/lvol1
WARNING: Reducing active logical volume to 446.75 GiB
THIS MAY DESTROY YOUR DATA (filesystem etc.)
Do you really want to reduce lvol1? [y/n]: y
Reducing logical volume lvol1 to 446.75 GiB
Logical volume lvol1 successfully resized
root@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 446.75 GiB
Current LE 114368
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2root@fox:~# lvextend -L+10G /dev/vg01/lvol1
Extending logical volume lvol1 to 456.75 GiB
Logical volume lvol1 successfully resizedroot@fox:~# lvdisplay /dev/vg01/lvol1
— Logical volume —
LV Name /dev/vg01/lvol1
VG Name vg01
LV UUID 3kQlAi-2aAd-cqyz-QF7D-zQ33-8Jrb-eXbWKz
LV Write Access read/write
LV Status available
# open 0
LV Size 456.75 GiB
Current LE 116928
Segments 1
Allocation inherit
Read ahead sectors auto
– currently set to 256
Block device 251:2
크기 조정후 다시 화일시스템을 생성해야 한다.
3. lvm mirroring.
3.1 현재 lv상태 확인.
root@fox:/# pvs
PV VG Fmt Attr PSize PFree
/dev/sda5 fox lvm2 a- 136.49g 0
/dev/sdb1 vg01 lvm2 a- 465.76g 9.01groot@fox:/# vgs
VG #PV #LV #SN Attr VSize VFree
fox 1 2 0 wz–n- 136.49g 0
vg01 1 1 0 wz–n- 465.76g 9.01groot@fox:/# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 -wi-a- 456.75g /dev/sdb1(0)
3.2. vgextend로 새 디스크(sdc1)를 기존 볼륨그룹 vg01에 추가한다. 처음엔 vg01을 생성하는것으로 생각해서 에러가 발생 하였다.
root@fox:~# vgcreate vg01 /dev/sdc1
/dev/vg01: already exists in filesystem
New volume group name “vg01” is invalid
Run `vgcreate –help’ for more information.root@fox:~# vgextend vg01 /dev/sdc1
Volume group “vg01” successfully extendedroot@fox:~# pvs
PV VG Fmt Attr PSize PFree
/dev/sda5 fox lvm2 a- 136.49g 0
/dev/sdb1 vg01 lvm2 a- 465.76g 9.01g
/dev/sdc1 vg01 lvm2 a- 465.76g 465.76g
3.3. lvconvert로 미러를 만든다.(꽤 오래 걸림…)
root@fox:~# lvconvert -m 1 vg01/lvol1
Insufficient suitable allocatable extents for logical volume : 116928 more required
Unable to allocate extents for mirror(s).root@fox:~# lvconvert -m 1 –corelog vg01/lvol1
vg01/lvol1: Converted: 0.5%
vg01/lvol1: Converted: 0.9%
vg01/lvol1: Converted: 1.4%
… … …
vg01/lvol1: Converted: 99.6%
vg01/lvol1: Converted: 99.8%
vg01/lvol1: Converted: 100.0%
Logical volume lvol1 converted.root@fox:~# lvs -a
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
root fox -wi-ao 130.91g
swap_1 fox -wi-ao 5.58g
lvol1 vg01 mwi-ao 456.75g 100.00
[lvol1_mimage_0] vg01 iwi-ao 456.75g
[lvol1_mimage_1] vg01 iwi-ao 456.75g
3.4. Failover.
아래 커맨드로 에러 유발.
root@fox:/# dd if=/dev/zero of=/dev/sdb1 count=10
lv상태 확인
root@fox:/# lvs -a -o +devices
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 mwi-a- 456.75g 100.00 lvol1_mimage_0(0),lvol1_mimage_1(0)
[lvol1_mimage_0] vg01 iwi-ao 456.75g unknown device(0)
[lvol1_mimage_1] vg01 iwi-ao 456.75g /dev/sdc1(0)
/etc/lvm/archive 로 이동하여, 최근의 vg화일에서 uuid를 확인하거나, vgchange 커맨드로 uuid를 확인한다.
root@fox:/etc/lvm/archive# ls -al
total 52
drwx—— 2 root root 4096 2012-07-31 16:04 .
drwxr-xr-x 5 root root 4096 2012-07-31 14:37 ..
-rw——- 1 root root 805 2012-07-31 14:37 vg01_00000.vg
-rw——- 1 root root 813 2012-07-31 14:49 vg01_00001.vg
-rw——- 1 root root 1149 2012-07-31 14:59 vg01_00002.vg
-rw——- 1 root root 1150 2012-07-31 15:01 vg01_00003.vg
-rw——- 1 root root 1150 2012-07-31 15:05 vg01_00004.vg
-rw——- 1 root root 1143 2012-07-31 15:47 vg01_00005.vg
-rw——- 1 root root 1142 2012-07-31 15:58 vg01_00006.vg
-rw——- 1 root root 1388 2012-07-31 16:02 vg01_00007.vg
-rw——- 1 root root 1398 2012-07-31 16:04 vg01_00008.vg
-rw——- 1 root root 805 2012-07-31 15:41 vg02_00000.vg
-rw——- 1 root root 795 2012-07-31 15:48 vg02_00001.vgroot@fox:/# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
/dev/vg01/lvol1: stat failed: No such file or directory
Path /dev/vg01/lvol1 no longer valid for device(251,2)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilev3kQlAi2aAdcqyzQF7DzQ338JrbeXbWKz: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilev3kQlAi2aAdcqyzQF7DzQ338JrbeXbWKz no longer valid for device(251,2)
/dev/mapper/vg01-lvol1: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1 no longer valid for device(251,2)
/dev/disk/by-uuid/f0438f20-ed5b-4420-9a77-d135e095e0ae: stat failed: No such file or directory
Path /dev/disk/by-uuid/f0438f20-ed5b-4420-9a77-d135e095e0ae no longer valid for device(251,2)
/dev/block/251:2: stat failed: No such file or directory
Path /dev/block/251:2 no longer valid for device(251,2)
/dev/disk/by-id/dm-name-vg01-lvol1: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1 no longer valid for device(251,2)
/dev/mapper/vg01-lvol1_mimage_0: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1_mimage_0 no longer valid for device(251,3)
/dev/disk/by-id/dm-name-vg01-lvol1_mimage_0: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1_mimage_0 no longer valid for device(251,3)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevXPR0nFbDT2JhMlYskHgHd0A20nq6m6Ow: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevXPR0nFbDT2JhMlYskHgHd0A20nq6m6Ow no longer valid for device(251,3)
/dev/block/251:3: stat failed: No such file or directory
Path /dev/block/251:3 no longer valid for device(251,3)
/dev/mapper/vg01-lvol1_mimage_1: stat failed: No such file or directory
Path /dev/mapper/vg01-lvol1_mimage_1 no longer valid for device(251,4)
/dev/disk/by-id/dm-name-vg01-lvol1_mimage_1: stat failed: No such file or directory
Path /dev/disk/by-id/dm-name-vg01-lvol1_mimage_1 no longer valid for device(251,4)
/dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevmvAccUwkO7r6M7encYPcLSuizu3qCdYt: stat failed: No such file or directory
Path /dev/disk/by-id/dm-uuid-LVM-PYgBHBoXKiu1ss9HrdfFuXlp9uNtilevmvAccUwkO7r6M7encYPcLSuizu3qCdYt no longer valid for device(251,4)
/dev/block/251:4: stat failed: No such file or directory
Path /dev/block/251:4 no longer valid for device(251,4)
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)
동일한 커맨드를 한번 더 내렸더니 아래와 같은 결과가…
root@fox:/# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV‘.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)root@fox:/etc/lvm/archive# pvcreate –uuid “h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV” –restore /etc/lvm/archive/vg01_00007.vg /dev/sdb1
Couldn’t find device with uuid ‘h13ERD-b6yk-lYUK-n0JZ-MIWP-4Xzh-5BktUV’.
Physical volume “/dev/sdb1” successfully created
저기를 vg01_0008로 해야하나?
root@fox:/etc/lvm/archive# ls -al
total 52
drwx—— 2 root root 4096 2012-07-31 16:04 .
drwxr-xr-x 5 root root 4096 2012-07-31 14:37 ..
-rw——- 1 root root 805 2012-07-31 14:37 vg01_00000.vg
-rw——- 1 root root 813 2012-07-31 14:49 vg01_00001.vg
-rw——- 1 root root 1149 2012-07-31 14:59 vg01_00002.vg
-rw——- 1 root root 1150 2012-07-31 15:01 vg01_00003.vg
-rw——- 1 root root 1150 2012-07-31 15:05 vg01_00004.vg
-rw——- 1 root root 1143 2012-07-31 15:47 vg01_00005.vg
-rw——- 1 root root 1142 2012-07-31 15:58 vg01_00006.vg
-rw——- 1 root root 1388 2012-07-31 16:02 vg01_00007.vg
-rw——- 1 root root 1398 2012-07-31 16:04 vg01_00008.vg
-rw——- 1 root root 805 2012-07-31 15:41 vg02_00000.vg
-rw——- 1 root root 795 2012-07-31 15:48 vg02_00001.vg
vg01의 메타데이타를 저장.
root@fox:/etc/lvm/archive# vgcfgrestore vg01
Restored volume group vg01
확인.
root@fox:/etc/lvm/archive# lvs -a -o +devices
LV VG Attr LSize Origin Snap% Move Log Copy% Convert Devices
root fox -wi-ao 130.91g /dev/sda5(0)
swap_1 fox -wi-ao 5.58g /dev/sda5(33513)
lvol1 vg01 mwi-a- 456.75g 2.75 lvol1_mimage_0(0),lvol1_mimage_1(0)
[lvol1_mimage_0] vg01 Iwi-ao 456.75g /dev/sdb1(0)
[lvol1_mimage_1] vg01 Iwi-ao 456.75g /dev/sdc1(0)
lvs로 확인해보면 동기화가 진행되는것을 볼 수 있다.
root@fox:/etc/lvm/archive# lvs -a
LV VG Attr LSize Origin Snap% Move Log Copy% Convert
root fox -wi-ao 130.91g
swap_1 fox -wi-ao 5.58g
lvol1 vg01 mwi-a- 456.75g 8.28
[lvol1_mimage_0] vg01 Iwi-ao 456.75g
[lvol1_mimage_1] vg01 Iwi-ao 456.75groot@fox:/etc/lvm/archive# vgchange -an –partial
Partial mode. Incomplete volume groups will be activated read-only.
0 logical volume(s) in volume group “vg01” now active
Can’t deactivate volume group “fox” with 2 open logical volume(s)