RHEL8 - VDO(virtual data optimizer)

# vdo :  가상 데이터 최적화 도구

https://www.redhat.com/en/blog/introducing-virtual-data-optimizer-reduce-cloud-and-premise-storage-costs

 

[특징]

- 블록 단위 deduplication(중복 제거)

- compression(압축)

- thin-provisioning(씬프로비저닝)

 

[커널 구성요소]

kvdo - manages block storage and compression (데이터 압축을 투명하게 제어)

uds - manages the deduplication index (중복 제거)

 

[userspace 유틸]

vdo - vdo manager used to create, remove or modify volumes

vdostats - view volume stastics

 

 

[동작 순서]

1. 0 블록 제거

2. 중복 데이터 블록 제거

3. kvdo 커널 모듈이 LZ4 압축을 사용하여 데이터 블록을 압축하고 4KB 블록으로 그룹화

---

1.  설치

# yum install vdo kmod-kvdo -y
# systemctl enable vdo --now

2. create vdo volume for an XFS filesystem

# lsblk
# vdo create --name=vdo1 --device=/dev/vdb --vdoLogicalSize=300G
# mkfs.xfs -K /dev/mapper/vdo1
meta-data=/dev/mapper/vdo1       isize=512    agcount=4, agsize=6553600 blks
         =                       sectsz=4096  attr=2, projid32bit=1
         =                       crc=1        finobt=1, sparse=1, rmapbt=0
         =                       reflink=1
data     =                       bsize=4096   blocks=26214400, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
log      =internal log           bsize=4096   blocks=12800, version=2
         =                       sectsz=4096  sunit=1 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

 VDO uses thin-provisioning to help achieve space savings. VDO is able to present a device to the filesystem that is larger than the physical size, since deduplication maps multiple (duplicate) logical blocks to the same physical block.

3. create  and write duplicate file to vdo

# mkdir /data
# mount -o discard /dev/mapper/vdo1 /data
# touch /data/abcd

4. view vdo stastics to see the data efficiency

# cd /data
# dd if=/dev/urandom of=/data/file.1 bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB, 1000 MiB) copied, 20.6926 s, 50.7 MB/s

# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1         10.0G      5.0G      5.0G  49%            4%

# df -h|grep vdo
/dev/mapper/vdo1       100G  1.8G   99G   2% /data

# for i in 2 3 4 5; do echo "copy $i";cp file.1 file.$i; done
copy 2
copy 3
copy 4
copy 5

# sync
# df -h /dev/mapper/vdo1
Filesystem        Size  Used Avail Use% Mounted on
/dev/mapper/vdo1  100G  5.7G   95G   6% /data

# vdostats --human-readable
Device                    Size      Used Available Use% Space saving%
/dev/mapper/vdo1         10.0G      5.0G      5.0G  49%           80%