DRBD 기술노트
- Opensource 정보 : DRBD / DRBDutils (https://www.linbit.com/en/drbd-oss-distribution/)
- DRBD Stack (Active / Passive 구성임)
- 작업 절차중 나오는 용어들은 Mantec 을 참고해도 좋을듯 : https://mantech.jira.com/wiki/spaces/WDRBDV9/pages/170098908/1.
- Mantech에 MCCS라는 솔루션이 DRBD을 기반으로 상품화되었음
- DRBD Data Flow
DRBD설치하기
- DRBD 모듈 설치 (8.x, 9.x 설치 방법 동일함)
> tar -zxvf drbd-8.4.5.tar.gz > cd drbd-8.4.5 > make;make install > echo "modprobe drbd" > /etc/sysconfig/modules/drbd.modules > chmod +x /etc/sysconfig/modules/drbd.modules > modprobe drbd > lsmod | grep drbd drbd 568788 0 libcrc32c 12644 4 xfs,drbd,nf_nat,nf_conntrack - DRBD Util 설치 (8.x, 9.x 설치 방법 동일함)
> yum install -y libxslt libxml2 > tar -zxvf drbd-utils-8.9.1.tar.gz > cd drbd-utils-8.9.1 > ./configure --prefix=/usr/local/drbd --sysconfdir=/etc/ --with-initscripttype=sysv > ./configure --sysconfdir=/etc/ --with-initscripttype=systemd --with-pacemaker (Centos7) > make; make install > mkdir -p /usr/local/drbd/var/run
DRBD구성하기
- Target 파티션 구성하기
># fdisk /dev/xvdb Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xcce44d25. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. \\Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite) \\WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u'). \\Command (m for help): n Command action e extended p primary partition (1-4) p (입력) Partition number (1-4): 1 (입력) First cylinder (1-2080500, default 1): Using default value 1 (입력) Last cylinder, +cylinders or +size{K,M,G} (1-2080500, default 2080500): Using default value 2080500 \\Command (m for help): w (입력) The partition table has been altered! \\Calling ioctl() to re-read partition table. Syncing disks. - DRBD구성하기
># cat /etc/drbd.d/drbd.res resource drbd0 { startup { wfc-timeout 30; outdated-wfc-timeout 20; degr-wfc-timeout 30; } \\ syncer { rate 1000M; verify-alg sha1; } \\ on web1 { device drbd0; disk /dev/xvdb1; address {노드1번IP}:7789; meta-disk internal; } \\ on web2 { device drbd0; disk /dev/xvdb1; address {노드2번IP}:7789; meta-disk internal; } } - DRBD 메타 데이터 구성 후 서비스 시작
># drbdadm create-md drbd0 ># /etc/init.d/drbd start - Disk Sync 시작 (1번 서버에서 수행) (DRBD 8.x 해당)
># drbdadm -- --overwrite-data-of-peer primary drbd0 - drbd 실행 후 primary 지정 (1번서버에서 수행)
># drbdadm invalidate drbd0 ># drbdadm primary --force drbd0 - 데이터 동기화 상태 확인DRBD 서비스 시작 (DRBD 8.x 해당)
># cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by root@web2, 2015-03-09 14:49:34 0: cs:SyncTarget ro:Secondary/Primary ds:Inconsistent/UpToDate C r----- ns:0 nr:491520 dw:487424 dr:0 al:0 bm:0 lo:16 pe:0 ua:16 ap:0 ep:1 wo:f oos:9986956 [>....................] sync'ed: 4.7% (9752/10228)M finish: 0:04:05 speed: 40,616 (40,616) want: 102,400 K/sec - 연결상태 확인 (DRBD 9.0)
$> drbdadm status drbd0 drbd0 role:Primary disk:UpToDate repotx-gitlab-dev02.tx.skp role:Secondary replication:SyncSource peer-disk:Inconsistent done:8.53
DRBD 기능 테스트
- DRBD의 Primary 선언 (master 노드수행)
$> drbdadm primary drbd0 $> cat /proc/drbd *Primary/Secondary으로 표기되어 있으면 됨 version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by root@techtx-base-dev02, 2019-04-26 17:54:37 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:0 nr:0 dw:0 dr:664 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 - 파일시스템 생성 (master 노드수행)
># mkfs.ext4 /dev/drbd0 mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 327680 inodes, 1310595 blocks 65529 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1342177280 40 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736 \\Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done \\This filesystem will be automatically checked every 26 mounts or 180 days, whichever comes first. Use tune2fs -c or -i to override. - 마운트 후 파일 생성 (master 노드수행)
$> mount /dev/drbd0 /mnt/ $> df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/drbd0 4.8G 10M 4.6G 1% /mnt - 임의 파일 생성 (master 노드수행)
$> touch 123 $> ls -l 123 -rw-r--r-- 1 root root 0 Apr 29 13:04 123 - 마운트 해제 및 DRBD의 secondary 선언 (master 노드수행)
$> umount /mnt $> drbdadm secondary drbd0 $> cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by root@techtx-base-dev02, 2019-04-26 17:54:37 0: cs:Connected ro:Secondary/Secondary ds:UpToDate/UpToDate C r----- ns:1235836 nr:0 dw:1235836 dr:1409 al:307 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 - DRBD의 Pimary 선언 (slave 노드 수행)
$> drbdadm primary drbd0 $> cat /proc/drbd version: 8.4.5 (api:1/proto:86-101) GIT-hash: 1d360bde0e095d495786eaeb2a1ac76888e4db96 build by root@techtx-base-dev05, 2019-04-26 17:54:29 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:4 nr:1235836 dw:1235840 dr:1017 al:1 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:0 - 파일시스템 마운트 후 master노드에서 생성한 파일 확인 (slave 노드 수행)
># mount /dev/drbd0 /mnt/ ># ls -l /mnt/ total 1000020 -rw-r--r-- 1 root root 0 Apr 29 13:04 123
DRBD장애로 인한 데이터 동기화 실패대응
- crm_mon -1 명령어 수행을 통한 Active / Standby 정보 확인
- Corocync / Pacemaker / DRBD 서비스 종료
- DRBD 서비스 시작
- 슬레이브 선언 (Standby 서버에서 수행)
$> drbdadm secondary all $> drbdadm disconnect all $> drbdadm -- --discard-my-data connect all - 마스터 선언 (Active 서버에서 수행)
$> drbdadm primary all $> drbdadm disconnect all $>drbdadm connect all - 데이터 강제 동기화 (마스터에서 작업 수행)
$> drbdadm invalidate drbd0