# keepalived기반의 DB이중화 구현

### 시스템 구성도

1. 서버정보  
    [![image.png](http://igoni.kr/uploads/images/gallery/2024-01/scaled-1680-/DVpimage.png)](http://igoni.kr/uploads/images/gallery/2024-01/DVpimage.png)
2. 로드밸런싱 종류 <div><table border="1" style="width: 90.6166%; height: 361.36px; border-collapse: collapse; border-style: groove; border-width: 1px;"><thead><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">**알고리즘 종류**</td><td style="width: 67.2107%; height: 29.7969px;">**알고리즘 설명**</td><td style="width: 8.23086%; height: 29.7969px;">**설정값**</td></tr></thead><tbody><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">라운드 로빈</td><td style="width: 67.2107%; height: 29.7969px;">하나씩 분배</td><td style="width: 8.23086%; height: 29.7969px;">rr</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">가중치 라운드 로빈</td><td style="width: 67.2107%; height: 29.7969px;">가중치가 큰 서버로 분배</td><td style="width: 8.23086%; height: 29.7969px;">wrr</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">최소 연결</td><td style="width: 67.2107%; height: 29.7969px;">접속수가 적은 서버로 분배</td><td style="width: 8.23086%; height: 29.7969px;">lc</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">가중치 최소연결</td><td style="width: 67.2107%; height: 29.7969px;">(접속수+1)/가중치 로 최소값 서버로 분배</td><td style="width: 8.23086%; height: 29.7969px;">wlc</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">지역 최소연결</td><td style="width: 67.2107%; height: 29.7969px;">접속수가 가중치를 넘기기전까지 한서버만 분배</td><td style="width: 8.23086%; height: 29.7969px;">lblc</td></tr><tr style="height: 46.5938px;"><td style="width: 24.4314%; height: 46.5938px;">복제기반의 지역 최소연결</td><td style="width: 67.2107%; height: 46.5938px;">가중치를 넘고 있을때 접속수가 가장 적은 서버 선택 (기본작동은 lblc와 동일)</td><td style="width: 8.23086%; height: 46.5938px;">lblcr</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">목적지 해시</td><td style="width: 67.2107%; height: 29.7969px;">목적지 IP로 부터 해시값 계산해서 분배</td><td style="width: 8.23086%; height: 29.7969px;">dh</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">소스 해시</td><td style="width: 67.2107%; height: 29.7969px;">소스 IP로 부터 해시값 계산해서 분배</td><td style="width: 8.23086%; height: 29.7969px;">sh</td></tr><tr style="height: 46.5938px;"><td style="width: 24.4314%; height: 46.5938px;">응답속도</td><td style="width: 67.2107%; height: 46.5938px;">응답속도(Short Expected Delay)가 가장 짧은 서버로 분배  
    (실제로는 접속수가 가장 적은 서버로 분배)</td><td style="width: 8.23086%; height: 46.5938px;">sed</td></tr><tr style="height: 29.7969px;"><td style="width: 24.4314%; height: 29.7969px;">큐 없음</td><td style="width: 67.2107%; height: 29.7969px;">Active 접속수r가 0인 서버를 우선 분배(기본작동은 sed알고리즘과 동일)</td><td style="width: 8.23086%; height: 29.7969px;">nq</td></tr></tbody></table>
    
    </div>

### 시스템 설치(전체서버)

1. 패키지 설치  
    ```Python
    $> yum install keepalived ipvsadm -y
    ```
    
      
    <div><div class="box"><div class="code">  
    </div></div></div>
2. keepalived 설정  
    ```Python
    
    
    $> vi /etc/keepalived/keepalived.conf
    global_defs {
      router_id MARIA_HA
      script_user root
    }
    
    vrrp_instance DB_HA {
      state BACKUP
      interface eth0
      virtual_router_id 10
      priority 100
      lvs_sync_daemon_interface eth0
    
      authentication {
        auth_type PASS
        auth_pass mariadb
     }
    
      virtual_ipaddress {
       172.21.115.207
     }
    }
    
    virtual_server 172.21.115.217 3306 {
      delay_loop 3
      lb_algo dh
      protocol TCP
      real_server 172.21.115.218 3306 {
        weight 100
        MISC_CHECK {
          misc_path "/svc/mysql_check.sh 172.21.115.218"
          misc_timeout 3
       }
     }
      real_server 172.21.115.215 3306 {
        weight 100
        MISC_CHECK {
          misc_path "/svc/mysql_check.sh 172.21.115.215"
          misc_timeout 3
       }
     }
    }
    ```
    
      
    <div><div class="box"><div class="code"><div class="linenoswrapper">  
    </div></div></div></div>
3. 모니터 스크립트 설정 ```Python
    $> vi /svc/mysql_check.sh
    
    #!/bin/bash
    
    /svc/mysql/bin/mysql -u monitor -pmonitor -h $1 -Nse 'select now()'
    
    $> chmod 755 /svc/mysql_check.sh
    ```

### 시스템 설치(DB#1서버)

1. 접근을 위한 db계정설정 ```SQL
    MariaDB [(none)]> grant process on *.* to  monitor@'172.21.115.215' identified by 'monitor' with grant option;
    MariaDB [(none)]> grant process on *.* to  monitor@'172.21.115.218' identified by 'monitor' with grant option;
    MariaDB [(none)]> flush privileges;
    ```
2. 커널 파라미터 수정 후 적용 ```Python
    $> vi /etc/sysctl.conf
    ...
    net.ipv4.ip_forward = 1
    ...
    $> sysctl -p
    ```
3. 서비스 실행 후 VIP할당여부 확인 ```Python
    $> systemctl start keepalived
    $> ip addr show | grep 32
        inet 172.21.115.217/32 scope global eth0   #실제로 Active된 노드에서만 VIP 할당됨
    ```
    
    <div><div class="box"><div class="code"><div class="linenoswrapper"><div class="linenos">  
    </div><div>  
    </div></div></div></div></div>

### keepalived 운영

1. 모니터링 방법  
    ```Python
    $> ipvsadm -Ln
    
    IP Virtual Server version 1.2.1 (size=4096)
    Prot LocalAddress:Port Scheduler Flags
      -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
    TCP  172.21.115.217:3306 sh
      -> 172.21.115.215:3306          Masq    100    2          0
      -> 172.21.115.218:3306          Masq    100    50         0
    ```
    
      
    <div><div class="box"><div class="code"><div class="linenoswrapper">  
    </div></div></div></div>
2. ​​​​​​현재 연결된 세션 정보 확인 ```Python
    $> ipvsadm -Lnc
    IPVS connection entries
    pro expire state       source             virtual            destination
    TCP 14:56  ESTABLISHED 172.21.114.22:52950 172.21.115.217:3306 172.21.115.218:3306
    TCP 14:56  ESTABLISHED 172.21.114.22:52968 172.21.115.217:3306 172.21.115.218:3306
    TCP 14:56  ESTABLISHED 172.21.114.22:52984 172.21.115.217:3306 172.21.115.218:3306
    TCP 14:54  ESTABLISHED 172.21.114.22:52928 172.21.115.217:3306 172.21.115.218:3306
    TCP 14:56  ESTABLISHED 172.21.114.22:52980 172.21.115.217:3306 172.21.115.218:3306
    ```
    
    <div><div class="box"><div class="code"><div class="linenoswrapper"><div class="linenos">  
    </div><div>  
    </div></div></div></div></div>

### 장애발생시 조치

1. 장애구현 (DB#1 Mariadb 접근 불가시)
    
    ```Python
    # Keepalived에서 Mysql 접근 불가 확인
    
    $> vi /var/log/messages
    ...
    Mar 11 14:36:43 dbms01 Keepalived_healthcheckers[666145]: Misc check for [[172.21.115.218]:tcp:3306 VS [172.21.115.217]:tcp:3306] by [/svc/mysql_check.sh] timed out with retry disabled.
    Mar 11 14:36:43 dbms01 Keepalived_healthcheckers[666145]: Removing service [172.21.115.218]:tcp:3306 to VS [172.21.115.217]:tcp:3306
    ...
    ```
    
    <div><div><div><div class="box">  
    </div></div><div><div class="box"><div class="code"><div class="linenoswrapper"><div class="linenos">  
    </div><div>\# ipvsadm에서 장애발생서버 제외  
    </div></div></div></div></div></div></div>```Python
    $> ipvsadm
    IP Virtual Server version 1.2.1 (size=4096)
    Prot LocalAddress:Port Scheduler Flags
      -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
    TCP  172.21.115.217:3306 sh
      -> 172.21.115.215:3306          Masq    100    33         13
    ```
    
    <div><div><div class="box"><div class="code"><div class="linenoswrapper"><div>  
    </div><div>  
    </div></div></div></div></div></div>
2. <div><div><div><div class="box"><div class="code"><div class="linenoswrapper"><div class="linenos">장애구현 (DB#1 서버 장애시)  
    </div></div></div></div></div></div></div>```Python
    #master 서버
    $> vi /var/log/messages
    Mar 11 14:19:21 dbms01 systemd[1]: Stopping LVS and VRRP High Availability Monitor...
    Mar 11 14:19:21 dbms01 Keepalived[664420]: Stopping
    Mar 11 14:19:21 dbms01 Keepalived_healthcheckers[664421]: Shutting down service [172.21.115.218]:tcp:3306 from VS [172.21.115.217]:tcp:3306
    Mar 11 14:19:21 dbms01 kernel: IPVS: stopping master sync thread 664516 ...
    Mar 11 14:19:21 dbms01 Keepalived_vrrp[664422]: (DB_HA) sent 0 priority
    Mar 11 14:19:21 dbms01 Keepalived_vrrp[664422]: (DB_HA) removing VIPs.
    Mar 11 14:19:21 dbms01 Keepalived_healthcheckers[664421]: Shutting down service [172.21.115.215]:tcp:3306 from VS [172.21.115.217]:tcp:3306
    Mar 11 14:19:21 dbms01 Keepalived_healthcheckers[664421]: Stopped - used 0.011017 user time, 0.141106 system time
    Mar 11 14:19:22 dbms01 Keepalived_vrrp[664422]: Stopped - used 0.011218 user time, 0.026105 system time
    Mar 11 14:19:22 dbms01 Keepalived[664420]: CPU usage (self/children) user: 0.000000/1.222181 system: 0.001086/1.218784
    Mar 11 14:19:22 dbms01 Keepalived[664420]: Stopped Keepalived v2.1.5 (07/13,2020)
    Mar 11 14:19:22 dbms01 systemd[1]: keepalived.service: Succeeded.
    Mar 11 14:19:22 dbms01 systemd[1]: Stopped LVS and VRRP High Availability Monitor.
    1
    2
    3
    4
    5
    6
    7
    8
    
    #Standby 서버
    $> vi /var/log/messages
    Mar 11 14:20:41 dbms02 Keepalived_vrrp[666146]: (DB_HA) Backup received priority 0 advertisement
    Mar 11 14:20:42 dbms02 Keepalived_vrrp[666146]: (DB_HA) Receive advertisement timeout
    Mar 11 14:20:42 dbms02 Keepalived_vrrp[666146]: (DB_HA) Entering MASTER STATE
    Mar 11 14:20:42 dbms02 Keepalived_vrrp[666146]: (DB_HA) setting VIPs.
    Mar 11 14:20:42 dbms02 Keepalived_vrrp[666146]: (DB_HA) Sending/queueing gratuitous ARPs on eth0 for 172.21.115.217
    Mar 11 14:20:42 dbms02 Keepalived_vrrp[666146]: Sending gratuitous ARP on eth0 for 172.21.115.217
    ```
    
    <div><div><div class="box"><div class="code"><div class="linenoswrapper"><div class="linenos">  
    </div></div></div></div></div></div>

*reference*

- *<span class="wikiexternallink">[<span class="wikigeneratedlinkcontent">https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/7/html/load\_balancer\_administration/index//</span>](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/load_balancer_administration/index//)</span>*
- *<span class="wikiexternallink">[<span class="wikigeneratedlinkcontent">https://www.slideshare.net/neoclova/maria-db-250919306//</span>](https://www.slideshare.net/neoclova/maria-db-250919306//)</span>*