架构图
1、本节架构 2、安装Nginx和Keepalived 3、配置Nginx 4、添加监控端口脚本 5、配置keepalived主 6、配置keepalived备 7、启动服务并检查
1、本节架构
2、安装Nginx和Keepalived以下部署在hdss7-11.host.com及hdss7-12.host.comyum install nginx keepalived -y
3、配置Nginx
cat >> /etc/nginx/nginx.conf <<'eof' stream { upstream kube-apiserver { server 10.4.7.21:6443 max_fails=3 fail_timeout=30s; server 10.4.7.22:6443 max_fails=3 fail_timeout=30s; } server { listen 7443; proxy_connect_timeout 2s; proxy_timeout 900s; proxy_pass kube-apiserver; } } eof nginx -t
由于是基于四层负载均衡,所以stream要放在http段外面
4、添加监控端口脚本
cat > /etc/keepalived/check_port.sh <<'EOF' #!/bin/bash # keepalived 监控端口脚本 CHK_PORT=$1 if [ -n "$CHK_PORT" ];then PORT_PROCESS=`ss -lnt|grep $CHK_PORT |wc -l` if [ $PORT_PROCESS -eq 0 ];then echo "Port $CHK_PORT Is Not Used,End." exit 1 fi else echo "Check Port Cant Be Empty!" fi EOF chmod +x /etc/keepalived/check_port.sh
5、配置keepalived主
cat > /etc/keepalived/keepalived.conf <<eof ! Configuration File for keepalived global_defs { router_id 10.4.7.11 } vrrp_script chk_nginx { script "/etc/keepalived/check_port.sh 7443" interval 2 weight -20 } vrrp_instance VI_1 { state MASTER interface ens33 virtual_router_id 251 priority 100 advert_int 1 mcast_src_ip 10.4.7.11 nopreempt authentication { auth_type PASS auth_pass 11111111 } track_script { chk_nginx } virtual_ipaddress { 10.4.7.10 } } eof
nopreempt非抢占式,即主nginx宕机,备接管vip,主nginx恢复,主vip不会恢复,需重启从keepalived,主vip才会恢复
6、配置keepalived备
cat > /etc/keepalived/keepalived.conf <<'eof' ! Configuration File for keepalived global_defs { router_id 10.4.7.12 } vrrp_script chk_nginx { script "/etc/keepalived/check_port.sh 7443" interval 2 weight -20 } vrrp_instance VI_1 { state BACKUP interface ens33 virtual_router_id 251 mcast_src_ip 10.4.7.12 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 11111111 } track_script { chk_nginx } virtual_ipaddress { 10.4.7.10 } } eof
7、启动服务并检查
systemctl start nginx keepalived systemctl enable nginx keepalived systemctl status keepalived netstat -lntup|grep nginx tcp 0 0 0.0.0.0:7443 0.0.0.0:* LISTEN 13779/nginx: master tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 13779/nginx: master tcp6 0 0 :::80 :::* LISTEN 13779/nginx: master ip addr | tail -8 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:d7:a2:c7 brd ff:ff:ff:ff:ff:ff inet 10.4.7.11/24 brd 10.4.7.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet 10.4.7.10/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::20c:29ff:fed7:a2c7/64 scope link valid_lft forever preferred_lft forever