Kubernetes(k8s)1.14 离线版集群 - 高可用kube-apiserver+部署KeepLived

mac2024-03-12 26

声明：如果您有更好的技术与作者分享，或者商业合作；请访问作者个人网站 http://www.esqabc.com/view/message.html 留言给作者。如果该案例触犯您的专利，请在这里：http://www.esqabc.com/view/message.html 留言给作者说明原由作者一经查实，马上删除。

. .

1、搭建前说明：

. a、前提提条件、服务器，请查看这个地址：https://blog.csdn.net/esqabc/article/details/102726771 .

2、搭建kube-apiserver 高可用

使用Nginx 4层实现k8s节点(master节点和worker节点)高可用，访问kube-apiserver的步骤注意：搭建服务器，没有特殊说明，一般默认在：k8s-01 操作

a、下载编译nginx （1）把下载好的文件上传到：/opt/k8s/work

[root@k8s-01 ~]# cd /opt/k8s/work 解压 [root@k8s-01 work]# tar -xzvf nginx-1.15.3.tar.gz 编译 [root@k8s-01 work]# cd /opt/k8s/work/nginx-1.15.3 [root@k8s-01 nginx-1.15.3]# mkdir nginx-prefix . 注意，下面命令有一个点的 . [root@k8s-01 nginx-1.15.3]# ./configure --with-stream --without-http --prefix=$(pwd)/nginx-prefix --without-http_uwsgi_module [root@k8s-01 nginx-1.15.3]# make && make install

说明一下：

without-http_scgi_module --without-http_fastcgi_modulewith-stream：开启 4 层透明转发(TCP Proxy)功能；without-xxx：关闭所有其他功能，这样生成的动态链接二进制程序依赖最小；

（2）创建目录结构

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# source /opt/k8s/bin/environment.sh

for node_ip in ${NODE_IPS[@]} do echo ">>> ${node_ip}" mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin} done

（3）分发到其他主机

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${NODE_IPS[@]} do echo ">>> ${node_ip}" scp /opt/k8s/work/nginx-1.15.3/nginx-prefix/sbin/nginx root@${node_ip}:/opt/k8s/kube-nginx/sbin/kube-nginx ssh root@${node_ip} "chmod a+x /opt/k8s/kube-nginx/sbin/*" ssh root@${node_ip} "mkdir -p /opt/k8s/kube-nginx/{conf,logs,sbin}" sleep 3 done

注意：如果出现下面错误，执行上面命令重复执行一次就OK：出现下图就代表成功：（4）配置Nginx文件

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# cat > kube-nginx.conf <<EOF 添加下面内容：

worker_processes 1; events { worker_connections 1024; } stream { upstream backend { hash $remote_addr consistent; server 172.26.16.249:6443 max_fails=3 fail_timeout=30s; server 172.26.16.250:6443 max_fails=3 fail_timeout=30s; server 172.26.16.251:6443 max_fails=3 fail_timeout=30s; } server { listen *:8443; proxy_connect_timeout 1s; proxy_pass backend; } } EOF

注意：只需要修改server 内容即可 . （5）分发配置文件

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${MASTER_IPS[@]} do echo ">>> ${node_ip}" scp kube-nginx.conf root@${node_ip}:/opt/k8s/kube-nginx/conf/kube-nginx.conf done

（6）配置Nginx启动文件

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# cat > kube-nginx.service <<EOF 添加下面内容：

[Unit] Description=kube-apiserver nginx proxy After=network.target After=network-online.target Wants=network-online.target [Service] Type=forking ExecStartPre=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -t ExecStart=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx ExecReload=/opt/k8s/kube-nginx/sbin/kube-nginx -c /opt/k8s/kube-nginx/conf/kube-nginx.conf -p /opt/k8s/kube-nginx -s reload PrivateTmp=true Restart=always RestartSec=5 StartLimitInterval=0 LimitNOFILE=65536 [Install] WantedBy=multi-user.target EOF

（7）分发nginx启动文件

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${MASTER_IPS[@]} do echo ">>> ${node_ip}" scp kube-nginx.service root@${node_ip}:/etc/systemd/system/ done

（8）启动 kube-nginx 服务

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${MASTER_IPS[@]} do echo ">>> ${node_ip}" ssh root@${node_ip} "systemctl daemon-reload && systemctl enable kube-nginx && systemctl start kube-nginx" done

（9）检查 kube-nginx 服务运行状态

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in ${MASTER_IPS[@]} do echo ">>> ${node_ip}" ssh root@${node_ip} "systemctl status kube-nginx |grep 'Active:'" done

出现下图，就说明正常

3、KeepLived 部署

在所有master节点安装keeplived a、安装keepalived

[root@k8s-01 ~]# yum install -y keepalived

b、配置keeplive服务

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# cat > /etc/keepalived/keepalived.conf <<EOF 添加下面内容

! Configuration File for keepalived global_defs { router_id 172.26.16.249 } vrrp_script chk_nginx { script "/etc/keepalived/check_port.sh 8443" interval 2 weight -20 } vrrp_instance VI_1 { state MASTER interface eth0 virtual_router_id 251 priority 100 advert_int 1 mcast_src_ip 172.26.16.249 nopreempt authentication { auth_type PASS auth_pass 11111111 } track_script { chk_nginx } virtual_ipaddress { 172.26.16.252 } } EOF

注意：## 172.26.16.249 为当前节点，172.26.16.253为node节点 . c、将配置拷贝到其他节点，并替换相关IP

[root@k8s-01 ~]# cd /opt/k8s/work

for node_ip in 172.26.16.249 172.26.16.250 172.26.16.251 do echo ">>> ${node_ip}" scp /etc/keepalived/keepalived.conf $node_ip:/etc/keepalived/keepalived.conf done

d、替换IP

ssh root@172.26.16.250 sed -i 's#172.26.16.249#172.26.16.250#g' /etc/keepalived/keepalived.conf ssh root@172.26.16.251 sed -i 's#172.26.16.249#172.26.16.251#g' /etc/keepalived/keepalived.conf

注意：不需修改172.26.16.249，只需修改其他IP即可

e、创建健康检查脚本

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# vi /opt/check_port.sh 添加下面内容：

CHK_PORT=$1 if [ -n "$CHK_PORT" ];then PORT_PROCESS=`ss -lt|grep $CHK_PORT|wc -l` if [ $PORT_PROCESS -eq 0 ];then echo "Port $CHK_PORT Is Not Used,End." exit 1 fi else echo "Check Port Cant Be Empty!" fi

f、启动keeplived

[root@k8s-01 ~]# cd /opt/k8s/work

for NODE in k8s-01 k8s-02 k8s-03; do echo "--- $NODE ---" scp -r /opt/check_port.sh $NODE:/etc/keepalived/ ssh $NODE 'systemctl enable --now keepalived' done

g、查看是否成功

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 ~]# ping 172.26.16.252

h、检查是否启动成功（分别在其他服务器执行）

[root@k8s-01 ~]# cd /opt/k8s/work [root@k8s-01 work]# ps -ef|grep keep 如果没有启动，请执行下面的命令： [root@k8s-01 ~]# systemctl start keepalived

最新回复(0)