本文通过Docker + Gitlab + Gitlab CI实现容器的自动化部署。
环境如下：
iproledomain192.168.1.51prod servernone192.168.1.55dnsnone192.168.1.56gitlab ci/dev servernone192.168.1.57gitlab servergitlab.lzxlinux.cn192.168.1.59harbor serverharbor.lzxlinux.cn注意：为了方便后面实验，建议自定义的域名不要与外部域名冲突，否则在连接外网情况下容易解析错误。
Harbor是一个用于存储Docker镜像的企业级Registry服务。
准备工作： # systemctl stop firewalld && systemctl disable firewalld # setenforce 0 && sed -i 's/=enforcing/=disabled/g' /etc/selinux/config 安装docker： # curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo # yum makecache fast # yum install -y docker-ce # systemctl start docker && systemctl enable docker提高docker pull速度
# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io # systemctl restart docker 下载最新的docker-compose二进制文件： # curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose # chmod +x /usr/local/bin/docker-compose 下载harbor离线安装包：github地址：https://github.com/goharbor/harbor/releases
# cd /software # wget https://storage.googleapis.com/harbor-releases/release-1.9.0/harbor-offline-installer-v1.9.1.tgz # tar zxf harbor-offline-installer-v1.9.1.tgz 安装harbor： # cd harbor/ # vim harbor.yml hostname: harbor.lzxlinux.cn #域名 harbor_admin_password: Harbor12345 #admin用户初始密码 data_volume: /data #数据存储路径，自动创建 log: level: info local: rotate_count: 50 rotate_size: 200M location: /var/log/harbor #日志路径 # sh install.sh 登录报错： # echo '192.168.1.59 harbor.lzxlinux.cn' >> /etc/hosts # docker login harbor.lzxlinux.cn Username: admin Password: Error response from daemon: Get https://harbor.lzxlinux.cn/v2/: dial tcp 192.168.1.59:443: connect: connection refused 解决： # vim /usr/lib/systemd/system/docker.service ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --insecure-registry=harbor.lzxlinux.cn # systemctl daemon-reload && systemctl restart docker.serviceharbor服务器
# cd /software/harbor # docker-compose down -v # docker-compose up -d 重新登录： # docker login harbor.lzxlinux.cn Username: admin Password: WARNING! Your password will be stored unencrypted in /root/.docker/config.json. Configure a credential helper to remove this warning. See https://docs.docker.com/engine/reference/commandline/login/#credentials-store Login Succeeded 推送镜像： # docker pull busybox # docker tag busybox:latest harbor.lzxlinux.cn/public/busybox # docker push harbor.lzxlinux.cn/public/busybox刷新harbor页面，可以看到刚推送到harbor仓库的镜像
以Centos7为例，准备一台至少内存为4G的机器。
准备工作： # systemctl stop firewalld && systemctl disable firewalld # setenforce 0 && sed -i 's/=enforcing/=disabled/g' /etc/selinux/config 安装依赖软件： # yum install -y curl policycoreutils openssh-server openssh-clients postfix # systemctl start postfix && systemctl enable postfix #启动postfix邮件服务 设置gitlab安装源：如果在国内的话，可以尝试使用清华大学的源。
# vim /etc/yum.repos.d/gitlab-ce.repo [gitlab-ce] name=Gitlab CE Repository baseurl=https://mirrors.tuna.tsinghua.edu.cn/gitlab-ce/yum/el$releasever/ gpgcheck=0 enabled=1如果在国外的话，可以使用
# curl https://packages.gitlab.com/install/repositories/gitlab/gitlab-ee/script.rpm.sh | sudo bash 添加本地dns： # vim /etc/hosts 192.168.1.57 gitlab.lzxlinux.cn在Windows电脑hosts文件中添加本地dns：
192.168.1.57 gitlab.lzxlinux.cn 安装gitlab-ce： # yum install -y gitlab-ce 证书创建与配置加载： # mkdir -p /etc/gitlab/ssl # openssl genrsa -out "/etc/gitlab/ssl/gitlab.lzxlinux.cn.key" 2048 # openssl req -new -key "/etc/gitlab/ssl/gitlab.lzxlinux.cn.key" -out "/etc/gitlab/ssl/gitlab.lzxlinux.cn.csr" Country Name (2 letter code) [XX]:cn State or Province Name (full name) []:hz Locality Name (eg, city) [Default City]:hz Organization Name (eg, company) [Default Company Ltd]: Organizational Unit Name (eg, section) []: Common Name (eg, your name or your server's hostname) []:gitlab.lzxlinux.cn Email Address []:admin@lzxlinux.cn Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []:123456 An optional company name []: # openssl x509 -req -days 365 -in "/etc/gitlab/ssl/gitlab.lzxlinux.cn.csr" -signkey "/etc/gitlab/ssl/gitlab.lzxlinux.cn.key" -out "/etc/gitlab/ssl/gitlab.lzxlinux.cn.crt" # openssl dhparam -out /etc/gitlab/ssl/dhparams.pem 2048 # chmod 600 /etc/gitlab/ssl/* # ll /etc/gitlab/ssl total 16 -rw------- 1 root root 424 Oct 29 10:45 dhparams.pem -rw------- 1 root root 1281 Oct 29 10:44 gitlab.lzxlinux.cn.crt -rw------- 1 root root 1074 Oct 29 10:38 gitlab.lzxlinux.cn.csr -rw------- 1 root root 1679 Oct 29 10:37 gitlab.lzxlinux.cn.key nginx SSL代理服务配置： # vim /etc/gitlab/gitlab.rb #修改下面内容 external_url 'https://gitlab.lzxlinux.cn' nginx['redirect_http_to_https'] = true nginx['ssl_certificate'] = "/etc/gitlab/ssl/gitlab.lzxlinux.com.cn" nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/gitlab.lzxlinux.cn.key" nginx['ssl_dhparam'] = "/etc/gitlab/ssl/dhparams.pem" 初始化gitlab相关服务并完成安装： # gitlab-ctl reconfigure # vim /var/opt/gitlab/nginx/conf/gitlab-http.conf #第一个 server_name gitlab.lzxlinux.cn; 下添加该行 rewrite ^(.*)$ https://$host$1 permanent; # gitlab-ctl restart #重启gitlab 登陆和修改密码：打开https://gitlab.lzxlinux.cn/修改root用户密码，然后使用root和新密码登陆。
Gitlab CI服务器建议另选一台服务器搭建，不要与Gitlab服务器部署在同一台机器上。
准备工作： # systemctl stop firewalld && systemctl disable firewalld # setenforce 0 && sed -i 's/=enforcing/=disabled/g' /etc/selinux/config 安装docker： # curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo # yum makecache fast # yum install -y docker-ce # systemctl start docker && systemctl enable docker提高docker pull速度
# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io # systemctl restart docker 安装gitlab ci runner： # echo '54.153.54.194 packages.gitlab.com' >> /etc/hosts # curl -L https://packages.gitlab.com/install/repositories/runner/gitlab-ci-multi-runner/script.rpm.sh | sudo bash # yum install -y gitlab-ci-multi-runner查看是否运行正常
# gitlab-ci-multi-runner status gitlab-runner: Service is running! 设置docker权限：为了能让gitlab-runner能正确的执行docker命令，需要把gitlab-runner用户添加到docker group里, 然后重启docker和gitlab ci runner。
# usermod -aG docker gitlab-runner # systemctl restart docker # gitlab-ci-multi-runner restart 注册到gitlab服务器：到gitlab上查看runner的注册token
# gitlab-ci-multi-runner register Running in system-mode. Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/): https://gitlab.lzxlinux.cn/ #输入gitlab url Please enter the gitlab-ci token for this runner: 4kr9ZmLMWasYxqB2tSzQ #输入token Please enter the gitlab-ci description for this runner: [worker4]: #输入该runner的描述信息 Please enter the gitlab-ci tags for this runner (comma separated): test,demo #输入该runner的tags Whether to run untagged builds [true/false]: [false]: #是否运行没有打上tag的构建 Whether to lock Runner to current project [true/false]: [false]: #是否将该runner锁定到当前项目 ERROR: Registering runner... failed runner=4kr9ZmLM status=couldn't execute POST against https://gitlab.lzxlinux.cn/api/v4/runners: Post https://gitlab.lzxlinux.cn/api/v4/runners: x509: certificate signed by unknown authority PANIC: Failed to register this runner. Perhaps you are having network problems报错，证书签名错误，这里因为前面gitlab服务器做了https。
解决证书报错： # mkdir -p /etc/gitlab/ssl # scp root@gitlab.lzxlinux.cn:/etc/gitlab/ssl/gitlab.lzxlinux.cn.crt /etc/gitlab/ssl # gitlab-ci-multi-runner register \ --tls-ca-file=/etc/gitlab/ssl/gitlab.lzxlinux.cn.crt \ --url "https://gitlab.lzxlinux.cn/" \ --registration-token "4kr9ZmLMWasYxqB2tSzQ" \ --tag-list "test,demo" \ --run-untagged \ --locked="false" \ --executor "shell" # gitlab-ci-multi-runner list Listing configured runners ConfigFile=/etc/gitlab-runner/config.toml worker4 Executor=shell Token=aLzTn6bfk1tXNBLRBbSD URL=https://gitlab.lzxlinux.cn/可以看到，成功地注册了一个runner到gitlab服务器。
演示gitlab runner工作：gitlab上新建一个组test，在test组中新建一个项目helloworld，然后项目中新建一个README.md文件。
再新建一个.gitlab-ci.yml文件，
打开CI/CD → 流水线，可以看到刚刚的提交已经成功完成，
整个pipeline按顺序执行，如果前面的stage出错，后面的stage不会执行。
另选同网段的一台主机搭建一个dns服务器，让同网段任意机器及其上的容器都可以解析到自定义的域名（如 gitlab.lzxlinux.cn）。
首先，在gitlab ci服务器上把gitlab.lzxlinux.cn从/etc/hosts里删除，此时在gitlab ci服务器上是ping不通gitlab.lzxlinux.cn的。
准备工作： # systemctl stop firewalld && systemctl disable firewalld # setenforce 0 && sed -i 's/=enforcing/=disabled/g' /etc/selinux/config 安装docker： # curl http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -o /etc/yum.repos.d/docker.repo # yum makecache fast # yum install -y docker-ce # systemctl start docker && systemctl enable docker提高docker pull速度
# curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io # systemctl restart docker 运行dns容器： # docker run -d -p 53:53/tcp -p 53:53/udp --cap-add=NET_ADMIN --name dns-server andyshinn/dnsmasq # docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES bcaf93792614 andyshinn/dnsmasq "dnsmasq -k" 10 seconds ago Up 9 seconds 0.0.0.0:53->53/tcp, 0.0.0.0:53->53/udp dns-server 配置dns服务： # docker exec -it dns-server sh / # vi /etc/resolv.dnsmasq nameserver 223.5.5.5 nameserver 114.114.114.114 / # vi /etc/dnsmasqhosts 192.168.1.57 gitlab.lzxlinux.cn 192.168.1.59 harbor.lzxlinux.cn / # vi /etc/dnsmasq.conf resolv-file=/etc/resolv.dnsmasq addn-hosts=/etc/dnsmasqhosts # docker restart dns-server 测试：gitlab ci服务器上操作
# vim /etc/resolv.conf #增加一行，且放在公网dns前面 nameserver 192.168.1.55 #该ip为dns容器所在主机ip nameserver 223.5.5.5 # ping gitlab.lzxlinux.cn PING gitlab.lzxlinux.cn (192.168.1.57) 56(84) bytes of data. 64 bytes from gitlab.lzxlinux.cn (192.168.1.57): icmp_seq=1 ttl=64 time=0.276 ms 64 bytes from gitlab.lzxlinux.cn (192.168.1.57): icmp_seq=2 ttl=64 time=0.310 ms 64 bytes from gitlab.lzxlinux.cn (192.168.1.57): icmp_seq=3 ttl=64 time=0.380 ms 64 bytes from gitlab.lzxlinux.cn (192.168.1.57): icmp_seq=4 ttl=64 time=0.319 ms ^C --- gitlab.lzxlinux.cn ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3001ms rtt min/avg/max/mdev = 0.276/0.321/0.380/0.039 ms # ping harbor.lzxlinux.cn PING harbor.lzxlinux.cn (192.168.1.59) 56(84) bytes of data. 64 bytes from harbor.lzxlinux.cn (192.168.1.59): icmp_seq=1 ttl=64 time=0.307 ms 64 bytes from harbor.lzxlinux.cn (192.168.1.59): icmp_seq=2 ttl=64 time=0.430 ms 64 bytes from harbor.lzxlinux.cn (192.168.1.59): icmp_seq=3 ttl=64 time=0.383 ms 64 bytes from harbor.lzxlinux.cn (192.168.1.59): icmp_seq=4 ttl=64 time=0.346 ms ^C --- harbor.lzxlinux.cn ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3000ms rtt min/avg/max/mdev = 0.307/0.366/0.430/0.049 ms # docker run -it --rm busybox sh / # ping gitlab.lzxlinux.cn PING gitlab.lzxlinux.cn (192.168.1.57): 56 data bytes 64 bytes from 192.168.1.57: seq=0 ttl=63 time=0.388 ms 64 bytes from 192.168.1.57: seq=1 ttl=63 time=0.472 ms ^C --- gitlab.lzxlinux.cn ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max = 0.388/0.430/0.472 ms / # ping harbor.lzxlinux.cn PING harbor.lzxlinux.cn (192.168.1.59): 56 data bytes 64 bytes from 192.168.1.59: seq=0 ttl=63 time=0.580 ms 64 bytes from 192.168.1.59: seq=1 ttl=63 time=0.332 ms 64 bytes from 192.168.1.59: seq=2 ttl=63 time=0.401 ms ^C --- harbor.lzxlinux.cn ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.332/0.437/0.580 ms可以看到，在搭建dns服务器后，即使gitlab ci服务器的/etc/hosts删除本地dns，主机和主机上的容器仍可以解析我们自定义的域名。
