※1.部署文档可参考如下连接跳转

K8S 单master部署: https://www.garafana.com/archives/k8s%E6%90%AD%E5%BB%BA%E5%8D%95%E6%9C%BA%E7%89%88--%E7%95%AA%E5%A4%96%E7%AF%87
K8S 高可用版本: https://www.garafana.com/archives/k8s%E6%90%AD%E5%BB%BA01-k8s%E9%9B%86%E7%BE%A4
Harbor仓库： https://www.garafana.com/archives/k8s%E6%90%AD%E5%BB%BA03-harbor%E9%95%9C%E5%83%8F%E7%AE%A1%E7%90%86%E4%B8%8E%E5%AE%89%E8%A3%85%E4%BD%BF%E7%94%A8
Jenkins 主从搭建：https://www.garafana.com/archives/jenkinszhu-cong-da-jian
gitlab 仓库搭建： https://www.garafana.com/archives/gitlab-14102-bu-shu-wen-dang

※2.gitlab 配置CI以及脚本说明

在项目根层新建一个文本<.gitlab-ci.yml>填写好CI文件规则，如下为参考模板，可自行百度看修改

stages:
  - test_branch
  - poc_branch
  - dev_branch

test_update:
  tags: 
    - runner02             #这个是runner标签，看你定义的标签是啥就修改啥
  stage: test_branch
  only:
    - test   #判断代码提交到test分支则触发CI
  script:
    - /usr/bin/jenkin run  $CI_PROJECT_NAME "BRANCH_NAME":"$CI_COMMIT_REF_NAME"    #这边的变量都是CI内置的，如果有多选项参数则按找这个模板填写
    - export LANG=zh_CN.UTF-8 ; /usr/bin/jenkin log  $CI_PROJECT_NAME                             #这是拉去Jenkins执行日志
    
poc_update:
  tags: 
    - runner02
  stage: poc_branch
  only:
    - finance-cloud-v1-prod
  script:
    - echo $CI_PROJECT_NAME
    
dev_update:
  tags: 
    - runner02
  stage: dev_branch
  only:
    - dev
  script:
    - /usr/bin/jenkin run  $CI_PROJECT_NAME "BRANCH_NAME":"$CI_COMMIT_REF_NAME"
    - export LANG=zh_CN.UTF-8 ; /usr/bin/jenkin log  $CI_PROJECT_NAME 

python驱动脚本信息模板，会在最下方说明用法

# -*- coding: utf-8 -*-
import time
import jenkins
import sys

jenkins_url = "http://192.168.1.101:8080"   # jenkins访问地址
jenkins_use = "root"    # jenkins用户
jenkins_passwd = "a123456"  # Jenkins密码或者token

def build_job(job_name, datas): # 构建job，需要外部传参job名称以及是否有多选项参数，可以为空
    server.build_job(name=job_name, parameters=datas)


def number_job(job_name):   # 获取job的ID方便后续抓取日志
    last_build_number = server.get_job_info(name=job_name)['lastBuild']['number']
    return last_build_number


def getJobResultStatus(job_id, job_name):
    '''
    role:获取job名为job_name的job的某次构建的执行结果状态
    SUCCESS : job执行成功
    FAILURE ：job执行失败
    ABORTED ：人为结束构建
    None : 正在构建中
    '''
    return server.get_build_info(job_name, job_id)['result']


def getJobBuilding(job_id, job_name):
    '''
    role:判断job名为job_name的job的某次构建是否还在构建中
    True:正在构建
    Fase:构建结束
    '''
    return server.get_build_info(job_name, job_id)['building']


if __name__ == '__main__':
    action = sys.argv[1]    # 运行动作 run 是执行job ,log 则是拉取日志
    server = jenkins.Jenkins(jenkins_url, username=jenkins_use, password=jenkins_passwd)
    jobs = sys.argv[2]   # job的名字，这边的做法是CI直接去仓库名称，Jenkins job的名字跟仓库名字一样
    if action == "run":
        datasource = sys.argv[3]   # 这是负责遍历是否有传多选项参数，比较懒不想写判断
        datas = {key: value for key, value in [item.split(":") for item in datasource.split(",")]}
        build_job(jobs, datas)
    elif action == "log":
        time.sleep(30)  # api出发怕jenkin没来得及响应
        job_id = number_job(jobs)
        job_status = getJobResultStatus(job_id, jobs)
        while True:
            current_number = 1
            status = getJobBuilding(job_id, jobs)
            if str(status) == "True":
                time.sleep(10)
                current_number += 1
            elif current_number == "50":
                print("程序执行超时500秒没有返回正确结果，程序强制终止！")
                sys.exit(1)
            else:
                break
        print("------------------------------------------------------------")
        print("-----------------------下面为Jenkins运行日志----------------------")
        files = server.get_build_console_output(name=jobs, number=job_id)
        print(files)

这是一份出版可参考类型的脚本，使用方式就是结合CI文件自动触发Jenkins_jobs

使用环境python3.8,需要安装Jenkins插件(pip3 install python-jenkins)

请在linux环境操作，最后打包成二进制可执行文件，具体打包命令可查看pyinstaller 这个插件用法

制作成二进制文件后上传到runner机器，存放在/usr/bin目录下，请注意runner是用什么用户执行的，如果不是root则建议授权一下属主以及添加可执行文件权限，如果是docker安装的runner记得下载语言包，否则脚本会执行错误，脚本会输出中文导致python报错

※3.Jenkins操作部分

###如下为Jenkins插件选择安装部分，请根据需求自行安装需要的

Run Condition	
Run Condition Extras	
Config File Provider
Pipeline Maven Integration	 
ruby-runtime	 
External Monitor Job Type	 
Javadoc	 
Maven Integration	 
Conditional BuildStep	 
GitLab	 
Build Authorization Token Root	 
Parameterized Trigger	 
jQuery	 
Build Pipeline	 
Gitlab Hook

在Jenkins中系统管理->系统配置操作

找到全局属性配置jdk目录以及maven目录

找到Global Pipeline Libraries 配置git仓库以及harbor仓库账号密码

然后配置maven仓库的setting文件，该步骤按需求操作

在系统管理->Managed files 中添加文件配置信息

以及在系统管理->全局工具配置配置setting文件，默认是使用maven自己的仓库地址

返回Jenkins首页，开始创建job

如果你按照我上方的方案走，则这边创建的job名称就是使用你仓库项目名称，为了更快演示这边就取名为demo

接下来进入流水线配置阶段，直接到默认有一个流水线，可根据如下提供模板自行修改，如果需要更新的脚本这边会在下方提供，说明使用方式

pipeline {
  agent {
    label 'master'
  }
   
    parameters {
    choice(choices: ['demo'], name: 'ServicesDeploy', description: '请选择要构建的服务,支持单个服务发布或全部服务发布')
    choice(choices: ['yes', 'no'], name: 'sonarQube', description: '是否进行sonarQube代码质量检查,默认为no')
    string(defaultValue:"test", description: 'enter the branch name to use', name: 'BRANCH_NAME')
  }
   
    environment {
    HARBOR_CREDENTIAL_ID = 'harbor'  // 这个就是上方添加的harbor凭据
    GITLAB_CREDENTIAL_ID = 'gitlab-use'  #这个是gitlab的凭据
    REGISTRY = '192.168.1.199'       // 这是harbor仓库地址
    HARBOR_NAMESPACE = 'project'     // 这是harbor仓库镜像存放目录，需要跟下面脚本配合
    K8s_NAMESPACE = 'test'          // 这是传参使用k8s脚本更新的命名空间
    _cnlab_msdp="${params.ServicesDeploy}"  //这是下面传参判断是否执行job
    BRANCHS="${params.BRANCH_NAME}"    //这个是传递分支版本
    _sonar="${params.sonarQube}"       //这个是判断sonarqube目前废弃了
  }
   
//请修改你的项目地址以及拉去git的用户信息   
  stages {
    stage('拉取demo代码') {
      agent none
      when { environment name: '_cnlab_msdp', value: 'demo' }
      steps {
        git(url: 'http://192.168.1.100:3000/demo/demo.git', credentialsId: 'gitlab-use', branch: 'test', changelog: true, poll: false)
         
      }
    }

    stage('打包') {
      agent none
      steps {
        script {
          sh "mvn -Dmaven.test.skip=true clean install -U -T 4 "
        }
      }
    }
    
    stage('构建镜像') {
      agent none
      steps {
          script {
            stage ("build") {
              sh "sudo chmod +x k8s && sudo docker build -t $REGISTRY/$HARBOR_NAMESPACE/`./k8s imgn`:$BUILD_NUMBER ."
          }
 
        }
 
      }
    }
     
    stage('镜像推送') {
      agent none
      steps {
          script {
              withCredentials([usernamePassword(passwordVariable : 'HARBOR_PASSWORD' ,usernameVariable : 'HARBOR_USERNAME' ,credentialsId : "$HARBOR_CREDENTIAL_ID" ,)]) {
            sh 'echo "$HARBOR_PASSWORD" |sudo  docker login $REGISTRY -u "$HARBOR_USERNAME" --password-stdin'
            sh "sudo chmod +x k8s && sudo docker push  $REGISTRY/$HARBOR_NAMESPACE/`./k8s imgn`:$BUILD_NUMBER"
                }
              }
            }
          }

    stage('K8S项目更新') {
      agent none
      steps {
          sh 'chmod +x k8s && ./k8s update $BUILD_NUMBER $K8s_NAMESPACE'
      }
    }
 
    stage('k8s邮件发送') {
      agent none
      steps {
        sh 'chmod +x k8s && ./k8s mail $BUILD_NUMBER $BRANCHS'
      }
    }
 
   }
 
}

这是上面流水线使用到的k8s脚本，还是使用python3.8操作的，有许多需要下载的库，请到时候根据报错在去下载，这边就不整理了

注意如下脚本写的比较懒惰，很多需要改的地方都备注了，麻烦要使用的朋友认真看吧，很多变量都在脚本中需要修改，然后还是使用pyinstaller 打成二进制文件，吧文件直接放在项目的根层即可

import logging
import os
import re
import smtplib
import sys
import time
from email.mime.text import MIMEText
from email.utils import formataddr
from kubernetes import client, config
import xml.etree.ElementTree as ET

linux_path = '/tmp/k8s.yaml'

# 全局配置

email = "100000@qq.com,10001@qq.com"  #这是配置收件人，可配置多个人

#载入pom.xml
tree = ET.parse("pom.xml")
root = tree.getroot()

for child in root:
    data = child.tag, child.text
    aaa = str(data).split("}", 1)[-1]
    eee = str(aaa).split("'")[0]
    if eee == "artifactId":
        projectr = str(aaa).split("'")[2]
        break

# 从配置文件读取
projectName = projectr    # 这部分是去上方pom文件中定义的artifactId作为K8S项目名称，如果不用这个方式则注释上面以及修改这边projectr为自己定义
servers_name = "img_" + projectName   # 这是定义docker镜像的名称，会关联后续更新，主要拼接的是img_项目名称，也可按照自己需求修改
prod_name = "pod-" + projectName      # 这是k8s的pod名称，也是拼接项目名称，也是可以按照自行修改
poc_name = "poc-" + projectName       # 跟上面同理
dev_name = "dev-" + projectName       # 跟上上面的同理
receives = email                      # 这步骤是传递邮箱，一开始定义多余了，但是也使用了，懒得改了就这样吧


class QQMailMan:
    def __init__(self, user, password):
        # 发件人邮箱qq号
        self.user = user
        # user登录邮箱的用户名，password登录邮箱的密码（授权码，即客户端密码，非网页版登录密码），但用腾讯邮箱的登录密码也能登录成功
        self.password = password

    def connect(self):
        try:
            self.smtp_host = "smtp.exmail.qq.com"
            self.smtp_port = 465
            self.mailman = smtplib.SMTP_SSL(self.smtp_host, self.smtp_port)
            self.mailman.set_debuglevel(1)
            self.mailman.login(self.user, self.password)
        except Exception:
            raise (Exception)

    def close(self):
        self.mailman.quit()

    def mail_to(self, to_addrs, subject, content):
        """
        发送邮件到指定地址（也可以是地址列表或用,拼接的字符串）
        """
        msg = MIMEText(content, 'html', 'utf-8')
        if type(to_addrs) == str:
            msg['To'] = to_addrs
            to_addrs = to_addrs.split(',')
        elif type(to_addrs) == list:
            msg['To'] = ','.join(to_addrs)
        else:
            logging.error("邮件地址类型错误，必须为str（可以用,拼接）或list")
            return

        time.sleep(1)
        msg['From'] = formataddr(["Kubesphere", self.user])
        # 邮件的主题
        msg['Subject'] = subject

        try:
            ret = self.mailman.sendmail(self.user, to_addrs, msg.as_string())
            logging.info("成功发送邮件至：{} {}".format(to_addrs, ret))
        except Exception:
            logging.error("发送邮件失败: {} {} {}".format(to_addrs, ret, Exception))


def k8s_ssl():
    ssl_kubernetes = linux_path
    ssl_key = '''这个位置请在master节点cat /root/.kube/config里面的内容请全部复制到这里，吧这写的中文全部删除覆盖'''  # 这是python操作k8s的主入口配置文件

    # 判断证书是否存在，不存在则创建
    if os.path.exists(ssl_kubernetes):
        return
    else:
        with open(file=ssl_kubernetes, mode="w", encoding="utf-8") as f:
            f.write(ssl_key)


def pod_update(servers_list, name, namespace, linux_paths):
    # 初始化Kubernetes api
    config.kube_config.load_kube_config(config_file=linux_paths)
    _AppsV1Api = client.AppsV1Api()
    old_deploy = _AppsV1Api.read_namespaced_deployment(
        name=name,
        namespace=namespace,
    )
    img_domain_rule = "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}"
    oldImgdoamin = re.findall(img_domain_rule, old_deploy.spec.template.spec.containers[0].image)[0]
    images = "{0}/project/{1}:{2}".format(oldImgdoamin, servers_list, text_output)                # 注意这个project是harbor仓库的镜像存放目录，还是懒得改脚本了
    old_deploy.spec.template.spec.containers[0].image = images
    _AppsV1Api.patch_namespaced_deployment(
        name=name,
        namespace=namespace,
        body=old_deploy
    )


def mailsend(buiid):
    content_template = ("""\
    <!DOCTYPE html>
    <html lang="en">

    <head>
    <meta charset="UTF-8">
      <meta http-equiv="X-UA-Compatible" content="IE=edge">
      <meta name="viewport" content="width=device-width, initial-scale=1.0">
      <title>智能邮件机器人</title>

      <style>
        body,html,div,ul,li,button,p,img,h1,h2,h3,h4,h5,h6 {
          margin: 0;
          padding: 0;
        }

        body,html {
          background: #fff;
          line-height: 1.8;
        }

        h1,h2,h3,h4,h5,h6 {
          line-height: 1.8;
        }

        .email_warp {
          height: 100vh;
          min-height: 500px;
          font-size: 14px;
          color: #212121;
          display: flex;
          /* align-items: center; */
          justify-content: center;
        }

        .logo {
          margin: 3em auto;
          width: 200px;
          height: 60px;
        }

        h1.email-title {
          font-size: 26px;
          font-weight: 500;
          margin-bottom: 15px;
          color: #252525;
        }

        a.links_btn {
          border: 0;
          background: #4C84FF;
          color: #fff;
          width: 100%%;
          height: 50px;
          line-height: 50px;
          font-size: 16px;
          margin: 40px auto;
          box-shadow: 0px 2px 4px 0px rgba(0, 0, 0, 0.15);
          border-radius: 4px;
          outline: none;
          cursor: pointer;
          transition: all 0.3s;
          text-align: center;
          display: block;
          text-decoration: none;
        }

        .warm_tips {
          color: #757575;
          background: #f7f7f7;
          padding: 20px;
        }

        .warm_tips .desc {
          margin-bottom: 20px;
        }

        .qr_warp {
          max-width: 140px;
          margin: 20px auto;
        }

        .qr_warp img {
          max-width: 100%%;
          max-height: 100%%;
        }

        .email-footer {
          margin-top: 2em;
        }

        #reset-password-email {
          max-width: 500px;
        }
        #reset-password-email .accout_email {
          color: #4C84FF;
          display: block;
          margin-bottom: 20px;
        }
      </style>
    </head>

    <body>
      <section class="email_warp">
        <div id="reset-password-email">

          <h1 class="email-title">
            这是一封系统发出的邮件：
          </h1>
          <p>您所更新的项目名称为：<span style="color:blue">%s</span></p>
          <p>您当前构建ID序号为  ：<span style="color:blue">%s</span></p>
          <p>您当前构建分支号为  ：<span style="color:blue">%s</span></p>
          <p>请注意，当收到邮件通知时，项目已更新完成，请忽略并关闭此邮件。</p>

          <a class="links_btn" href="http://172.19.70.201:30880/devops/clusters/default/devops/deploy-projectkgx4f/pipelines">查看部署信息</a>

          <div class="warm_tips">
            <div class="desc">
              为安全起见，以上按钮为一次性链接，且仅在24小时内有效，请您尽快完成操作。
            </div>

            <p>如有任何疑问，请通过如下方式与我们联系：</p>
            <p>邮箱：huangxichen@tansun.com.cn</p>

            <p>本邮件由系统自动发送，请勿回复。</p>
          </div>
        </div>
      </section>
    </body>

    </html>
    """) % (projectName, buiid, branchid)
    user = "admin@qq.com"                 # 需要一个专门发送邮箱的账号
    password = "a123456"                  # QQ邮箱密码
    qqmail = QQMailMan(user, password)
    qqmail.connect()
    to_addrs = email
    content = content_template
    biaoti = "[Kubesphere]：%s项目更新通知，序号：%s。" % (projectName, buiid)
    qqmail.mail_to(to_addrs, biaoti, content)
    qqmail.close()


def help():
    print("""This is a helpful message
    If you use the 'mail' function send mail function
    You need two location variables. Get the first one because the build ID is used in the email, 
    and get the second parameter because the email tells you the branch information used by gitlab.
    eg：python3 xxxx.py 2  dev

    If you use the 'update' function send mail function
    This function also requires two entries. The first positional parameter is the image version number to be updated,
    which the script will automatically concatenate for you, and the second variable is the namespaces to operate kubernetes
    eg: python3 xxxx.py 2 dev

    If you use the 'imgn' function send mail function
    This function is simply to help you print out the name of the image, 
    please do not use this function, is personal hidden use

    If you use the 'demo' function send mail function
    This function also takes one parameters, 
    the first being the namespace in which the environment is to be updated,
    eg: python3 xxxx.py tanrui
    """)

if __name__ == "__main__":
    if len(sys.argv) == len('a'):
        help()
        sys.exit()
    action = sys.argv[1]
    # action = "demo"
    if action == "mail":
        if not email:
            print("没有配置发送邮箱，本次执行不发送邮件")
            sys.exit()
        buiid = sys.argv[2]
        branchid = sys.argv[3]
        mailsend(buiid)
    elif action == "update":
        text_output = sys.argv[2]
        prod_namespace = sys.argv[3]

        k8s_ssl()
        pod_update(servers_name, prod_name, prod_namespace, linux_path)
        os.remove(linux_path)

    elif action == "imgn":
        print(servers_name)
    else:
        help()