linuxea:tekton与gitlab hook的实现(7) 很多时候，我们希望提交代码就进行触发构建，那么我们需要在gitlab上进行配置，以往使用jenkins也需要进行在jenkins上进行配置，一旦提交动作，将会触发jenkins构建。而到了tekton也是一样。不同的是，tekton是由几个crd组合起来的。Triggers：CRD说明：TriggerTemplate: 创建资源的模板，比如用来创建 PipelineResource 和 PipelineRunTriggerBinding: 校验事件并提取相关字段属性ClusterTriggerBinding: 和 TriggerBinding 类似，全局Interceptor: 处理事件以进行自定义验证或过滤的拦截器EventListener: 连接 TriggerBinding 和 TriggerTemplate 到事件接收器，使用从各个 TriggerBinding 中提取的参数来创建 TriggerTemplate 中指定的 resources，同样通过 interceptor 字段来指定外部服务对事件属性进行预处理安装参考版本支持，我的集群的是1.25.11，因此，我安装最新的mkdir tekton-trigger && cd tekton-trigger wget https://storage.googleapis.com/tekton-releases/triggers/previous/v0.24.1/release.yaml -O v0.24.1-release.yaml wget https://storage.googleapis.com/tekton-releases/triggers/previous/v0.24.1/interceptors.yaml -O v0.24.1-interceptors.yaml替换镜像sed -i 's#gcr.io/tekton-releases/github.com/tektoncd/triggers/cmd/interceptors:v0.24.1@sha256:162607d0901cd65e525b3563bc19aa55802f0635ca425df05ccc3eb7f420c2e4#uhub.service.ucloud.cn/marksugar-k8s/interceptors:v0.24.1#g' *.yaml sed -i 's#gcr.io/tekton-releases/github.com/tektoncd/triggers/cmd/controller:v0.24.1@sha256:d928774533783ce98fdc95f1fa2b7ac3b2c519e8366512a2e65ed9f540e3c0bb#uhub.service.ucloud.cn/marksugar-k8s/controller:v0.24.1#g' *.yaml sed -i 's#gcr.io/tekton-releases/github.com/tektoncd/triggers/cmd/webhook:v0.24.1@sha256:314d28791b9adeee9e38c6fb0b1ed40deeee097f495f73a82e5831ff6393dfcc#uhub.service.ucloud.cn/marksugar-k8s/webhook:v0.24.1#g' *.yaml镜像替换完成，按照顺序进行安装kubectl apply -f v0.24.1-release.yaml kubectl apply -f v0.24.1-interceptors.yaml 如下:[root@master-01 ~/tekton-trigger]# kubectl -n tekton-pipelines get pod NAME READY STATUS RESTARTS AGE tekton-dashboard-f6dc6975c-9z88h 1/1 Running 3 (7d6h ago) 10d tekton-pipelines-controller-5575f56d66-vtjwx 1/1 Running 3 (7d6h ago) 10d tekton-pipelines-webhook-779b88db8c-m972h 1/1 Running 3 (7d6h ago) 10d tekton-triggers-controller-57bc549d8-6lhf9 1/1 Running 0 4m40s tekton-triggers-core-interceptors-5fd6d88b88-9kms2 1/1 Running 0 4m41s tekton-triggers-webhook-57d6f96479-p6htl 1/1 Running 0 4m40s1.auth我们仍然需要创建必要的用户名密码信息1.创建Secret，将认证信息写入apiVersion: v1 kind: Secret metadata: name: gitlab-auth annotations: tekton.dev/git-0: https://172.16.100.47 type: kubernetes.io/basic-auth stringData: username: root password: examplepassword --- apiVersion: v1 kind: Secret metadata: name: gitlab-secret-triggers type: Opaque stringData: secretToken: "www.linuxea.com" secretToken是gitlab的webhook的令牌2.EventListener我们需要创建一个EventListener对象来处理触发的信息，EventListener处理传入请求并执行Trigger. 我们的事件监听器如下所示apiVersion: triggers.tekton.dev/v1beta1 kind: EventListener metadata: name: gitlab-listener # 该事件监听器会创建一个名为 el-gitlab-listener 的Service对象 spec: serviceAccountName: tekton-triggers-gitlab-sa triggers: - name: gitlab-push-events-trigger interceptors: - ref: name: gitlab params: - name: secretRef # 引用 gitlab-secret 的 Secret 对象中的 secretToken 的值 value: secretName: gitlab-secret-trigger secretKey: secretToken - name: eventTypes value: - Push Hook # 只接收 GitLab Push 事件 bindings: - ref: devops-binding template: ref: java-demmo-template收到传入请求后，将执行gitlab-listener触发器。在这种情况下，触发器嵌入到 EventListener 资源中，而不是在单独的资源中指定。然后 gitlab-listener 触发器将使用Interceptors。拦截器让我们可以在传入请求触发管道运行之前对其进行验证或修改。我们嵌入拦截器资源，而不是将其放入单独的资源清单中。这里的gitlab内置的，因此，如果是github，那这里应该填写github.EventListener创建完成后会生成一个service对象，要确保这个service是可以被访问到的，如果不能被访问到，那将无从进行传入请求。EventListener并且将如下两个资源对象组合在一起 bindings: - ref: devops-binding template: ref: java-dem-template于是，我们创建TriggerBinding3.TriggerBindingapiVersion: triggers.tekton.dev/v1beta1 kind: TriggerBinding metadata: name: devops-binding spec: params: - name: gitrevision value: $(body.checkout_sha) - name: gitrepositoryurl value: $(body.repository.git_http_url)TriggerBinding中的$(body.checkout_sha)和$(body.repository.git_http_url)是根据gitlab返回的webhook值来获取的大致内容结构如下{ "object_kind": "push", "event_name": "push", "before": "59d6de7d1b59f50a9d5e850db346bb6e8bde059c", "after": "9ddeecff98e1f6eecb0ce5837cf43c2cc1c9d28c", "ref": "refs/heads/main", "checkout_sha": "9ddeecff98e1f6eecb0ce5837cf43c2cc1c9d28c", "message": null, "user_id": 1, "user_name": "Administrator", "user_username": "root", "user_email": null, "user_avatar": "https://www.gravatar.com/avatar/e64c7d89f26bd1972efa854d13d7dd61?s=80&d=identicon", "project_id": 2, "project": { "id": 2, "name": "Argocd Example", "description": null, "web_url": "https://172.16.100.47/devops/argocd-example", "avatar_url": null, "git_ssh_url": "ssh://git@172.16.100.47:23857/devops/argocd-example.git", "git_http_url": "https://172.16.100.47/devops/argocd-example.git", "namespace": "devops", "visibility_level": 0, "path_with_namespace": "devops/argocd-example", "default_branch": "main", "ci_config_path": null, "homepage": "https://172.16.100.47/devops/argocd-example", "url": "ssh://git@172.16.100.47:23857/devops/argocd-example.git", "ssh_url": "ssh://git@172.16.100.47:23857/devops/argocd-example.git", "http_url": "https://172.16.100.47/devops/argocd-example.git" }, "commits": [ { "id": "9ddeecff98e1f6eecb0ce5837cf43c2cc1c9d28c", "message": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-172651\n", "title": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-172651", "timestamp": "2023-07-12T09:29:15+00:00", "url": "https://172.16.100.47/devops/argocd-example/-/commit/9ddeecff98e1f6eecb0ce5837cf43c2cc1c9d28c", "author": { "name": "marksugar", "email": "[REDACTED]" }, "added": [ ], "modified": [ "kustmoze/overlays/production/kustomization.yaml" ], "removed": [ ] }, { "id": "912c7675352b49dc870ed4cf067aec949023f88e", "message": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-140251\n", "title": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-140251", "timestamp": "2023-07-12T06:05:15+00:00", "url": "https://172.16.100.47/devops/argocd-example/-/commit/912c7675352b49dc870ed4cf067aec949023f88e", "author": { "name": "marksugar", "email": "[REDACTED]" }, "added": [ ], "modified": [ "kustmoze/overlays/production/kustomization.yaml" ], "removed": [ ] }, { "id": "59d6de7d1b59f50a9d5e850db346bb6e8bde059c", "message": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-135953\n", "title": "kustomize change image tag 172.16.100.46/test/java-demo:v0.1-20230712-135953", "timestamp": "2023-07-12T06:02:26+00:00", "url": "https://172.16.100.47/devops/argocd-example/-/commit/59d6de7d1b59f50a9d5e850db346bb6e8bde059c", "author": { "name": "marksugar", "email": "[REDACTED]" }, "added": [ ], "modified": [ "kustmoze/overlays/production/kustomization.yaml" ], "removed": [ ] } ], "total_commits_count": 3, "push_options": { }, "repository": { "name": "Argocd Example", "url": "ssh://git@172.16.100.47:23857/devops/argocd-example.git", "description": null, "homepage": "https://172.16.100.47/devops/argocd-example", "git_http_url": "https://172.16.100.47/devops/argocd-example.git", "git_ssh_url": "ssh://git@172.16.100.47:23857/devops/argocd-example.git", "visibility_level": 0 } }我们创建上述配置[root@master-01 ~/tekton/workspace/triggers]# kubectl apply -f event.yaml eventlistener.triggers.tekton.dev/gitlab-listener created [root@master-01 ~/tekton/workspace/triggers]# kubectl apply -f binding.yaml triggerbinding.triggers.tekton.dev/devops-binding created [root@master-01 ~/tekton/workspace/triggers]# kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE el-gitlab-listener ClusterIP 10.68.17.70 <none> 8080/TCP,9000/TCP 5s kubernetes ClusterIP 10.68.0.1 <none> 443/TCP 12d [root@master-01 ~/tekton/workspace/triggers]# kubectl get EventListener NAME ADDRESS AVAILABLE REASON READY REASON gitlab-listener https://el-gitlab-listener.default.svc.cluster.local:8080 True MinimumReplicasAvailable True [root@master-01 ~/tekton/workspace/triggers]# kubectl get TriggerBinding NAME AGE devops-binding 14s [root@master-01 ~]# kubectl get pod -l eventlistener=gitlab-listener NAME READY STATUS RESTARTS AGE el-gitlab-listener-7678bc4b57-87hpl 1/1 Running 4m 4m我们创建一个ingress-nginx暴漏el-gitlab-listenerapiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tekton-trigger-ingress annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/ssl-redirect: "false" spec: # ingressClassName: nginx rules: - host: local.tekton-trigger.com http: paths: - path: /hooks pathType: Exact backend: service: name: el-gitlab-listener port: number: 8080创建完成后即可[root@master-01 ~/tekton/workspace/triggers]# kubectl get ingress NAME CLASS HOSTS ADDRESS PORTS AGE tekton-trigger-ingress <none> local.tekton-trigger.com 172.16.100.80 80 18s4.gitlab hook我们的这个ingress域名local.tekton-trigger.com是要被gitlab请求通才行的，于是我们在gitlab的配置中添加私有域名映射关系 extra_hosts: - "local.tekton-trigger.com:172.16.100.80"version: '3' services: gitlab-ce: container_name: gitlab-ce #image: gitlab/gitlab-ce:15.0.3-ce.0 image: registry.cn-zhangjiakou.aliyuncs.com/marksugar/gitlab-ce:15.5.3-ce.0 restart: always # network_mode: host hostname: 172.16.100.47 extra_hosts: - "local.tekton-trigger.com:172.16.100.80" environment: TZ: 'Asia/Shanghai' GITLAB_OMNIBUS_CONFIG: | external_url 'https://172.16.100.47' gitlab_rails['time_zone'] = 'Asia/Shanghai' gitlab_rails['gitlab_shell_ssh_port'] = 23857 ports: - '80:80' - '443:443' - '23857:22' volumes: - /etc/localtime:/etc/localtime - /data/gitlab/config:/etc/gitlab - /data/gitlab/logs:/var/log/gitlab - /data/gitlab/data:/var/opt/gitlab logging: driver: "json-file" options: max-size: "50M" deploy: resources: limits: memory: 4096m reservations: memory: 4096能够ping通即可[root@Node-172_16_100_47 /data]# docker exec -i gitlab-ce ping -c 3 local.tekton-trigger.com PING local.tekton-trigger.com (172.16.100.80): 56 data bytes 64 bytes from 172.16.100.80: seq=0 ttl=63 time=0.278 ms 64 bytes from 172.16.100.80: seq=1 ttl=63 time=0.422 ms 64 bytes from 172.16.100.80: seq=2 ttl=63 time=1.056 ms --- local.tekton-trigger.com ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 0.278/0.585/1.056 ms而后登录gitlab的项目中->设置->Webhook设置输入暴漏的域名，填写一个www.linuxea.com的令牌，勾选推送事件，而后在最下面不勾选启用SSL验证，而后add webhook如果提示Url is blocked: Requests to the local network are not allowed开启即可创建完成后，稍后进行测试5.rbac及令牌我们还需要创建Secret，将刚才令牌为www.linuxea.com的添加apiVersion: v1 kind: Secret metadata: name: gitlab-secret-trigger type: Opaque stringData: secretToken: "www.linuxea.com" 而后还需要将创建的secret绑定到rbac上apiVersion: v1 kind: ServiceAccount metadata: name: tekton-triggers-gitlab-sa secrets: - name: gitlab-secret-triggers - name: gitlab-auth-triggers --- apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: tekton-triggers-gitlab-minimal rules: # EventListeners need to be able to fetch all namespaced resources - apiGroups: ["triggers.tekton.dev"] resources: ["eventlisteners", "triggerbindings", "triggertemplates", "triggers"] verbs: ["get", "list", "watch"] - apiGroups: [""] # configmaps is needed for updating logging config resources: ["configmaps"] verbs: ["get", "list", "watch"] # Permissions to create resources in associated TriggerTemplates - apiGroups: ["tekton.dev"] resources: ["pipelineruns", "pipelineresources", "taskruns"] verbs: ["create"] - apiGroups: [""] resources: ["serviceaccounts"] verbs: ["impersonate"] - apiGroups: ["policy"] resources: ["podsecuritypolicies"] resourceNames: ["tekton-triggers"] verbs: ["use"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: tekton-triggers-gitlab-binding subjects: - kind: ServiceAccount name: tekton-triggers-gitlab-sa roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: tekton-triggers-gitlab-minimal --- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: tekton-triggers-gitlab-clusterrole rules: # EventListeners need to be able to fetch any clustertriggerbindings - apiGroups: ["triggers.tekton.dev"] resources: ["clustertriggerbindings", "clusterinterceptors","interceptors"] verbs: ["get", "list", "watch"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: tekton-triggers-gitlab-clusterbinding subjects: - kind: ServiceAccount name: tekton-triggers-gitlab-sa namespace: default roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: tekton-triggers-gitlab-clusterrole全部创建完成后，开始配置template[root@master-01 ~/tekton/workspace/triggers]# kubectl apply -f secret.yaml secret/gitlab-auth-triggers created secret/gitlab-secret-triggers created [root@master-01 ~/tekton/workspace/triggers]# kubectl apply -f rbac.yaml serviceaccount/tekton-triggers-gitlab-sa created role.rbac.authorization.k8s.io/tekton-triggers-gitlab-minimal created rolebinding.rbac.authorization.k8s.io/tekton-triggers-gitlab-binding created clusterrole.rbac.authorization.k8s.io/tekton-triggers-gitlab-clusterrole created clusterrolebinding.rbac.authorization.k8s.io/tekton-triggers-gitlab-clusterbinding created6.TriggerTemplate这些数据可根据需要进行获取。而后TriggerTemplate 对象中通过参数来读取上面 TriggerBinding 中定义的参数值了，定义一个如下所示的 TriggerTemplate 对象，声明一个 TaskRun 的模板TriggerTemplate的模板就是把Pipelinerun的模板改一下就好了java-demmo-template对应的就是TriggerTemplate，这在上面的配置已经写过了apiVersion: triggers.tekton.dev/v1beta1 kind: TriggerTemplate metadata: name: java-demmo-template spec: params: # 定义参数，和 TriggerBinding 中的保持一致 - name: gitrevision - name: gitrepositoryurl resourcetemplates: # 定义资源模板 - apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: mvn-pipelinerun spec: serviceAccountName: build-sa pipelineRef: name: mvn-pipeline podTemplate: hostAliases: - ip: "172.16.100.80" hostnames: - "grpc.argocd.k8s.local" # securityContext: # fsGroup: 65532 workspaces: - name: local-maven-repo persistentVolumeClaim: claimName: tekton-workspaces # - name: local-maven-repo-config # emptyDir: {} # persistentVolumeClaim: # claimName: tekton-workspaces-config # - name: ssl-creds # secret: # secretName: local-gitlab-ssh-key params: # harbor image tag - name: image value: 172.16.100.46/test/java-demo - name: image-tag # 传入版本号 value: "v0.1" # git code path - name: gitrepourl # value: ssh://git@172.16.100.47:23857/devops/argocd-example.git value: $(tt.params.gitrepositoryurl) # Trigger传来的参数 - name: revision value: main # gitops yaml path - name: git_yaml_url value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: git_email value: linuxea@k8s.local - name: git_name value: marksugar - name: git_manifest_dir value: kustmoze/overlays/production/ - name: git_branch value: main # argocd auth - name: argocd_secret_auth value: argocd-auth - name: gitlab_configmap_ssh_auth value: gitlab-id-rsa # argocd env - name: argocd_url value: grpc.argocd.k8s.local - name: argocd_secret value: argocd-auth - name: app_name value: argocd/java-demo - name: app_revision value: main # sonarqube # - name: gitrepourl # value: $(params.gitrepourl) - name: sonar_project_type value: java # - name: sonar_project_name # value: $(params.sonar_project_name) - name: sonar_project_sources value: src - name: sonar_project_links_homepage value: https://www.linuxea.com - name: sonar_project_links_ci value: https://172.16.100.47/devops/argocd-example - name: sonar_host_url value: 172.16.100.79:9000 # echo "admin:admin.com" |base64 # sonarqube web username and password # YWRtaW46YWRtaW4uY29t - name: sonarAuth value: YWRtaW46YWRtaW4uY29t - name: qualityProfile # rule name value: java7.构建测试现在我们回到gitlab push一个事件查看日志[root@master-01 ~]# kubectl logs el-gitlab-listener-7678bc4b57-87hpl | grep eventlistener {"severity":"info","timestamp":"2023-07-15T14:43:36.308Z","logger":"eventlistener","caller":"resources/create.go:98","message":"Generating resource: kind: &APIResource{Name:pipelineruns,Namespaced:true,Kind:PipelineRun,Verbs:[delete deletecollection get list patch create update watch],ShortNames:[pr prs],SingularName:pipelinerun,Categories:[tekton tekton-pipelines],Group:tekton.dev,Version:v1beta1,StorageVersionHash:RcAKAgPYYoo=,}, name: mvn-pipelinerun","commit":"2ec8bc6-dirty"} {"severity":"info","timestamp":"2023-07-15T14:43:36.308Z","logger":"eventlistener","caller":"resources/create.go:106","message":"For event ID \"897871f2-2682-43b1-93c7-607b630ee5df\" creating resource tekton.dev/v1beta1, Resource=pipelineruns","commit":"2ec8bc6-dirty"} [root@master-01 ~]# 在返回到tekton ui查看已经开始构建。既然webhooks可以根据触发进行访问，那意味着，也可以模拟这个接口发起请求进行构建，更方便二次开发。参考https://github.com/tektoncd/triggers/blob/main/docs/triggerbindings.md#event-variable-interpolationhttps://docs.gitlab.com/ee/user/project/integrations/webhooks.html

linuxea:tekton与sonqube集成(6) 想要把sonarqube和流水线一起连接在tekton中，我们需要将这些步骤拆分，并联在每个阶段即可。大致流程如下：查找项目是否存在，如果不在就创建，并且添加预设或者安装好的sonar质量配置创建配置文件，开始扫描获取扫描结果，如果符合预期就继续，否则就中止1.环境安装我们开始安装一套sonarqube的服务，在集群外。mkdir -p /data/sonarqube/{conf,extensions,logs,data} chmod 777 -R /data/sonarqube/ chown -R 1000.1000 /data/sonarqubedocker-composeversion: '3.3' services: postgres: image: uhub.service.ucloud.cn/marksugar-k8s/postgres:13 restart: always container_name: postgres ports: - 5432:5432 volumes: - /data/postgres/postgresql/:/var/lib/postgresql - /data/postgres/data/:/var/lib/postgresql/data environment: TZ: Asia/Shanghai POSTGRES_USER: sonar POSTGRES_PASSWORD: sonar123 POSTGRES_DB: sonar sonarqube: image: uhub.service.ucloud.cn/marksugar-k8s/sonarqube:8.9.2-community restart: always container_name: sonarqube hostname: 172.16.100.79 ulimits: memlock: soft: -1 hard: -1 depends_on: - postgres volumes: - /data/sonarqube/extensions:/opt/sonarqube/extensions - /data/sonarqube/logs:/opt/sonarqube/logs - /data/sonarqube/data:/opt/sonarqube/data - /data/sonarqube/conf:/opt/sonarqube/conf ports: - 9000:9000 environment: SONARQUBE_JDBC_USERNAME: sonar SONARQUBE_JDBC_PASSWORD: sonar123 SONARQUBE_JDBC_URL: jdbc:postgresql://postgres:5432/sonar stop-timeout: 3600 ES_JAVA_OPTS: -Xms2048m -Xmx2048m2.插件安装我们需要安装一些插件：1.在Administration---> Marketplace，如下，点击·I understand the risk。输入chinese安装第一个2.根据版本支持下载插件 https://github.com/mc1arke/sonarqube-community-branch-plugin/releases/download/1.8.0/sonarqube-community-branch-plugin-1.8.0.jar到/data/sonarqube/extensions/plugins并且在conf下的配置文件中指定[root@Node-172_16_100_79 /data]# cat /data/sonarqube/conf/sonar.properties sonar.web.javaAdditionalOpts=-javaagent:./extensions/plugins/sonarqube-community-branch-plugin-1.8.0.jar=web sonar.ce.javaAdditionalOpts=-javaagent:./extensions/plugins/sonarqube-community-branch-plugin-1.8.0.jar=ce而后重启docker rm -f sonarqube && docker-compose -f /data/docker-compose.yaml up -d3.sonar-jacoco8.9.2默认安装了，如果没有就进行安装在https://github.com/SonarSource/sonar-jacoco/releases下载https://github.com/SonarSource/sonar-jacoco/releases/download/1.1.0.898/sonar-jacoco-plugin-1.1.0.898.jar，存放到/data/sonarqube/extensions/downloads此时页面提示，我们重启4.gitlab下载https://github.com/gabrie-allaigre/sonar-gitlab-plugin/releases/download/4.1.0/sonar-gitlab-plugin-4.1.0-SNAPSHOT.jar存放到/data/sonarqube/extensions/plugins5.右上角->我的账号->安全输入一个令牌，点击生成即可复制这个创建的值在后续中将会被用到6.拉取github的sonar-pmd-p3c项目后，编译完成，复制sonar-pmd-plugin-3.2.1.jar到/data/sonarqube/extensions/plugins中3.修改pom文件我们需要在项目中的pom.xml中添加jacoco的配置，如下：分别在 添加如下 <dependency> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.3</version> <scope>test</scope> </dependency> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency>在build plugins中添加如下 </plugin> <plugin> <groupId>org.jacoco</groupId> <artifactId>jacoco-maven-plugin</artifactId> <version>0.8.3</version> <configuration> <destFile>target/jacoco.exec</destFile> <detaFile>target/jacoco.exec</detaFile> </configuration> <executions> <execution> <id>jacoco-initialize</id> <goals> <goal>prepare-agent</goal> </goals> </execution> <execution> <id>jacoco-site</id> <phase>test</phase> <goals> <goal>report</goal> </goals> </execution> </executions> </plugin>4.创建质量阈添加条件我已经创建好了5.创建质量配置我们创建一个新配置: java而后激活p3c到java中此时，创建的java中已经有了56个规则如果不想用p3c还可以使用sonar-pmd-plugin，不过他们两个不兼容安装插件sonar-pmd-plugin-3.4.0.jar此页面https://github.com/jborgers/sonar-pmd/releases下载后存放到6.scanner顺序在开始扫描之前，我们需要通过api接口创建项目，添加质量配置1.查询项目curl --location --request POST 'https://172.16.100.79:9000/api/projects/search?projects=argocd-example' \ --header 'Authorization: Basic YWRtaW46YWRtaW4uY29t'YWRtaW46YWRtaW4uY29t密码是通过sonarqube的登录密码 : echo "username:password" | base64 得到的2.创建并添加质量配置curl --location --request POST 'https://172.16.100.79:9000/api/qualityprofiles/add_project?language=java&project=argocd-example&qualityProfile=java' \ --header 'Authorization: Basic YWRtaW46YWRtaW4uY29t'这两步分别我使用golang编写脚本替代了3.开始扫描第三步才是进行扫描如下参数:# 指定代码覆盖率工具为jacoco sonar.core.codeCoveragePlugin=jacoco # 指定exec二进制文件存放路径 sonar.jacoco.reportPaths=target/jacoco.exec和分支扫描-Dsonar.branch.name=master如下sonar-scanner -Dsonar.host.url=https://172.16.100.79:9000 \ -Dsonar.projectKey=java-20210928-demo \ -Dsonar.projectName=java-20210928-demo \ -Dsonar.projectVersion=1.0 \ -Dsonar.login=f0d470e82ff7e7ffc51b024f0ed603ee1097d6b0 \ -Dsonar.ws.timeout=30 \ -Dsonar.projectDescription="my first project!" \ -Dsonar.links.homepage=https://www.linuxea.com \ -Dsonar.sources=src \ -Dsonar.sourceEncoding=UTF-8 \ -Dsonar.java.binaries=target/classes \ -Dsonar.java.test.binaries=target/test-classes \ -Dsonar.java.surefire.report=target/surefire-reports \ -Dsonar.core.codeCoveragePlugin=jacoco \ -Dsonar.jacoco.reportPaths=target/jacoco.exec \ -Dsonar.branch.name=master4.获取扫描结果当扫描完成，我们需要得知扫描的结果，是否继续下一步curl --location --request GET 'https://172.16.100.79:9000/api/project_branches/list?project=argocd-example' \ --header 'Authorization: Basic YWRtaW46YWRtaW4uY29t'7.tekton上面，已经得知了sonarqube扫描的顺序和配置信息，现在我们将这些配置添加到tekton流水线中我们将此前创建的token写入configmapapiVersion: v1 kind: ConfigMap metadata: name: sonarqube-login data: login-auth: ZWY5NjcxOGE1YmM0N2FlYjhhYWIwYTY5ODk3YWEyY2E2ZjQzNWM1NAo= # echo "ef96718a5bc47aeb8aab0a69897aa2ca6f435c54" | base64 # ZWY5NjcxOGE1YmM0N2FlYjhhYWIwYTY5ODk3YWEyY2E2ZjQzNWM1NAo=在上面的顺序中，curl命令已经转换为go，分别打入镜像内，有以下：1.registry.cn-zhangjiakou.aliyuncs.com/marksugar/sonar-init:v1： 自行打包的，查询项目，创建项目，并且添加质量配置2.registry.cn-hangzhou.aliyuncs.com/marksugar-k8s/bash:5.2.15 ： 创建配置文件到workspace中3.uhub.service.ucloud.cn/marksugar-k8s/sonar-scanner-cli:4.8.0：官方的sonar-scanner-cli包，执行扫描任务的4.registry.cn-zhangjiakou.aliyuncs.com/marksugar/get-coverage-rate:v2： 自行打包的，获取sonarqube的扫描结果的那么现在的tekton的scanner的task如下：apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: sonarqube-scanner labels: app.kubernetes.io/version: "0.4" annotations: tekton.dev/displayName: "sonarqube scanner" spec: description: >- The task sonarqube workspaces: - name: maven-build-task params: - name: gitrepourl - name: sonar_project_type description: SonarQube project type | java | node | go | ... default: "" # - name: sonar_project_key # value: $(params.sonar_project_key) # - name: sonar_project_name # value: $(params.sonar_project_name) - name: sonar_project_sources description: sources default: "" - name: sonar_project_links_homepage description: jump links page default: "" - name: sonar_project_links_ci description: jump links default: "" - name: sonar_host_url description: sonar host address default: "" - name: revision description: sonar branch default: "" - name: SONAR_SCANNER_IMAGE description: "The sonarqube scanner CLI image which will run the scan" default: "uhub.service.ucloud.cn/marksugar-k8s/sonar-scanner-cli:4.8.0" - name: qualityProfile description: qualityProfile is web ui configure rule name - name: sonarAuth description: sonarqube web ui auth ; and username:password | base64 - name: git_branch description: steps: - name: sonar-init image: registry.cn-zhangjiakou.aliyuncs.com/marksugar/sonar-init:v1 # echo "admin:admin.com" |base64 # YWRtaW46YWRtaW4uY29t script: | #!/usr/bin/env sh export GITPATH=$(params.gitrepourl) export PROJECT_NAMES=$(echo $GITPATH | awk -F '[/]' '{print $NF}' | awk -F . '{print $1}' ) # export TOEKN=$(echo $SONAR_AUTH | base64 -d) cat > /sonar.ini << EOF hostip=$(params.sonar_host_url) projectsName=$PROJECT_NAMES sonarAuth=$(params.sonarAuth) language=$(params.sonar_project_type) qualityProfile=$(params.qualityProfile) EOF cat /sonar.ini /sonar-init - name: sonar-properties-create # image: registry.access.redhat.com/ubi8/ubi-minimal:8.2 image: registry.cn-hangzhou.aliyuncs.com/marksugar-k8s/bash:5.2.15 workingDir: $(workspaces.maven-build-task.path) env: - name: SONAR_AUTH valueFrom: configMapKeyRef: name: sonarqube-login key: login-auth - name: sonar_project_type value: $(params.sonar_project_type) # - name: sonar_project_key # value: $(params.sonar_project_key) # - name: sonar_project_name # value: $(params.sonar_project_name) - name: sonar_project_sources value: $(params.sonar_project_sources) - name: sonar_project_links_homepage value: $(params.sonar_project_links_homepage) - name: sonar_project_links_ci value: $(params.sonar_project_links_ci) - name: sonar_host_url value: $(params.sonar_host_url) # - name: sonar_login_token # value: $(params.sonar_login_token) script: | #!/usr/bin/env bash mkdir -p $(workspaces.maven-build-task.path)/sonarqube export GITPATH=$(params.gitrepourl) export PROJECT_NAMES=$(echo $GITPATH | awk -F '[/]' '{print $NF}' | awk -F . '{print $1}' ) export TOEKN=$(echo $SONAR_AUTH | base64 -d) if [[ "${sonar_project_type}" = "java" ]]; then echo "[i] project type java" cat > $(workspaces.maven-build-task.path)/sonarqube/sonar-project.properties <<EOF sonar.projectKey=${PROJECT_NAMES} sonar.projectName=${PROJECT_NAMES} sonar.projectVersion=1.0 sonar.ws.timeout=30 sonar.sources=${sonar_project_sources} sonar.projectDescription="${PROJECT_NAMES} is {$sonar_project_type}" sonar.links.homepage=${sonar_project_links_homepage} sonar.links.ci=${sonar_project_links_ci} sonar.sourceEncoding=UTF-8 sonar.host.url=https://${sonar_host_url} sonar.login=${TOEKN} sonar.java.binaries=target/classes sonar.java.test.binaries=target/test-classes sonar.java.surefire.report=target/surefire-reports sonar.core.codeCoveragePlugin=jacoco sonar.jacoco.reportPaths=target/jacoco.exec sonar.branch.name=$(params.revision) EOF fi if [[ "${sonar_project_type}" = "npm" ]]; then echo "[i] project type java" cat > $(workspaces.maven-build-task.path)/sonarqube/sonar-project.properties <<EOF sonar.projectKey=${PROJECT_NAMES} sonar.projectName=${PROJECT_NAMES} sonar.projectVersion=1.0 sonar.ws.timeout=30 sonar.sources=${sonar_project_sources} sonar.projectDescription="${PROJECT_NAMES} is {$sonar_project_type}" sonar.links.homepage=${sonar_project_links_homepage} sonar.links.ci=${sonar_project_links_ci} sonar.sourceEncoding=UTF-8 sonar.host.url=https://${sonar_host_url} sonar.login=${TOEKN} sonar.core.codeCoveragePlugin=jacoco sonar.jacoco.reportPaths=target/jacoco.exec sonar.branch.name=$(params.revision) EOF fi - name: sonar-scan image: $(params.SONAR_SCANNER_IMAGE) workingDir: $(workspaces.maven-build-task.path)/argocd-example/demo # command: script: | #!/usr/bin/env bash ls $(workspaces.maven-build-task.path)/sonarqube/ sonar-scanner -Dproject.settings=$(workspaces.maven-build-task.path)/sonarqube/sonar-project.properties - name: coverage-rate image: registry.cn-zhangjiakou.aliyuncs.com/marksugar/get-coverage-rate:v2 script: | #!/usr/bin/env sh export GITPATH=$(params.gitrepourl) export PROJECT_NAMES=$(echo $GITPATH | awk -F '[/]' '{print $NF}' | awk -F . '{print $1}' ) # export TOEKN=$(echo $SONAR_AUTH | base64 -d) cat > /sonar-coverage.ini << EOF hostip=$(params.sonar_host_url) projectsName=$PROJECT_NAMES sonarAuth=$(params.sonarAuth) branch=$(params.git_branch) EOF cat /sonar-coverage.ini /get-coverage-rate在pipeline中，sonarqube-scanner在代码克隆后进行扫描，如果返回的QualityGateStatus等于OK就可以通过，pipeline的sonarqube-scanner如下 # sonarqube - name: sonarqube-scanner runAfter: - git-clone taskRef: name: sonarqube-scanner workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo params: - name: gitrepourl value: $(params.gitrepourl) - name: sonar_project_type value: $(params.sonar_project_type) # - name: sonar_project_name # value: $(params.sonar_project_name) - name: sonar_project_sources value: $(params.sonar_project_sources) - name: sonar_project_links_homepage value: $(params.sonar_project_links_homepage) - name: sonar_project_links_ci value: $(params.sonar_project_links_ci) - name: git_branch value: $(params.git_branch) - name: sonar_host_url value: $(params.sonar_host_url) - name: revision value: $(params.revision) # - name: sonar_login_token # value: $(params.sonar_login_token) - name: qualityProfile value: $(params.qualityProfile) - name: sonarAuth value: $(params.sonarAuth)在Pipelinrun中，我们需要添加所需的变量。整体如下apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: mvn-pipelinerun spec: serviceAccountName: build-sa pipelineRef: name: mvn-pipeline podTemplate: hostAliases: - ip: "172.16.100.80" hostnames: - "grpc.argocd.k8s.local" # securityContext: # fsGroup: 65532 workspaces: - name: local-maven-repo persistentVolumeClaim: claimName: tekton-workspaces # - name: local-maven-repo-config # emptyDir: {} # persistentVolumeClaim: # claimName: tekton-workspaces-config # - name: ssl-creds # secret: # secretName: local-gitlab-ssh-key params: # harbor image tag - name: image value: 172.16.100.46/test/java-demo - name: image-tag # 传入版本号 value: "v0.1" # git code path - name: gitrepourl value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: revision value: main # gitops yaml path - name: git_yaml_url value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: git_email value: linuxea@k8s.local - name: git_name value: marksugar - name: git_manifest_dir value: kustmoze/overlays/production/ - name: git_branch value: main # argocd auth - name: argocd_secret_auth value: argocd-auth - name: gitlab_configmap_ssh_auth value: gitlab-id-rsa # argocd env - name: argocd_url value: grpc.argocd.k8s.local - name: argocd_secret value: argocd-auth - name: app_name value: argocd/java-demo - name: app_revision value: main # sonarqube # - name: gitrepourl # value: $(params.gitrepourl) - name: sonar_project_type value: java # - name: sonar_project_name # value: $(params.sonar_project_name) - name: sonar_project_sources value: src - name: sonar_project_links_homepage value: https://www.linuxea.com - name: sonar_project_links_ci value: https://172.16.100.47/devops/argocd-example - name: sonar_host_url value: 172.16.100.79:9000 # echo "admin:admin.com" |base64 # sonarqube web username and password # YWRtaW46YWRtaW4uY29t - name: sonarAuth value: YWRtaW46YWRtaW4uY29t - name: qualityProfile # rule name value: java开始构建后，如果QualityGateStatus的扫描结果不等于OK，阶段就会终止只有QualityGateStatus的扫描结果等于OK。管道顺利运行上述完成后，回到soanrqube新代码sonarqube和tekton的短暂配置，到此结束。而深入的pmd规则或者p3c规则，则需要开发人员进行按需定制才行。

linuxea:tekton与argocd集成(5) 在上个阶段，镜像已经存放在harbor了，此时我门要通过一些方式将它发布到k8s，我门可以使用kubectl apply -f yaml的方式直接更新，也可以使用helm或者kustmoize编排工具进行直接使用。并且这些yaml可以被gitlab管理。或者我门在进一步，直接采用argocd来进行管理和同步。1. kustomize我们现在需要准备一个Kustomize的配置清单，将清单存放在git上便于argocd读取kustmoize的git地址是：https://gitee.com/marksugar/devops-note/tree/master/kustmoze2. argocd1.安装argocd我们安装一版新的argocdkubectl create namespace argocd wget https://raw.githubusercontent.com/argoproj/argo-cd/v2.7.6/manifests/install.yaml sed -i 's@ghcr.io/dexidp/dex:v2.36.0@uhub.service.ucloud.cn/marksugar-k8s/dex:v2.36.0@g' v2.7.6.yaml sed -i 's@haproxy:2.6.14-alpine@uhub.service.ucloud.cn/marksugar-k8s/haproxy:2.6.14-alpine@g' v2.7.6.yaml sed -i 's@quay.io/argoproj/argocd:v2.7.6@uhub.service.ucloud.cn/marksugar-k8s/argocd:v2.7.6@g' v2.7.6.yaml sed -i 's@redis:7.0.11-alpine@uhub.service.ucloud.cn/marksugar-k8s/redis:7.0.11-alpine@g' v2.7.6.yaml kubectl -n argocd apply -f v2.7.6.yaml配置ingressapiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: argocd-server-http-ingress namespace: argocd annotations: nginx.ingress.kubernetes.io/force-ssl-redirect: "true" nginx.ingress.kubernetes.io/backend-protocol: "HTTP" spec: ingressClassName: nginx rules: - http: paths: - path: / pathType: Prefix backend: service: name: argocd-server port: name: http host: argocd.k8s.local tls: - hosts: - argocd.k8s.local secretName: argocd-secret # do not change, this is provided by Argo CD --- apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: argocd-server-grpc-ingress namespace: argocd annotations: nginx.ingress.kubernetes.io/backend-protocol: "GRPC" spec: ingressClassName: nginx rules: - http: paths: - path: / pathType: Prefix backend: service: name: argocd-server port: name: https host: grpc.argocd.k8s.local tls: - hosts: - grpc.argocd.k8s.local secretName: argocd-secret # do not change, this is provided by Argo CD而后，我门在deploymen中给argocd-server的args中添加--insecure， 如下：... containers: - args: - /usr/local/bin/argocd-server - --insecure env: ...修改登录密码[root@master-01 ~/tekton]# argocd account bcrypt --password www.linuxea.com && echo $2a$10$FaBGIv4CCgaMWs8j6eF1iOfuc4BzTV891irdcmNXOIqRCuOqueVgu kubectl -n argocd patch secret argocd-secret \ -p '{"stringData": { "admin.password": "$2a$10$FaBGIv4CCgaMWs8j6eF1iOfuc4BzTV891irdcmNXOIqRCuOqueVgu", "admin.passwordMtime": "'$(date +%FT%T%Z)'" }}'配置hosts解析echo "172.16.100.80 grpc.argocd.k8s.local" >> /etc/hosts用新密码进修登录[root@master-01 ~/tekton]# argocd login grpc.argocd.k8s.local --username admin --password www.linuxea.com WARNING: server certificate had error: x509: certificate is valid for ingress.local, not grpc.argocd.k8s.local. Proceed insecurely (y/n)? y 'admin:login' logged in successfully Context 'grpc.argocd.k8s.local' updatedargocd登录后会失效，因此我们添加一个计划任务自动登录login.shargocd login grpc.argocd.k8s.local --username admin --password www.linuxea.com --insecurecrontab1 */6 * * * /bin/bash /root/argocd/login.sh2.定义集群仍然需要重申下环境变量的配置当前KUBECONFIG指定的参数将会是添加的集群export KUBECONFIG=$HOME/.kube/config-1.25.11-dev而后在查看当前的集群[root@master-01 ~/argocd]# kubectl config get-contexts -o name context-cluster1将context-cluster1 添加到当前的argocd环境中[root@master-01 ~/argocd]# argocd cluster add context-cluster1 --insecure WARNING: This will create a service account `argocd-manager` on the cluster referenced by context `context-cluster1` with full cluster level privileges. Do you want to continue [y/N]? y INFO[0001] ServiceAccount "argocd-manager" created in namespace "kube-system" INFO[0001] ClusterRole "argocd-manager-role" created INFO[0001] ClusterRoleBinding "argocd-manager-role-binding" created INFO[0006] Created bearer token secret for ServiceAccount "argocd-manager" Cluster 'https://172.16.100.80:6443' added现在就有了一个自己创建的集群如果要添加其他的集群也是如此。3.定义存储库创建应用之前我们需要添加git库。我们需要让git仓库信任，添加认证信息这个git仓库也就是我存放所有内容的，正常情况下应该是会有一个单独的存储库用来存放所有的yaml清单argocd repo add ssh://git@172.16.100.47:23857/devops/argocd-example.git --ssh-private-key-path ~/.ssh/id_rsa --insecure-ignore-host-key添加完成后在settings->repositories界面将会看到一个存储库4.定义项目我们创建一个名称空间，指定project的资源 就运行在这里, 而后创建一个名称为linuxea的AppProject。其中声明了权限apiVersion: v1 kind: Namespace metadata: name: linuxea-dev --- apiVersion: argoproj.io/v1alpha1 kind: AppProject metadata: name: linuxea namespace: argocd spec: description: dev-env sourceRepos: - '*' destinations: - namespace: linuxea-dev server: 'https://172.16.100.80:6443' # clusterResourceWhitelist: # - group: '' # kind: Namespace # namespaceResourceBlacklist: # - group: '' # kind: ResourceQuota # - group: '' # kind: LimitRange # - group: '' # kind: NetworkPolicy - group: 'apps' kind: 'Deployment' - group: '' kind: 'Service' - group: '' kind: 'ConfigMap' - group: 'networking.k8s.io' kind: 'Ingress' - group: 'apps' kind: StatefulSet - group: '' kind: 'Secret' # roles: # - name: read-only # description: Read-only privileges to my-project # policies: # - p, proj:my-project:read-only, applications, get, my-project/*, allow # groups: # - test-env # - name: ci-role # description: Sync privileges for guestbook-dev # policies: # - p, proj:my-project:ci-role, applications, sync, my-project/guestbook-dev, allow # jwtTokens: # - iat: 1535390316开始创建[root@master-01 ~/argocd/appject]# kubectl apply -f linuxea.yaml appproject.argoproj.io/linuxea created创建完成后在settings/projects中就会有刚创建的Linux的projetcs了5.定义应用一个最小的应用程序规范如下：apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: java-demo namespace: argocd labels: marksugar/linuxea: prod # 标签 spec: project: linuxea # 定义的项目名 source: repoURL: ssh://git@172.16.100.47:23857/devops/argocd-example.git # git地址 targetRevision: main # git分支 path: kustmoze/overlays/production/ # git路径 destination: server: https://172.16.100.80:6443 # k8s api namespace: linuxea-dev # 名称空间 #syncPolicy: # 打开自动更新 # automated: # prune: false # selfHeal: false 为了避免不必要的自动更新导致的健康状态检测异常，我们选择手动更新开始创建[root@master-01 ~/argocd/appject]# kubectl apply -f java-demo.yaml application.argoproj.io/java-demo created那么现在已经创建完成可以通过命令查看[root@master-01 ~/argocd/appject]# argocd app wait argocd/java-demo --timeout 120 .... [root@master-01 ~/argocd/appject]# kubectl -n linuxea-dev get pod NAME READY STATUS RESTARTS AGE production-java-demo-7bddf56f67-k6kff 1/1 Running 0 2m2s production-java-demo-7bddf56f67-nb86g 1/1 Running 0 2m2s production-java-demo-7bddf56f67-pwcjw 1/1 Running 0 2m2s3. tekton的argocd上面我们准备了argocd的文件清单，并且配置好了。现在我们需要在tekton中实现这些。此前的tekton已经从代码clone、mvn build、docker build、docker push结束。 欠缺的是 deploy 和 rollback 部分我门的步骤如下：0，给节点添加本地的私有镜像1，登录argocd2, 克隆git编排清单清单提供拉取镜像的认证文件3，更新并同步配置清单4，手动同步argocd app我们此前已经配置过了本地的镜像containerd的配置，并且还使用nerdctl拉取了镜像测试。1.创建必要的认证信息与以往不同的是，这次使用ConfigMap而不是直接在script中进修添加，我们使用base64进修简单的处理创建密钥认证ssh-keyget -t rsa而后将id_rsa.pub存放到gitlab，并且创建如下的configcat > ~/.ssh/config <<EOF Host * StrictHostKeyChecking no # IdentityFile /tekton/creds/.ssh/id_rsa EOF分别将上述文件使用base64简单的编码cat ~/.ssh/id_rsa | base64 -w0 cat ~/.ssh/config | base64 -w0将得到的结果存放到configmap--- apiVersion: v1 kind: ConfigMap metadata: name: gitlab-id-rsa data: id-rsa-key: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBbjRsUzJYeHY3K2NDUG4zMmE1eHRPRW5HWUI1VU1tSzFGZDRGK3Q1b3pqVUlCN0g0CmxDTjlSeThNQXhaa2FGYXFJTGt3c3lWd2FiUVphWjdEK213aUUwalVsZWVxQ01CV1R0Y1JsN25nUlAxSnFVNzkKc2Z0R2cyUVZzMEZibmg5VXRHRHRzZVRtR2JGMFVGR2RlU2h5SnQwMUdlbk1VQ3VJUDFHWnpOeWErQWNxbEgrZgpLSzJDd1dZT2Y0VVgvMjZoQ2NNTUtkWnFoYWhaS0FPQzRTZU91Uy8vT0FTR2J5WENWQW1FVlIwUkpNNTd5ZjJVCm5mQnJldk9JY2pVZGEyRHM3Uk9iRkhZZkxBNWRrQzFEVVVDU0l5OU85NmZickFiS2ZSQnpHYU93dk50SEdoQ28KdWxuQzZld3JUdzZrOU5hZm9jMmEvUWU4YWF4Rkx4ZVhyVVArdVFJREFRQUJBb0lCQURkV3ZodGMwOE5Hd1h1RgozQTExNUZqckZsblBDMnV5MjVlclBTM0pLbGVsQTVVUHdlYzQ3RUxkNmUvRU82NEpxZGZTQlRlUEhCQStmYllxCmNWYVRYYnFNdjlrRDAvSlhMVmtKdHp1SEtXa2s1d0ZJdWkyYnVOam94Ykk1VUp0bnlNME05a2FIM04wNnRkc0sKaEhJN2VtVkt4Qm1xQk1vbVg0dC9OYmRmS280ditmcWsvVElZUHhvaytBdjFYUDlzZjZyc1FaOHRXK0VRS05MUAp3a0hZVDVUQUJHSjRpYmFpT2UwcFhobkhMYmxMSWowWnhMVFo0cDBnUzdXbUZkR2M1VFBsMDBQcVJ4NHpRa1Z1CkU2L2FnVWZ5RFNWbkk0ZUdmTWN6WXlyQ25MT1gzc1IrQWZJd1djUzZDNmJXMVRSWDY2eTd5dGZTbmdHeitHTnQKK3JWVmsrVUNnWUVBekFMdlBMZVYrNXhoc3J3Nk1Va0dJbnUrNlUwM3p2ZkpyT1RrWk9xSlV1UlF6dE5UQnhaZwpBWjczM05LalpLNFlDYm1yL2lWNW91Qm1JNlNkcE5VVFQ0OTViUktxbEtrdE5nWmlMVHk5Qm10c3FUNEFOL2JUCk54bkFMMzFNMlZUbUIyQUtpZlUxWG5jMHBHd3c2aUNYQWpUZFRaV0pGb0EvYnVsWlhBd0xPVDhDZ1lFQXlERDYKTlIxWlV4MFduelR2aEhNMURXWFpNOWE0QzhSVUQzN3lEM1BCeXFyY1ovQnNOM25oaHhvaWMvb1pkc1BqRkMxQwpndUZqaTFmR1R6MFVXaUVMUGxYRG9WWGpKOFNiVHJKaXlMZDd0TnhzUmZHa1lzSFkrZXloV2dIcTJscXN0ak9GCjZ3NStlSXhrK25NVkZ0U3ZyNUk0R05kczdNazFERlV3ZEE1UkVnY0NnWUJQSVdSQ0hNTmtFTWdrVjdZS245UnQKVTZvOU8rSkZ1UDdVdlFPSjBSN3FzL3hUL012YmwrWXRPc2ZSTXpLRlByRzhNL1J1WTRMR2xsdUh2N1lWMVQvawpRRFpOQkV4OTFSRGdQNkdkQWs0ZlMxT2VZWm9jR3ZUYURuYm85S1QyTjJQeHdHTFdGL0pYYmMxUmFWV2Z1bnhZCk1zRVQ4c2gxbmREODZDM3M4Zzc0YVFLQmdDL0s4WkVJcHJDZSt5MmJGNnVKVFpzYk81V1NwM0czdDFIVHZycE4KcW5CV0JqU1NRMk9qSjJKM3JNMy9ZV0RuTnlrei9QTCtHQ1BHR3NRRUV2M0pjY081K2ZCa2JzeXB0V0krV0RNUApDdXVNekJTT3NlNFoxVkVsV3dmLytTK1VBak9sYlUxR3NvN1pCYXA5R3dYT1ZQc1hiMmtPMVNSYWpOMmE0T3gwCkZvcTFBb0dCQUt1VlpDN1lEaCttZjhRT2IvS3FwMFFDbzJWUko5bzNEeDFwc3BGamJ4dDNDMFFwRmJLZHJlN0IKdzNIZ2dRdVRpMjU0RnV5blpsSGp3cjBtRnRBcXAvQmRONTVXbUhiTy9uTXMzWVkvblZCYUYvYldmTU1sTWxVOApXSFd4ZDhQM0wwZ3ZrMXh0emVNa0dGdDBlSmo3ckQ0MWJXcjVZU080aGdGTEsxRVB4ZDFOCi0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg== config: SG9zdCAqClN0cmljdEhvc3RLZXlDaGVja2luZyBubwo=而后在创建一个argocd的apiVersion: v1 kind: Secret metadata: name: argocd-auth type: Opaque stringData: username: admin password: www.linuxea.com2.kustmoze我们照搬在不使用tekton的流程进行处理A.克隆yaml仓库清单B.set image镜像tagC.推送yaml仓库清单完成如上三步就得到了如下的task:apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: change-manifests spec: workspaces: - name: maven-build-task # 通过pipeline传递过来的name params: - name: git_yaml_url description: Git repository containing manifest files to update - name: git_email default: pipeline@k8s.local - name: git_name default: Tekton Pipeline - name: git_manifest_dir description: Manifests files dir - name: git_branch description: Manifests files branch env - name: tool_image default: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/git-kustomize-jq:alpine-3.17.4-latest - name: image description: Deploy docker image tag steps: - name: git-manifests-push # image: $(params.tool_image) image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/git-kustomize-jq:v2 workingDir: $(workspaces.maven-build-task.path) env: - name: GIT_ID_RSA valueFrom: configMapKeyRef: name: gitlab-id-rsa key: id-rsa-key - name: GIT_SSH_CONF valueFrom: configMapKeyRef: name: gitlab-id-rsa key: config script: | #!/usr/bin/env sh # create id_rsa export GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" rm -rf ~/.ssh mkdir -p ~/.ssh echo ${GIT_ID_RSA} | base64 -d > ~/.ssh/id_rsa echo ${GIT_SSH_CONF} | base64 -d > ~/.ssh/config chmod 600 ~/.ssh/id_rsa mkdir -p yaml export GITPATH=$(params.git_yaml_url) export PROJECT_NAME=$(echo $GITPATH | awk -F '[/]' '{print $NF}' | awk -F . '{print $1}' ) git config --global user.email "$(params.git_email)" git config --global user.name "$(params.git_name)" if [ -d yaml/$PROJECT_NAME ]; then cd yaml/$PROJECT_NAME && git pull && cd $(params.git_manifest_dir) git checkout -b "$(params.git_branch)" kustomize edit set image demo/linuxea=$(params.image) git add . git commit -m "kustomize change image tag $(params.image)" git push else git clone --branch "$(params.git_branch)" --depth 1 $(params.git_yaml_url) yaml/$PROJECT_NAME cd yaml/$PROJECT_NAME/$(params.git_manifest_dir) git config --global user.email "$(params.git_email)" git config --global user.name "$(params.git_name)" kustomize edit set image demo/linuxea=$(params.image) git add . git commit -m "kustomize change image tag $(params.image)" git push #git clone --branch "$(params.git_branch)" --depth 1 https://${GIT_USERNAME}:${GIT_PASSWORD}@$(params.git_yaml_url) yaml fi3.argocd此时我们已经将镜像的tag修改后推送到了git仓库的yaml清单中。现在我们只需要进行argo app sync即可apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: argo-sync-app spec: volumes: - name: argocd-secret secret: secretName: $(inputs.params.argocd_secret) params: - name: argocd_url description: "The URL of the ArgoCD server" - name: argocd_secret description: "The secret containing the username and password for the tekton task to connect to argo" - name: app_name description: "The name of the argo app to update" - name: app_revision default: "HEAD" description: "The revision of the argo app to update" steps: - name: deploy image: registry.cn-zhangjiakou.aliyuncs.com/marksugar/argocd:v2.7.2 volumeMounts: - name: argocd-secret mountPath: /var/secret script: | #!/usr/bin/env sh echo "starting argocd sync app" argocd login --insecure $(params.argocd_url) --username $(/bin/cat /var/secret/username) --password $(/bin/cat /var/secret/password) argocd app sync $(params.app_name) --revision $(params.app_revision) argocd app wait $(params.app_name) --health --timeout 120而后我们在Pipline中进行修改# workspace-mvn-pipeline.yaml apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: mvn-pipeline spec: workspaces: # 声明 workspaces - name: local-maven-repo params: - name: gitrepourl type: string description: The git repo URL to clone from. - name: revision type: string default: "master" - name: image type: string - name: image-tag type: string default: "v0.1.0" - name: registry_url type: string default: "172.16.100.46" - name: registry_mirror type: string default: "https://ot2k4d59.mirror.aliyuncs.com/" - name: insecure_registry type: string default: "172.16.100.46" # gitops yaml - name: git_yaml_url description: Git repository containing manifest files to update - name: git_email default: linuxea@k8s.local - name: git_name default: Tekton Pipeline - name: git_manifest_dir description: Manifests files dir - name: git_branch description: Manifests files branch env # argocd env - name: argocd_url description: argocd_url - name: argocd_secret description: argocd_secret - name: app_name description: app_name - name: app_revision description: app_revision tasks: # git clone - name: git-clone taskRef: name: git-clone workspaces: - name: output workspace: local-maven-repo params: - name: gitrepourl value: $(params.gitrepourl) - name: revision value: $(params.revision) # image tag - name: get-build-id taskRef: name: generate-build-id params: - name: base-version value: $(params.image-tag) # mvn build - name: maven-build # 构建任务 taskRef: name: maven-java # 引用上面的 mvn 任务 runAfter: - get-build-id - git-clone workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo # - name: maven-settings # workspace: local-maven-repo-config # docker build and push - name: docker-build-push taskRef: name: docker-build-push runAfter: ["maven-build"] # 需要 build 任务执行完成后 params: - name: image value: "$(params.image):$(tasks.get-build-id.results.build-id)" - name: registry_url value: $(params.registry_url) - name: insecure_registry value: $(params.insecure_registry) - name: registry_mirror value: $(params.registry_mirror) workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo # git yaml change - name: git-change-manifests taskRef: name: change-manifests runAfter: - docker-build-push params: - name: image value: "$(params.image):$(tasks.get-build-id.results.build-id)" - name: git_yaml_url value: $(params.git_yaml_url) - name: git_email value: $(params.git_email) - name: git_name value: $(params.git_name) - name: git_manifest_dir value: $(params.git_manifest_dir) - name: git_branch value: $(params.git_branch) workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo # argocd app sync - name: argo-sync-app taskRef: name: argo-sync-app runAfter: - git-change-manifests params: - name: argocd_url value: $(params.argocd_url) - name: argocd_secret value: $(params.argocd_secret) - name: app_name value: $(params.app_name) - name: app_revision value: $(params.app_revision)在pipeline中添加了阶段后，还需要在pipelinerun中添加apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: mvn-pipelinerun spec: serviceAccountName: build-sa pipelineRef: name: mvn-pipeline podTemplate: hostAliases: - ip: "172.16.100.80" hostnames: - "grpc.argocd.k8s.local" # securityContext: # fsGroup: 65532 workspaces: - name: local-maven-repo persistentVolumeClaim: claimName: tekton-workspaces # - name: local-maven-repo-config # emptyDir: {} # persistentVolumeClaim: # claimName: tekton-workspaces-config # - name: ssl-creds # secret: # secretName: local-gitlab-ssh-key params: # harbor image tag - name: image value: 172.16.100.46/test/java-demo - name: image-tag # 传入版本号 value: "v0.1" # git code path - name: gitrepourl value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: revision value: main # gitops yaml path - name: git_yaml_url value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: git_email value: linuxea@k8s.local - name: git_name value: marksugar - name: git_manifest_dir value: kustmoze/overlays/production/ - name: git_branch value: main # auth - name: argocd_secret_auth value: argocd-auth - name: gitlab_configmap_ssh_auth value: gitlab-id-rsa # argocd env - name: argocd_url value: grpc.argocd.k8s.local - name: argocd_secret value: argocd-auth - name: app_name value: argocd/java-demo - name: app_revision value: main

linuxea: tekton ci Workspaces(4) Workspaces 是共享卷的声明，在流水线定义中，Workspaces 可以作为共享卷传递给相关任务，同时多个任务提供相同的 Workspaces 的时候，它们就可以从相同的 Volumes 中读取和写入数据。Tekton提供了workspaces的支持，我门只需要添加pv或者pvc或者configmap和Secret，根据使用任选其一。我门希望是可以复用Maven 和 NPM的构建依赖，以便于加速构建。而后在pipelin的task中添加指定的Workspaces 即可。要在流水线中构建 Maven 项目，需要定义一个 Maven 的 Task 任务gcr.io/cloud-builders/mvn:3.8-jdk-8 to uhub.service.ucloud.cn/marksugar-k8s/mvn:3.8-jdk-8已知问题：1，一个task不支持两个pvc(3480,6543,6740)，因此在mvn中，我取消了2个pvc的使用，另外一个配置使用的是临时的，后又改成只使用一个，但是你可以使用configmap实现2，git-clone是一个有问题的组件(5611)，推荐使用的是0.6，但是经过测试还是不行，因此我参考配置后重新构建了git克隆的方式，手动添加并修改了自定义的克隆在开始之前，我门生成一个jar包https://start.spring.io/，我使用的jdk8，生成的包将用来构建。并且，在这个案例中，我将使用本地的资源进行配置，而不是公共仓库的。nfs pvc我门说过，Workspaces是用来共享卷的，因此如果你要使用Workspaces的内容就需要声明引入他，这意味着我门需要提供并创建一个持久化的卷。我门使用nfs来实现它。1.安装nfscentos7yum -y install nfs-utils rpcbind mkdir /data/nfs-share chown nobody.nobody /data/nfs-share chmod 755 /data/nfs-share echo '/data/nfs-share 172.16.100.0/24(rw,sync,no_root_squash)' >> /etc/exports echo "MOUNTD_PORT=4001" >> /etc/sysconfig/nfs echo "STATD_PORT=4002" >> /etc/sysconfig/nfs echo "LOCKD_TCPPORT=4003" >> /etc/sysconfig/nfs echo "LOCKD_UDPPORT=4003" >> /etc/sysconfig/nfs echo "RQUOTAD_PORT=4004" >> /etc/sysconfig/nfs systemctl start rpcbind.service systemctl enable rpcbind systemctl restart rpcbind.service systemctl restart nfs-utils.service systemctl restart nfs-server.service rpcinfo -p|grep nfs而后，给每个node节点安装nfs-utils，使用showmount -e IPADDRESS测试。2.创建pvcapiVersion: v1 kind: PersistentVolume metadata: name: tekton-workspaces namespace: default spec: capacity: storage: 200Gi accessModes: - ReadWriteMany persistentVolumeReclaimPolicy: Retain nfs: server: 172.16.100.49 path: /data/nfs-share/tekton-workspaces-build # mkdir -p /data/nfs-share/tekton-workspaces-build --- kind: PersistentVolumeClaim apiVersion: v1 metadata: name: tekton-workspaces namespace: default spec: accessModes: - ReadWriteMany resources: requests: storage: 200Gi创建完成，在后续将被使用。1.harbor我门首先创建好harbor仓库，创建一个test项目harbor-https1.创建harbor证书#!/bin/bash HARBOR_URL=172.16.100.46 sudo openssl genrsa -out ca.key 4096 sudo openssl req -x509 -new -nodes -sha512 -days 3650 \ -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$HARBOR_URL" \ -key ca.key \ -out ca.crt sudo openssl genrsa -out $HARBOR_URL.key 4096 openssl req -sha512 -new \ -subj "/C=CN/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=$HARBOR_URL" \ -key $HARBOR_URL.key \ -out $HARBOR_URL.csr cat > v3.ext <<-EOF authorityKeyIdentifier=keyid,issuer basicConstraints=CA:FALSE keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment extendedKeyUsage = serverAuth subjectAltName = IP:$HARBOR_URL EOF openssl x509 -req -sha512 -days 3650 \ -extfile v3.ext \ -CA ca.crt -CAkey ca.key -CAcreateserial \ -in $HARBOR_URL.csr \ -out $HARBOR_URL.crt mkdir -p /etc/harbor/ cp $HARBOR_URL.crt /etc/harbor/ cp $HARBOR_URL.key /etc/harbor/2.修改harbor证书配置位置hostname: 172.16.100.46 http: port: 80 https: port: 443 certificate: /etc/harbor/172.16.100.46.crt private_key: /etc/harbor/172.16.100.46.key而后打开页面就是https自动跳转了接着我们修改containerd配置... [plugins."io.containerd.grpc.v1.cri".registry] [plugins."io.containerd.grpc.v1.cri".registry.auths] [plugins."io.containerd.grpc.v1.cri".registry.configs] [plugins."io.containerd.grpc.v1.cri".registry.configs."172.16.100.46".tls] insecure_skip_verify = true [plugins."io.containerd.grpc.v1.cri".registry.configs."harbor.k8s.local".auth] username = "admin" password = "Harbor12345" [plugins."io.containerd.grpc.v1.cri".registry.headers] [plugins."io.containerd.grpc.v1.cri".registry.mirrors] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."easzlab.io.local:5000"] endpoint = ["https://easzlab.io.local:5000"] [plugins."io.containerd.grpc.v1.cri".registry.mirrors."172.16.100.46"] endpoint = ["https://172.16.100.46"] ...nerdctl3.安装nerdctlwget https://github.com/containerd/nerdctl/releases/download/v1.4.0/nerdctl-1.4.0-linux-amd64.tar.gz tar xf nerdctl-1.4.0-linux-amd64.tar.gz cp -r nerdctl /usr/local/sbin/ nerdctl login -u admin --insecure-registry 172.16.100.46此时已经将认证存放在本地cat > ~/.docker/config.json << EOF { "auths": { "172.16.100.46": { "auth": "YWRtaW46SGFyYm9yMTIzNDU=" } } } EOF现在就可以镜像拉取nerdctl pull --insecure-registry 172.16.100.46/test/java-demo:v0.1-20230706-212558pod拉取创建secretkubectl create secret docker-registry harbor-auth --docker-server=https://172.16.100.46 --docker-username=admin --docker-password=Harbor12345 --docker-email=marksugar@linuxea.io -n default2.auth接着，创建认证使用的必要资源-镜像仓库。这个sa将会在pipeline和pipelinerun中进行绑定使用apiVersion: v1 kind: ServiceAccount metadata: name: build-sa secrets: - name: local-harbor-auth --- apiVersion: v1 kind: Secret metadata: name: local-harbor-auth annotations: tekton.dev/docker-0: https://172.16.100.46 type: kubernetes.io/basic-auth stringData: username: admin password: Harbor12345 3.镜像id仍然使用上面的task生成镜像id，其中传入了params参数，build-id，base-version，这些也可以从pipeline和pipelinerun中传递过来的apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: generate-build-id spec: description: >- Given a base version, this task generates a unique build id by appending the base-version to the current timestamp. params: - name: base-version description: Base product version type: string default: "1.0" results: - name: timestamp description: Current timestamp - name: build-id description: ID of the current build steps: - name: get-timestamp image: uhub.service.ucloud.cn/marksugar-k8s/bash:5.0.18 script: | #!/usr/bin/env bash sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories apk add -U tzdata cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime ts=`date "+%Y%m%d-%H%M%S"` echo "Current Timestamp: ${ts}" echo ${ts} | tr -d "\n" | tee $(results.timestamp.path) - name: get-buildid image: uhub.service.ucloud.cn/marksugar-k8s/bash:5.0.18 script: | #!/usr/bin/env bash ts=`cat $(results.timestamp.path)` buildId=$(inputs.params.base-version)-${ts} echo ${buildId} | tr -d "\n" | tee $(results.build-id.path)4.git clone接着 配置git clone task，其中gitrepourl和revision也是从pipeline和pipelinerun传来的参数，分别代表git地址和分支。而在script中我将id_rsa直接写入到这个task中了，而不是通过其他方式传递进来。这些配置可以通过挂载的方式引入。但是我更喜欢直接写入在workspaces中使用的output，而这个workspaces的output是从pipeline获取的名字，其值指的是pipelinerun传入的一个nfs pvcapiVersion: tekton.dev/v1beta1 kind: Task metadata: name: git-clone description: Read and display README file. spec: params: - name: gitrepourl description: The git repo to clone. default: Some message. type: string - name: revision default: main type: string description: "" workspaces: - name: output steps: - name: read image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/git:alpine-3.17.4 workingDir: $(workspaces.output.path) script: | #!/usr/bin/env sh export GIT_SSH_COMMAND="ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no" echo $GIT_SSH_COMMAND rm -rf ~/.ssh mkdir -p ~/.ssh cat > ~/.ssh/id_rsa << EOF -----BEGIN OPENSSH PRIVATE KEY----- b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn NhAAAAAwEAAQAAAYEA54hh+noXtIX0PAvpbkJgz7BQHT0Bd6EvUpAMfx5GEntDvcEQIA0q ikHbLfjQ2/Dp7CT10AN1sMlMkfaC6pA59fK5QZ5/qBs/BWCvgoDgCsSXwORos5O91m04FF Nlw0gX4VIbbHnLCEgS1uLj0VKFAGrLpM0kBAiBspAHFkRS0GlcO3hfTrPf4SyvolPAWp5w oroFlvGT21e6vZ21uxu4zKxP0d5vDtwruRMD4fm4Q/UHj1+neu3TwimLYqxNk9SCBNT3O4 ptApRwEB/fhFjCZxiZqPEwvCMO6ga44CHLMuY90GXpNq0fc7GBsgAk18erRXgcVpjrS/+/ U/5unRQNzxE60EWUQZC61VCjK0/QK752YmFEh9/W6wyJMOVWS5TKpBfwjE6Il7ADDeUzy5 0koX6mN6cq1QOAnn4+vF50hl+u0M25UZMXojJZYDognqGRZupt13tLK+Y1fmZFJxPI9QF7 E7IMsP5yoDq0VXMtb60dMLFYjT73llTXR6QHvR1fAAAFkCic0yUonNMlAAAAB3NzaC1yc2 EAAAGBAOeIYfp6F7SF9DwL6W5CYM+wUB09AXehL1KQDH8eRhJ7Q73BECANKopB2y340Nvw 6ewk9dADdbDJTJH2guqQOfXyuUGef6gbPwVgr4KA4ArEl8DkaLOTvdZtOBRTZcNIF+FSG2 x5ywhIEtbi49FShQBqy6TNJAQIgbKQBxZEUtBpXDt4X06z3+Esr6JTwFqecKK6BZbxk9tX ur2dtbsbuMysT9Hebw7cK7kTA+H5uEP1B49fp3rt08Ipi2KsTZPUggTU9zuKbQKUcBAf34 RYwmcYmajxMLwjDuoGuOAhyzLmPdBl6TatH3OxgbIAJNfHq0V4HFaY60v/v1P+bp0UDc8R OtBFlEGQutVQoytP0Cu+dmJhRIff1usMiTDlVkuUyqQX8IxOiJewAw3lM8udJKF+pjenKt UDgJ5+PrxedIZfrtDNuVGTF6IyWWA6IJ6hkWbqbdd7SyvmNX5mRScTyPUBexOyDLD+cqA6 tFVzLW+tHTCxWI0+95ZU10ekB70dXwAAAAMBAAEAAAGBAIcx/+Q+frEOtvqJChkz8UvU7D LmeIDeQb+4FVP03X3VNE0M+tl8Vqa8pbtGWs7S8gTN4CtCmepM4MUqaG+HIBEZVkuWvXvv zPDa8CEIF8qx2NrthKW3U1ZuGBzmRbmt0yPiydihlYiUbUvE2LpVsQv5UEg+8RTZ0sHtj/ XUl3bSzuR6+Uv/hG77pOLofJ1tft/GN5VYIrollAk8me1T1qjMxR8K/1ZyAaXkEJRXON2V qoBxnl4NmWT61UUcz3/OybB4x38l+prXQCAFAJ7hwRyw13/4Lzoo9o2Op8uzafggdNeUJQ EBxoCGzuucen3gMolEbFWMgMQu6+caRhDrPhTAlUbvBTaImHV4gYyFwAlfxX34rYHjb4JL nAbPqE8RP6p9bNhC8ksBDr0+muwvA+p2UAAADBAOpVOVDs/xWwfWrHEXEhgEb2zMvKP9Ew iWoHjQj7gQMxwtOvXz/jaA1Sp5Fm5za9v9EiZNtVBMZxgxiLZjSlyFKiAshZirZ9lFPP+t R2ff+SdCeQrwyIdrSQznuUBZWFCKc9eky8mdRGrfAafZgidl+BcfRhVQqMS9W418fVMzYb Vsq65Z3xSlzB0knqWl5YfRJ7upLIWrsVMBjixvizUwl/ZWXe9lmbb7nfIjv44PGtCDBjxC EQ0RzNJ1Lf2pUPcwAAABV1c2VydEBMQVBUT1AtUEhLRjg5MkgBAgME -----END OPENSSH PRIVATE KEY----- EOF cat > ~/.ssh/config <<EOF Host * StrictHostKeyChecking no # IdentityFile /tekton/creds/.ssh/id_rsa EOF chmod 600 ~/.ssh/id_rsa export GITPATH=$(params.gitrepourl) export PROJECT_NAME=$(echo $GITPATH | awk -F '[/]' '{print $NF}' | awk -F . '{print $1}' ) if [ -d $PROJECT_NAME ]; then cd $PROJECT_NAME && git pull else git clone -b $(params.revision) $(params.gitrepourl) fi5.mvn build我从https://hub.tekton.dev/tekton/task/git-clone获得了下面这串代码，也正如我开头所说，原本的代码不能再我的环境成功使用，于是我修改了如下的代码。 mvn-settings提供了settings.xml，maven-build中是一些常用的构建命令。而这两个阶段的配置的结果都会再maven-build-task的这个workspaces中共享。你也可以将构建的参数作为形参传递进来，但鉴于我的命令过长，我直接写死了settings.xml中的逻辑是判断使用本地的nexus还是其他的，通过if判断变量来进行sed插入。需要注意，这个settings.xml配置文件如果存在将不会进行逻辑处理。apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: maven-java spec: # resources: # inputs: # - name: source # type: git workspaces: - name: maven-build-task # 通过pipeline传递过来的name # mountPath: /workspace/repo # - name: maven-settings # path: ~/ params: - name: ALIYUN_PATH description: aliyun mirrors url type: string # default: "https://maven.aliyun.com/nexus/content/groups/public/" #old default: "https://maven.aliyun.com/repository/public" - name: NEXUS_USER description: local nexus auth user type: string default: "" - name: NEXUS_PASSWORD description: local nexus auth user password type: string default: "" - name: NEXUS_PATH description: local nexus http url type: string default: "172.16.15.136:8081/repository/maven2-group/" steps: - name: mvn-settings #image: registry.access.redhat.com/ubi8/ubi-minimal:8.2 image: registry.cn-hangzhou.aliyuncs.com/marksugar-k8s/bash:5.2.15 script: | #!/usr/bin/env bash [[ -f $(workspaces.maven-build-task.path)/settings.xml ]] && \ echo 'using existing $(workspaces.maven-build-task.path)/settings.xml' && exit 0 cat > $(workspaces.maven-build-task.path)/settings.xml <<EOF <settings> <servers> <!-- The servers added here are generated from environment variables. Don't change. --> <!-- ### SERVER's USER INFO from ENV ### --> </servers> <localRepository>$(workspaces.maven-build-task.path)/Maven/repository</localRepository> <mirrors> <!-- The mirrors added here are generated from environment variables. Don't change. --> <!-- ### mirrors from ENV ### --> </mirrors> <proxies> <!-- The proxies added here are generated from environment variables. Don't change. --> <!-- ### HTTP proxy from ENV ### --> </proxies> </settings> EOF xml="" if [ -n "$(params.ALIYUN_PATH)" ]; then xml=" <mirror>\ <id>aliyun</id>\ <url>$(params.ALIYUN_PATH)</url>\ <mirrorOf>*</mirrorOf>\ </mirror>" sed -i "s|<!-- ### mirrors from ENV ### -->|$xml|" $(workspaces.maven-build-task.path)/settings.xml fi if [ -n "$(params.NEXUS_USER)" -a -n "$(params.NEXUS_PASSWORD)" ]; then xml="<server>\ <id>nexus</id>" xml="$xml\ <username>$(params.NEXUS_USER)</username>\ <password>$(params.NEXUS_PASSWORD)</password>" xml="$xml\ </server>" sed -i "s|<!-- ### SERVER's USER INFO from ENV ### -->|$xml|" $(workspaces.maven-build-task.path)/settings.xml if [ -n "$(params.NEXUS_PATH)" ]; then xml=" <mirror>\ <id>nexus</id>\ <url>https://$(params.NEXUS_PATH)</url>\ <mirrorOf>central</mirrorOf>\ </mirror>" sed -i "s|<!-- ### mirrors from ENV ### -->|$xml|" $(workspaces.maven-build-task.path)/settings.xml fi fi - name: maven-build image: uhub.service.ucloud.cn/linuxea/maven:3.8.6-jdk-8 workingDir: $(workspaces.maven-build-task.path)/argocd-example/demo script: | #!/usr/bin/env bash # cd java-demo/java-demo # cd argocd-example/java-demo mkdir -p $(workspaces.maven-build-task.path)/m2 mvn clean install -Dautoconfig.skip=true \ -Dmaven.repo.local=$(workspaces.maven-build-task.path)/m2 \ -Dmaven.wagon.http.ssl.insecure=true \ -Dmaven.wagon.http.ssl.allowall=true \ -Dmaven.test.skip=false \ -Dmaven.test.failure.ignore=true -s $(workspaces.maven-build-task.path)/settings.xml6.docker buildworkspaces中的name字段为maven-build-task是从pipelinerun传入的pvc使用的名称，params字段为传入的参数，如registry_url等。在workingDir中是根据实际项目地址拼接的路径，整个过程是sidecars用的是registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/docker:20.10.16-dind的镜像。相信用过肯定知道这是在做什么。# task-docker-build.yaml apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: docker-build-push spec: workspaces: - name: maven-build-task # 通过pipeline传递过来的name # resources: # inputs: # 定义输入资源 # - name: source # 源代码仓库 # type: git params: - name: image description: Reference of the image docker will produce. - name: builder_image description: The location of the docker builder image. default: docker:stable - name: dockerfile description: Path to the Dockerfile to build. default: ./Dockerfile - name: context description: Path to the directory to use as context. default: . - name: build_extra_args description: Extra parameters passed for the build command when building images. default: "" - name: push_extra_args description: Extra parameters passed for the push command when pushing images. default: "" - name: insecure_registry description: Allows the user to push to an insecure registry that has been specified default: "" - name: registry_mirror description: Specific the docker registry mirror default: "" - name: registry_url description: private docker images registry url steps: - name: docker-build # 构建步骤 image: $(params.builder_image) env: - name: DOCKER_HOST # 用 TLS 形式通过 TCP 链接 sidecar value: tcp://localhost:2376 - name: DOCKER_TLS_VERIFY # 校验 TLS value: "1" - name: DOCKER_CERT_PATH # 使用 sidecar 守护进程生成的证书 value: /certs/client workingDir: $(workspaces.maven-build-task.path)/argocd-example/demo script: | # docker 构建命令 docker login $(params.registry_url) docker build \ $(params.build_extra_args) \ --no-cache \ -f $(params.dockerfile) -t $(params.image) $(params.context) volumeMounts: # 声明挂载证书目录 - mountPath: /certs/client name: dind-certs - name: docker-push # image: $(params.builder_image) env: - name: DOCKER_HOST value: tcp://localhost:2376 - name: DOCKER_TLS_VERIFY value: "1" - name: DOCKER_CERT_PATH value: /certs/client workingDir: $(workspaces.maven-build-task.path)/argocd-example/demo script: | # 推送 docker 镜像 docker login $(params.registry_url) echo $(params.image) docker push $(params.push_extra_args) $(params.image) volumeMounts: - mountPath: /certs/client name: dind-certs sidecars: # sidecar 模式，提供 docker daemon服务，实现真正的 DinD 模式 - image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/docker:20.10.16-dind name: server args: - --storage-driver=vfs - --userland-proxy=false - --debug - --insecure-registry=$(params.insecure_registry) - --registry-mirror=$(params.registry_mirror) securityContext: privileged: true env: - name: DOCKER_TLS_CERTDIR # 将生成的证书写入与客户端共享的路径 value: /certs volumeMounts: - mountPath: /certs/client name: dind-certs readinessProbe: # 等待 dind daemon 生成它与客户端共享的证书 periodSeconds: 1 exec: command: ["ls", "/certs/client/ca.pem"] volumes: # 使用 emptyDir 的形式即可 - name: dind-certs emptyDir: {}7.pipelinepipeline是将task整理在一个管道内，声明workspaces的名称，声明params参数，定义tasks的传入的workspaces和参数，以及每个task的等待，或者说启动顺序# workspace-mvn-pipeline.yaml apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: mvn-pipeline spec: workspaces: # 声明 workspaces - name: local-maven-repo # - name: local-maven-repo-config # resources: # 声明使用的资源 # - name: java-app-git # type: git params: - name: gitrepourl type: string description: The git repo URL to clone from. - name: revision type: string default: "master" - name: image type: string - name: image-tag type: string default: "v0.1.0" - name: registry_url type: string default: "172.16.100.46" - name: registry_mirror type: string default: "https://ot2k4d59.mirror.aliyuncs.com/" - name: insecure_registry type: string default: "172.16.100.46" tasks: # git clone - name: git-clone taskRef: name: git-clone workspaces: - name: output workspace: local-maven-repo params: - name: gitrepourl value: $(params.gitrepourl) - name: revision value: $(params.revision) # image tag - name: get-build-id taskRef: name: generate-build-id params: - name: base-version value: $(params.image-tag) # mvn build - name: maven-build # 构建任务 taskRef: name: maven-java # 引用上面的 mvn 任务 runAfter: - get-build-id - git-clone workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo # - name: maven-settings # workspace: local-maven-repo-config # docker build and push - name: docker-build-push # 测试任务 taskRef: name: docker-build-push runAfter: ["maven-build"] # 需要 build 任务执行完成后 params: - name: image value: "$(params.image):$(tasks.get-build-id.results.build-id)" - name: registry_url value: $(params.registry_url) - name: insecure_registry value: $(params.insecure_registry) - name: registry_mirror value: $(params.registry_mirror) workspaces: # 传递 workspaces - name: maven-build-task workspace: local-maven-repo8.pipelineRun在pipelineRun中，定义了此前创建的build-sa, pipelineRef注明使用的是上面的mvn-pipeline这个Pipeline，workspaces使用的pvc是上面创建好的nfs的pvc: tekton-workspaces，而这个pvc的名称在管道中是叫local-maven-repo，local-maven-repo也因此被 每个task声明和传递，而params中的环境变量是通过pipelinrun传递到pipeline后，在分别给每个需要的task传递使用的。apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: mvn-pipelinerun spec: serviceAccountName: build-sa pipelineRef: name: mvn-pipeline # podTemplate: # securityContext: # fsGroup: 65532 workspaces: - name: local-maven-repo persistentVolumeClaim: claimName: tekton-workspaces # - name: local-maven-repo-config # emptyDir: {} # persistentVolumeClaim: # claimName: tekton-workspaces-config # - name: ssl-creds # secret: # secretName: local-gitlab-ssh-key params: - name: image value: 172.16.100.46/test/java-demo - name: image-tag # 传入版本号 value: "v0.1" - name: gitrepourl value: ssh://git@172.16.100.47:23857/devops/argocd-example.git - name: revision value: main构建完成后，通过tr查看[root@Node-172_16_100_100 ~/tekton]# tkn tr list NAME STARTED DURATION STATUS mvn-pipelinerun-docker-build-push 10 minutes ago 4m49s Succeeded mvn-pipelinerun-maven-build 10 minutes ago 21s Succeeded mvn-pipelinerun-get-build-id 20 minutes ago 16s Succeeded mvn-pipelinerun-git-clone 20 minutes ago 21s Succeeded [root@Node-172_16_100_100 ~/tekton]# tkn pr list NAME STARTED DURATION STATUS mvn-pipelinerun 20 minutes ago 5m32s Succeeded如下生成镜像id[root@Node-172_16_100_100 ~/tekton]# tkn tr logs mvn-pipelinerun-get-build-id [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/main/x86_64/APKINDEX.tar.gz [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/community/x86_64/APKINDEX.tar.gz [get-timestamp] (1/1) Installing tzdata (2023c-r1) [get-timestamp] OK: 11 MiB in 19 packages [get-timestamp] Current Timestamp: 20230625-160257 [get-timestamp] 20230625-160257 [get-buildid] v0.1-20230625-160257克隆代码[root@Node-172_16_100_100 ~/tekton]# tkn tr logs mvn-pipelinerun-git-clone [read] ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no [read] Warning: Permanently added '[172.16.100.47]:23857' (ED25519) to the list of known hosts. [read] Already up to date.mven build[root@Node-172_16_100_100 ~/tekton]# tkn tr logs mvn-pipelinerun-maven-build [mvn-settings] using existing /workspace/maven-build-task/settings.xml [maven-build] [INFO] Scanning for projects... [maven-build] [INFO] [maven-build] [INFO] -----------------------< com.linuxea:java-demo >------------------------ [maven-build] [INFO] Building java-demo 0.0.1-SNAPSHOT [maven-build] [INFO] --------------------------------[ jar ]--------------------------------- [maven-build] [INFO] [maven-build] [INFO] --- maven-clean-plugin:3.2.0:clean (default-clean) @ java-demo --- [maven-build] [INFO] Deleting /workspace/maven-build-task/argocd-example/demo/target [maven-build] [INFO] [maven-build] [INFO] --- maven-resources-plugin:3.2.0:resources (default-resources) @ java-demo --- [maven-build] [INFO] Using 'UTF-8' encoding to copy filtered resources. [maven-build] [INFO] Using 'UTF-8' encoding to copy filtered properties files. [maven-build] [INFO] Copying 1 resource [maven-build] [INFO] Copying 0 resource [maven-build] [INFO] [maven-build] [INFO] --- maven-compiler-plugin:3.10.1:compile (default-compile) @ java-demo --- [maven-build] [INFO] Changes detected - recompiling the module! [maven-build] [INFO] Compiling 1 source file to /workspace/maven-build-task/argocd-example/demo/target/classes [maven-build] [INFO] [maven-build] [INFO] --- maven-resources-plugin:3.2.0:testResources (default-testResources) @ java-demo --- [maven-build] [INFO] Using 'UTF-8' encoding to copy filtered resources. [maven-build] [INFO] Using 'UTF-8' encoding to copy filtered properties files. [maven-build] [INFO] skip non existing resourceDirectory /workspace/maven-build-task/argocd-example/demo/src/test/resources [maven-build] [INFO] [maven-build] [INFO] --- maven-compiler-plugin:3.10.1:testCompile (default-testCompile) @ java-demo --- [maven-build] [INFO] Changes detected - recompiling the module! [maven-build] [INFO] Compiling 1 source file to /workspace/maven-build-task/argocd-example/demo/target/test-classes [maven-build] [INFO] [maven-build] [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ java-demo --- [maven-build] [INFO] [maven-build] [INFO] ------------------------------------------------------- [maven-build] [INFO] T E S T S [maven-build] [INFO] ------------------------------------------------------- [maven-build] [INFO] Running com.example.demo.DemoApplicationTests [maven-build] 08:03:22.820 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating CacheAwareContextLoaderDelegate from class [org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate] [maven-build] 08:03:22.828 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating BootstrapContext using constructor [public org.springframework.test.context.support.DefaultBootstrapContext(java.lang.Class,org.springframework.test.context.CacheAwareContextLoaderDelegate)] [maven-build] 08:03:22.852 [main] DEBUG org.springframework.test.context.BootstrapUtils - Instantiating TestContextBootstrapper for test class [com.example.demo.DemoApplicationTests] from class [org.springframework.boot.test.context.SpringBootTestContextBootstrapper] [maven-build] 08:03:22.861 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Neither @ContextConfiguration nor @ContextHierarchy found for test class [com.example.demo.DemoApplicationTests], using SpringBootContextLoader [maven-build] 08:03:22.864 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.example.demo.DemoApplicationTests]: class path resource [com/example/demo/DemoApplicationTests-context.xml] does not exist [maven-build] 08:03:22.864 [main] DEBUG org.springframework.test.context.support.AbstractContextLoader - Did not detect default resource location for test class [com.example.demo.DemoApplicationTests]: class path resource [com/example/demo/DemoApplicationTestsContext.groovy] does not exist [maven-build] 08:03:22.864 [main] INFO org.springframework.test.context.support.AbstractContextLoader - Could not detect default resource locations for test class [com.example.demo.DemoApplicationTests]: no resource found for suffixes {-context.xml, Context.groovy}. [maven-build] 08:03:22.865 [main] INFO org.springframework.test.context.support.AnnotationConfigContextLoaderUtils - Could not detect default configuration classes for test class [com.example.demo.DemoApplicationTests]: DemoApplicationTests does not declare any static, non-private, non-final, nested classes annotated with @Configuration. [maven-build] 08:03:22.898 [main] DEBUG org.springframework.test.context.support.ActiveProfilesUtils - Could not find an 'annotation declaring class' for annotation type [org.springframework.test.context.ActiveProfiles] and class [com.example.demo.DemoApplicationTests] [maven-build] 08:03:22.945 [main] DEBUG org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider - Identified candidate component class: file [/workspace/maven-build-task/argocd-example/demo/target/classes/com/example/demo/DemoApplication.class] [maven-build] 08:03:22.946 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Found @SpringBootConfiguration com.example.demo.DemoApplication for test class com.example.demo.DemoApplicationTests [maven-build] 08:03:23.016 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - @TestExecutionListeners is not present for class [com.example.demo.DemoApplicationTests]: using defaults. [maven-build] 08:03:23.017 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Loaded default TestExecutionListener class names from location [META-INF/spring.factories]: [org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener, org.springframework.boot.test.autoconfigure.webservices.client.MockWebServiceServerTestExecutionListener, org.springframework.test.context.web.ServletTestExecutionListener, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener, org.springframework.test.context.event.ApplicationEventsTestExecutionListener, org.springframework.test.context.support.DependencyInjectionTestExecutionListener, org.springframework.test.context.support.DirtiesContextTestExecutionListener, org.springframework.test.context.transaction.TransactionalTestExecutionListener, org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener, org.springframework.test.context.event.EventPublishingTestExecutionListener] [maven-build] 08:03:23.025 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Skipping candidate TestExecutionListener [org.springframework.test.context.transaction.TransactionalTestExecutionListener] due to a missing dependency. Specify custom listener classes or make the default listener classes and their required dependencies available. Offending class: [org/springframework/transaction/TransactionDefinition] [maven-build] 08:03:23.025 [main] DEBUG org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Skipping candidate TestExecutionListener [org.springframework.test.context.jdbc.SqlScriptsTestExecutionListener] due to a missing dependency. Specify custom listener classes or make the default listener classes and their required dependencies available. Offending class: [org/springframework/transaction/interceptor/TransactionAttribute] [maven-build] 08:03:23.025 [main] INFO org.springframework.boot.test.context.SpringBootTestContextBootstrapper - Using TestExecutionListeners: [org.springframework.test.context.web.ServletTestExecutionListener@428640fa, org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener@d9345cd, org.springframework.test.context.event.ApplicationEventsTestExecutionListener@2d710f1a, org.springframework.boot.test.mock.mockito.MockitoTestExecutionListener@29215f06, org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener@59505b48, org.springframework.test.context.support.DirtiesContextTestExecutionListener@4efac082, org.springframework.test.context.event.EventPublishingTestExecutionListener@6bd61f98, org.springframework.boot.test.mock.mockito.ResetMocksTestExecutionListener@48aca48b, org.springframework.boot.test.autoconfigure.restdocs.RestDocsTestExecutionListener@13fd2ccd, org.springframework.boot.test.autoconfigure.web.client.MockRestServiceServerResetTestExecutionListener@b9b00e0, org.springframework.boot.test.autoconfigure.web.servlet.MockMvcPrintOnlyOnFailureTestExecutionListener@506ae4d4, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverTestExecutionListener@7d4f9aae, org.springframework.boot.test.autoconfigure.webservices.client.MockWebServiceServerTestExecutionListener@72e5a8e] [maven-build] 08:03:23.027 [main] DEBUG org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener - Before test class: context [DefaultTestContext@60bd273d testClass = DemoApplicationTests, testInstance = [null], testMethod = [null], testException = [null], mergedContextConfiguration = [WebMergedContextConfiguration@121314f7 testClass = DemoApplicationTests, locations ='{}', classes = '{class com.example.demo.DemoApplication}', contextInitializerClasses = '[]', activeProfiles = '{}', propertySourceLocations = '{}', propertySourceProperties = '{org.springframework.boot.test.context.SpringBootTestContextBootstrapper=true}', contextCustomizers = set[org.springframework.boot.test.context.filter.ExcludeFilterContextCustomizer@1b1473ab, org.springframework.boot.test.json.DuplicateJsonObjectContextCustomizerFactory$DuplicateJsonObjectContextCustomizer@7880cdf3, org.springframework.boot.test.mock.mockito.MockitoContextCustomizer@0, org.springframework.boot.test.web.client.TestRestTemplateContextCustomizer@3c130745, org.springframework.boot.test.autoconfigure.actuate.metrics.MetricsExportContextCustomizerFactory$DisableMetricExportContextCustomizer@1622f1b, org.springframework.boot.test.autoconfigure.properties.PropertyMappingContextCustomizer@0, org.springframework.boot.test.autoconfigure.web.servlet.WebDriverContextCustomizerFactory$Customizer@30ee2816, org.springframework.boot.test.context.SpringBootTestArgs@1, org.springframework.boot.test.context.SpringBootTestWebEnvironment@1554909b], resourceBasePath = 'src/main/webapp', contextLoader = 'org.springframework.boot.test.context.SpringBootContextLoader', parent = [null]], attributes = map['org.springframework.test.context.web.ServletTestExecutionListener.activateListener' -> true]], class annotated with @DirtiesContext [false] with mode [null]. [maven-build] [maven-build] . ____ _ __ _ _ [maven-build] /\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \ [maven-build] ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \ [maven-build] \\/ ___)| |_)| | | | | || (_| | ) ) ) ) [maven-build] ' |____| .__|_| |_|_| |_\__, | / / / / [maven-build] =========|_|==============|___/=/_/_/_/ [maven-build] :: Spring Boot :: (v2.7.1) [maven-build] [maven-build] 2023-06-25 08:03:23.209 INFO 51 --- [ main] com.example.demo.DemoApplicationTests : Starting DemoApplicationTests using Java 1.8.0_342 on mvn-pipelinerun-maven-build-pod with PID 51 (started by root in /workspace/maven-build-task/argocd-example/demo) [maven-build] 2023-06-25 08:03:23.209 INFO 51 --- [ main] com.example.demo.DemoApplicationTests : No active profile set, falling back to 1 default profile: "default" [maven-build] 2023-06-25 08:03:24.188 INFO 51 --- [ main] com.example.demo.DemoApplicationTests : Started DemoApplicationTests in 1.134 seconds (JVM running for 1.724) [maven-build] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.568 s - in com.example.demo.DemoApplicationTests [maven-build] [INFO] [maven-build] [INFO] Results: [maven-build] [INFO] [maven-build] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 [maven-build] [INFO] [maven-build] [INFO] [maven-build] [INFO] --- maven-jar-plugin:3.2.2:jar (default-jar) @ java-demo --- [maven-build] [INFO] Building jar: /workspace/maven-build-task/argocd-example/demo/target/java-demo-0.0.1-SNAPSHOT.jar [maven-build] [INFO] [maven-build] [INFO] --- spring-boot-maven-plugin:2.7.1:repackage (repackage) @ java-demo --- [maven-build] [INFO] Replacing main artifact with repackaged archive [maven-build] [INFO] [maven-build] [INFO] --- maven-install-plugin:2.5.2:install (default-install) @ java-demo --- [maven-build] [INFO] Installing /workspace/maven-build-task/argocd-example/demo/target/java-demo-0.0.1-SNAPSHOT.jar to /workspace/maven-build-task/m2/com/linuxea/java-demo/0.0.1-SNAPSHOT/java-demo-0.0.1-SNAPSHOT.jar [maven-build] [INFO] Installing /workspace/maven-build-task/argocd-example/demo/pom.xml to /workspace/maven-build-task/m2/com/linuxea/java-demo/0.0.1-SNAPSHOT/java-demo-0.0.1-SNAPSHOT.pom [maven-build] [INFO] ------------------------------------------------------------------------ [maven-build] [INFO] BUILD SUCCESS [maven-build] [INFO] ------------------------------------------------------------------------ [maven-build] [INFO] Total time: 4.750 s [maven-build] [INFO] Finished at: 2023-06-25T08:03:25Z [maven-build] [INFO] ------------------------------------------------------------------------docker build and push[root@Node-172_16_100_100 ~/tekton]# tkn tr logs mvn-pipelinerun-docker-build-push [docker-build] Authenticating with existing credentials... [docker-build] WARNING! Your password will be stored unencrypted in /root/.docker/config.json. [docker-build] Configure a credential helper to remove this warning. See [docker-build] https://docs.docker.com/engine/reference/commandline/login/#credentials-store [docker-build] [docker-build] Login Succeeded [docker-build] total 28 [docker-build] -rw-r--r-- 1 root root 764 Jun 21 09:42 Dockerfile [docker-build] -rwxr-xr-x 1 root root 10284 Jun 21 09:42 mvnw [docker-build] -rw-r--r-- 1 root root 6734 Jun 21 09:42 mvnw.cmd [docker-build] drwxr-xr-x 3 root root 29 Jun 25 02:40 org [docker-build] -rw-r--r-- 1 root root 1236 Jun 25 03:11 pom.xml [docker-build] drwxr-xr-x 4 root root 30 Jun 21 09:42 src [docker-build] drwxr-xr-x 9 root root 243 Jun 25 08:03 target [docker-build] /workspace/maven-build-task/argocd-example/demo [docker-build] Sending build context to Docker daemon 17.77MB [docker-build] Step 1/8 : FROM registry.cn-hangzhou.aliyuncs.com/marksugar/jdk:8u202 [docker-build] 8u202: Pulling from marksugar/jdk [docker-build] 59bf1c3509f3: Pulling fs layer [docker-build] e6a31d9a895c: Pulling fs layer [docker-build] 59bf1c3509f3: Verifying Checksum [docker-build] 59bf1c3509f3: Download complete [docker-build] 59bf1c3509f3: Pull complete [docker-build] e6a31d9a895c: Verifying Checksum [docker-build] e6a31d9a895c: Download complete [docker-build] e6a31d9a895c: Pull complete [docker-build] Digest: sha256:516cd5bd65041d4b00587127417c1a9a3aea970fa533d330f60b07395aa5e5ca [docker-build] Status: Downloaded newer image for registry.cn-hangzhou.aliyuncs.com/marksugar/jdk:8u202 [docker-build] ---> 5919494d49c0 [docker-build] Step 2/8 : MAINTAINER by mark [docker-build] ---> Running in 389213cdfde9 [docker-build] Removing intermediate container 389213cdfde9 [docker-build] ---> 156303217e69 [docker-build] Step 3/8 : ENV JAVA_OPTS=" -server -Xms2048m -Xmx2048m -Xmn512m -Xss256k -XX:+UseConcMarkSweepGC -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/data/logs" MY_USER=linuxea MY_USER_ID=316 [docker-build] ---> Running in 603f88dc4220 [docker-build] Removing intermediate container 603f88dc4220 [docker-build] ---> e497f3c9cfeb [docker-build] Step 4/8 : RUN addgroup -g ${MY_USER_ID} -S ${MY_USER} && adduser -u ${MY_USER_ID} -S -H -s /sbin/nologin -g 'java' -G ${MY_USER} ${MY_USER} && mkdir /data/logs -p [docker-build] ---> Running in 546a2a4184bf [docker-build] Removing intermediate container 546a2a4184bf [docker-build] ---> c2819b82d54d [docker-build] Step 5/8 : COPY target/*.jar /data/ [docker-build] ---> a64079063010 [docker-build] Step 6/8 : WORKDIR /data [docker-build] ---> Running in 6aa81bb7dc46 [docker-build] Removing intermediate container 6aa81bb7dc46 [docker-build] ---> 4b327c71ca83 [docker-build] Step 7/8 : USER linuxea [docker-build] ---> Running in b3b59503c716 [docker-build] Removing intermediate container b3b59503c716 [docker-build] ---> 581ec55ea9c9 [docker-build] Step 8/8 : CMD java ${JAVA_OPTS} -jar *.jar [docker-build] ---> Running in 90d06d689c44 [docker-build] Removing intermediate container 90d06d689c44 [docker-build] ---> 8c24653cbf98 [docker-build] Successfully built 8c24653cbf98 [docker-build] Successfully tagged 172.16.100.46/test/java-demo:v0.1-20230625-160257 [docker-push] Authenticating with existing credentials... [docker-push] WARNING! Your password will be stored unencrypted in /root/.docker/config.json. [docker-push] Configure a credential helper to remove this warning. See [docker-push] https://docs.docker.com/engine/reference/commandline/login/#credentials-store [docker-push] [docker-push] Login Succeeded [docker-push] 172.16.100.46/test/java-demo:v0.1-20230625-160257 [docker-push] The push refers to repository [172.16.100.46/test/java-demo] [docker-push] 0b761ff8c9e2: Preparing [docker-push] e7dc3a4f5772: Preparing [docker-push] 788766eb7d3e: Preparing [docker-push] 8d3ac3489996: Preparing [docker-push] e7dc3a4f5772: Pushed [docker-push] 8d3ac3489996: Pushed [docker-push] 0b761ff8c9e2: Pushed [docker-push] 788766eb7d3e: Pushed [docker-push] v0.1-20230625-160257: digest: sha256:c36dc94db60e5942a0bbacbf7c92d048cd9ac23d61a6bf12ee958226d6498349 size: 1161 [sidecar-server] 2023/06/25 08:08:17 Exiting...镜像推送完成后再harbor可以查看到此时tekton dashboard也可以看到构建的细节现在，通过tekton 进行打包，打包镜像，推送镜像已经完成。

linuxea:skywalking最新版本9.6.0链路和日志关联 我们使用一个前后端分离的权限管理开源系统来模拟skywalking的日志和链路关联行为。为什么我不在去讨论OpenTelemetry了？因为skywalking更加简单的实现我需要的功能。观测性实际需要付出的东西是很多的，怎么样用最简单的办法来实现这部分的功能的便利，是我考虑的事情。而OpenTelemetry目前只有signoz能够做到所谓的，目前最容易做到的日志，链路，指标的关联。但是signoz并非像skywalking这样简单。值得一提的是，仅全链路观测角度来看，nodejs方面，无论是OpenTelemetry还是单独的skywalking提供的解决方案，都是不太容易使用的，更别提需要手动埋点写入一大堆只有业务开发才能明白的链路请求生产的上下文span。而自动埋点是永远都需要的，恰巧OpenTelemetry和skywalking都支持 。因此，这些内容都基于自动埋点。当然，暂且不会说什么nodejs相关的，这是个头疼的事情。我们需要让子弹多飞一会。写这篇的目的在于，我又倾向skywalking了。在9.6.0中界面UI简洁了，这也是我为什么又选择使用skywalking重要的原因 之一。开始我使用的是https://gitee.com/y_project/RuoYi-Vue，由java SpringBoot和Vue组成。我将演示如何进行日志和链路关联。其他的语言支持也同样简单 ，这可能更多的来着社区wusheng的支持，感谢他。也感谢罗总提醒我使用这个项目来测试。在skywalking.apache.org/downloads/下载最新的9.0.0 java-agent jar包。这里的版本号和后续pom文件中的版本号相关联而日志如何修改和配置取决于你使用了什么组件,skywalking支持log4j，log4j2和logback，以RuoYi-Vue项目为例，他使用的是logback，那么就参考logback如果没有其他说明，下面所有内容出现的配置都是修改RuoYi-Vue项目我们快速拉起一套skywalking的环境，使用9.6.0最新的版本。docker-compose如下：version: '2' services: skywalking_oap: image: uhub.service.ucloud.cn/marksugar-k8s/skywalking-oap-server:9.6.0 #dapache/skywalking-oap-server:9.6.0 container_name: skywalking_oap ports: - "11800:11800" - "12800:12800" depends_on: - skywalking_elasticsearch environment: SW_STORAGE: elasticsearch SW_STORAGE_ES_CLUSTER_NODES: skywalking_elasticsearch:9200 SW_HEALTH_CHECKER: default SW_TELEMETRY: prometheus JAVA_OPTS: "-Xms2048m -Xmx2048m" skywalking_ui: image: uhub.service.ucloud.cn/marksugar-k8s/skywalking-ui:9.6.0 #apache/skywalking-ui:9.6 container_name: skywalking_ui depends_on: - skywalking_oap ports: - "8080:8080" environment: SW_OAP_ADDRESS: https://skywalking_oap:12800 SW_ZIPKIN_ADDRESS: https://skywalking_oap:9412 skywalking_elasticsearch: # image: registry.cn-hangzhou.aliyuncs.com/marksugar/elasticsearch:6.8.6 image: uhub.service.ucloud.cn/marksugar-k8s/elasticsearch:7.17.13 container_name: skywalking_elasticsearch ulimits: memlock: soft: -1 hard: -1 #network_mode: host hostname: elasticsearch restart: always environment: - cluster.name="elasticsearch" # - network.host="0.0.0.0" - discovery.type=single-node # - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms2048m -Xmx4096m -XX:-UseConcMarkSweepGC -XX:-UseCMSInitiatingOccupancyOnly -XX:+UseG1GC -XX:InitiatingHeapOccupancyPercent=75 -Duser.timezone=Asia/Shanghai" user: root ports: - 9200:9200 - 9300:9300 # docker-compose 3.x # healthcheck: # test: [ "CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1" ] # interval: 30s # timeout: 10s # retries: 3 # start_period: 10s volumes: - /etc/localtime:/etc/localtime:ro - /data/skywalking_elasticsearch/data:/usr/share/elasticsearch/data # mkdir -p /data/elasticsearch/data #chown -R 1000.1001 /data/elasticsearch/data logging: driver: "json-file" options: max-size: "50M" mem_limit: 6144m skywalking_kabana: # image: uhub.service.ucloud.cn/marksugar-k8s/kibana:6.8.6 image: uhub.service.ucloud.cn/marksugar-k8s/kibana:7.17.13 container_name: skywalking_kabana ulimits: memlock: soft: -1 hard: -1 #network_mode: host hostname: kibana restart: always environment: - ELASTICSEARCH_URL=https://skywalking_elasticsearch:9200 user: root ports: - 5601:5601 volumes: - /etc/localtime:/etc/localtime:ro logging: driver: "json-file" options: max-size: "50M" mem_limit: 2048mlogback要使用它，我们修改pom.xml，添加两个apm的标签，如下 <dependency> <groupId>org.apache.skywalking</groupId> <artifactId>apm-toolkit-logback-1.x</artifactId> <version>{project.release.version}</version> </dependency> <dependency> <groupId>org.apache.skywalking</groupId> <artifactId>apm-toolkit-trace</artifactId> <version>{project.release.version}</version> </dependency>{project.release.version}填写当前skywalking agent的版本，我这里使用的是9.0.0，参考apm-toolkit-logback-1.x查看最新版本而后我们需要创建一个文件ruoyi-admin/src/main/resources/logback-spring.xml，并添加如下注意：1，这里使用的是ch.qos.logback.core.ConsoleAppender，那么可能在k8s中，日志打印或许就不能使用STDOUT收集日志了，STDOUT只能传递给skywalking2，skywalking已经收集了日志和链路，那为什么还要在收集日志呢。这取决于ES存储维护性考虑。你至少需要将日志收集到另外一个日志管理系统。skywalking的日志和链路用于近期查看。而日志收集用作与日志搜索。综上所述，如果不能将日志打印到STDOUT的同时还发送到skywalking，那么在K8S中就需要写入到容器内。这可不是一个好主意。<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.TraceIdPatternLogbackLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%tid] [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </encoder> </appender> <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{tid}] [%thread] %-5level %logger{36} -%msg%n</Pattern> </layout> </encoder> </appender>而后我们修改ruoyi-admin中的logback.xml，片段如下 <!-- 日志输出格式 --> <property name="APM_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{tid}] [%thread] %-5level %logger{36} -%msg%n" /> <!-- skyWalking日志采集 --> <appender name="APM_LOG" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 控制台输出 --> <appender name="console" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> 其他如下<?xml version="1.0" encoding="UTF-8"?> <configuration scan="true" scanPeriod="5 seconds"> <!-- 日志输出格式 --> <property name="APM_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{tid}] [%thread] %-5level %logger{36} -%msg%n" /> <!-- skyWalking日志采集 --> <appender name="APM_LOG" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 控制台输出 --> <appender name="console" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 系统日志输出 --> <appender name="file_info" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <appender name="file_error" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 用户访问日志输出 --> <appender name="sys-user" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 系统模块日志级别控制 --> <logger name="com.ruoyi" level="info" /> <!-- Spring日志级别控制 --> <logger name="org.springframework" level="warn" /> <root level="info"> <appender-ref ref="console" /> </root> <!--系统操作日志--> <root level="info"> <appender-ref ref="file_info" /> <appender-ref ref="file_error" /> </root> <!--系统用户操作日志--> <logger name="sys-user" level="info"> <appender-ref ref="sys-user"/> </logger> </configuration> 如上所言，我们还需要一种方式提供日志系统的收集，因此同时将日志写入到本地文件。片段如下： <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> 完整xml如下<?xml version="1.0" encoding="UTF-8"?> <configuration scan="true" scanPeriod="5 seconds"> <!-- 日志输出格式 --> <property name="APM_PATTERN" value="%d{yyyy-MM-dd HH:mm:ss.SSS} [%X{tid}] [%thread] %-5level %logger{36} -%msg%n" /> <!-- skyWalking日志采集 --> <appender name="APM_LOG" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 控制台输出 --> <appender name="console" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 系统日志输出 --> <appender name="file_info" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <appender name="file_error" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 用户访问日志输出 --> <appender name="sys-user" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender"> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 写入文件 --> <!-- 日志存放路径 --> <property name="log.path" value="/data/ruoyi/logs" /> <!-- 控制台输出 --> <appender name="console_file" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${log.path}/sys-console.log</file> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 系统日志输出 --> <appender name="file_info_file" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${log.path}/sys-info.log</file> <!-- 循环政策：基于时间创建日志文件 --> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- 日志文件名格式 --> <fileNamePattern>${log.path}/sys-info.%d{yyyy-MM-dd}.log</fileNamePattern> <!-- 日志最大的历史 60天 --> <maxHistory>1</maxHistory> </rollingPolicy> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> <filter class="ch.qos.logback.classic.filter.LevelFilter"> <!-- 过滤的级别 --> <level>INFO</level> <!-- 匹配时的操作：接收（记录） --> <onMatch>ACCEPT</onMatch> <!-- 不匹配时的操作：拒绝（不记录） --> <onMismatch>DENY</onMismatch> </filter> </appender> <appender name="file_error_file" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${log.path}/sys-err.log</file> <!-- 循环政策：基于时间创建日志文件 --> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- 日志文件名格式 --> <fileNamePattern>${log.path}/sys-err.%d{yyyy-MM-dd}.log</fileNamePattern> <!-- 日志最大的历史 60天 --> <maxHistory>1</maxHistory> </rollingPolicy> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> <filter class="ch.qos.logback.classic.filter.LevelFilter"> <!-- 过滤的级别 --> <level>ERROR</level> <!-- 匹配时的操作：接收（记录） --> <onMatch>ACCEPT</onMatch> <!-- 不匹配时的操作：拒绝（不记录） --> <onMismatch>DENY</onMismatch> </filter> </appender> <!-- 用户访问日志输出 --> <appender name="sys-user_file" class="ch.qos.logback.core.rolling.RollingFileAppender"> <file>${log.path}/sys-user.log</file> <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy"> <!-- 按天回滚 daily --> <fileNamePattern>${log.path}/sys-user.%d{yyyy-MM-dd}.log</fileNamePattern> <!-- 日志最大的历史 60天 --> <maxHistory>1</maxHistory> </rollingPolicy> <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder"> <layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout"> <Pattern>${APM_PATTERN}</Pattern> </layout> </encoder> </appender> <!-- 系统模块日志级别控制 --> <logger name="com.ruoyi" level="info" /> <!-- Spring日志级别控制 --> <logger name="org.springframework" level="warn" /> <root level="info"> <appender-ref ref="console" /> </root> <!--系统操作日志--> <root level="info"> <appender-ref ref="file_info" /> <appender-ref ref="file_error" /> </root> <!--系统用户操作日志--> <logger name="sys-user" level="info"> <appender-ref ref="sys-user"/> </logger> <root level="info"> <appender-ref ref="console_file" /> <appender-ref ref="file_info_file" /> <appender-ref ref="file_error_file" /> <appender-ref ref="sys-user_file"/> </root> </configuration> 现在日志会发送到skywalking，本地也会保留一份，并且，我们可以通过日志的TID和skywalking中的id关联进行查询。接着开始打包mvn clean package -Dmaven.test.skip=true构建镜像构建完成后，我们定制一个Dockerfile来构建一个容器的镜像FROM registry.cn-hangzhou.aliyuncs.com/marksugar/jdk:8u202 MAINTAINER by mark ENV JAVA_OPTS="\ -server \ -Xms2048m \ -Xmx2048m \ -Xmn512m \ -Xss256k \ -XX:+UseConcMarkSweepGC \ -XX:+UseCMSInitiatingOccupancyOnly \ -XX:CMSInitiatingOccupancyFraction=70 \ -XX:+HeapDumpOnOutOfMemoryError \ -XX:HeapDumpPath=/data/ruoyi/logs" \ MY_USER=linuxea \ MY_USER_ID=316 COPY ruoyi-admin/target/*.jar /data/ COPY skywalking-agent /data/ruoyi # copy文件到/data下后需要chonw普通用户权限 RUN addgroup -g ${MY_USER_ID} -S ${MY_USER} \ && adduser -u ${MY_USER_ID} -S -H -s /sbin/nologin -g 'java' -G ${MY_USER} ${MY_USER} \ && mkdir /data/ruoyi/logs -p \ && echo "Asia/Shanghai" > /etc/timezone \ && chown -R linuxea.linuxea /data/ruoyi WORKDIR /data # 使用linuxea运行 java程序 USER linuxea CMD java ${JAVA_OPTS} -javaagent:/data/ruoyi/skywalking-agent.jar -jar *.jar进行构建docker build -t uhub.service.ucloud.cn/marksugar-k8s/ruoyi-admin:v1 .在k8s中，我们需要传递如下 环境变量 - name: SW_AGENT_NAME value: mark::test1 - name: SW_AGENT_COLLECTOR_BACKEND_SERVICES value: skywalking-oap.skywalking:11800 docker 使用-e即可docker run --rm -e SW_AGENT_NAME=linuxea:ruoyi-admin -e SW_AGENT_COLLECTOR_BACKEND_SERVICES=172.16.100.151:11800 --net=host uhub.service.ucloud.cn/marksugar-k8s/ruoyi-admin:v1接着，我们进入当前创建的容器中在日志里面随意拿取一个TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001[root@Node-172_16_100_151 /data/ruoyi/logs]# docker exec -it distracted_leakey bash bash-5.1$ cd /data/ruoyi/logs/ bash-5.1$ tail -f sys-user.log 2023-09-21 22:14:17.281 [TID:e5bebb3ccecc4f0d88512263c809653b.77.16953056571440001] [http-nio-8080-exec-6] DEBUG c.r.s.m.S.selectUserList_COUNT -<== Total: 1 2023-09-21 22:14:17.283 [TID:e5bebb3ccecc4f0d88512263c809653b.77.16953056571440001] [http-nio-8080-exec-6] DEBUG c.r.s.m.SysUserMapper.selectUserList -==> Preparing: select u.user_id, u.dept_id, u.nick_name, u.user_name, u.email, u.avatar, u.phonenumber, u.sex, u.status, u.del_flag, u.login_ip, u.login_date, u.create_by, u.create_time, u.remark, d.dept_name, d.leader from sys_user u left join sys_dept d on u.dept_id = d.dept_id where u.del_flag = '0' LIMIT ? 2023-09-21 22:14:17.285 [TID:e5bebb3ccecc4f0d88512263c809653b.77.16953056571440001] [http-nio-8080-exec-6] DEBUG c.r.s.m.SysUserMapper.selectUserList -==> Parameters: 10(Integer) 2023-09-21 22:14:17.290 [TID:e5bebb3ccecc4f0d88512263c809653b.77.16953056571440001] [http-nio-8080-exec-6] DEBUG c.r.s.m.SysUserMapper.selectUserList -<== Total: 7 2023-09-21 22:14:18.367 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.S.selectUserList_COUNT -==> Preparing: SELECT count(0) FROM sys_user u LEFT JOIN sys_dept d ON u.dept_id = d.dept_id WHERE u.del_flag = '0' AND (u.dept_id = ? OR u.dept_id IN (SELECT t.dept_id FROM sys_dept t WHERE find_in_set(?, ancestors))) 2023-09-21 22:14:18.371 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.S.selectUserList_COUNT -==> Parameters: 103(Long), 103(Long) 2023-09-21 22:14:18.373 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.S.selectUserList_COUNT -<== Total: 1 2023-09-21 22:14:18.373 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.SysUserMapper.selectUserList -==> Preparing: select u.user_id, u.dept_id, u.nick_name, u.user_name, u.email, u.avatar, u.phonenumber, u.sex, u.status, u.del_flag, u.login_ip, u.login_date, u.create_by, u.create_time, u.remark, d.dept_name, d.leader from sys_user u left join sys_dept d on u.dept_id = d.dept_id where u.del_flag = '0' AND (u.dept_id = ? OR u.dept_id IN ( SELECT t.dept_id FROM sys_dept t WHERE find_in_set(?, ancestors) )) LIMIT ? 2023-09-21 22:14:18.374 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.SysUserMapper.selectUserList -==> Parameters: 103(Long), 103(Long), 10(Integer) 2023-09-21 22:14:18.377 [TID:e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001] [http-nio-8080-exec-9] DEBUG c.r.s.m.SysUserMapper.selectUserList -<== Total: 1回到skywalking查询traceID为 e5bebb3ccecc4f0d88512263c809653b.80.16953056583460001而后点击查看日志点击标记就可以看到日志信息了我特意将UI界面点开。简洁性不言而喻参考：https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/application-toolkit-logback-1.x/#logstash-logback-pluginhttps://www.apache.org/dyn/closer.cgi/skywalking/java-agent/9.0.0/apache-skywalking-java-agent-9.0.0.tgzhttps://skywalking.apache.org/downloads/https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/application-toolkit-log4j-1.x/https://skywalking.apache.org/docs/skywalking-java/next/en/setup/service-agent/java-agent/application-toolkit-logback-1.x/

linuxea: tekton ci Sidecar(3) 相比较此前在k8s上运行一个Docker Outside of Docker, DooD的方式，由于这个镜像一直运行，因此缓存存在在容器内，可以加速构建和复用已经拉取到资源。而Sidecar就不需要提前部署，Tekton 会将 Sidecar 注入属于 TaskRun 的 Pod，一旦 Task 中的所有 Steps 完成执行，Pod 内运行的每一个 Sidecar 就会终止掉，如果 Sidecar 成功退出，kubectl get pods 命令会将 Pod 的状态返回为 Completed，如果 Sidecar 退出时出现了错误，则返回 Error，而忽略实际执行 Pod 内部 Steps 的容器镜像的退出码值。在每个容器里面都是一个独立的 Docker Daemon，因此也支持并行运行，互不影响，DinD显然更加安全、更加干净。我门仍然需要准备相关的配置1.凭据和SAapiVersion: v1 kind: Secret metadata: name: ucloud-auth annotations: tekton.dev/docker-0: https://uhub.service.ucloud.cn type: kubernetes.io/basic-auth stringData: username: username password: password --- apiVersion: v1 kind: ServiceAccount metadata: name: build-sa secrets: - name: ucloud-auth --- apiVersion: tekton.dev/v1alpha1 kind: PipelineResource metadata: name: git-res namespace: default spec: params: - name: url value: https://gitee.com/marksugar/argocd-example - name: revision value: master type: git --- apiVersion: tekton.dev/v1alpha1 kind: PipelineResource metadata: name: ucloud-image-go spec: type: image params: - name: url value: uhub.service.ucloud.cn/linuxea/golang #构建完的镜像名称 接着配置一个task的测试用例2.测试taskapiVersion: tekton.dev/v1beta1 kind: Task metadata: name: test spec: resources: inputs: - name: repo type: git steps: - name: run-test image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/golang:1.18.10-alpine3.17 workingDir: /workspace/repo script: | #!/usr/bin/env sh cd tekton/go && go test3.构建task# task-docker-build.yaml apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: docker-build-push spec: resources: inputs: # 定义输入资源 - name: source # 源代码仓库 type: git params: - name: image description: Reference of the image docker will produce. - name: builder_image description: The location of the docker builder image. default: docker:stable - name: dockerfile description: Path to the Dockerfile to build. default: ./Dockerfile - name: context description: Path to the directory to use as context. default: . - name: build_extra_args description: Extra parameters passed for the build command when building images. default: "" - name: push_extra_args description: Extra parameters passed for the push command when pushing images. default: "" - name: insecure_registry description: Allows the user to push to an insecure registry that has been specified default: "" - name: registry_mirror description: Specific the docker registry mirror default: "" - name: registry_url description: private docker images registry url steps: - name: docker-build # 构建步骤 image: $(params.builder_image) env: - name: DOCKER_HOST # 用 TLS 形式通过 TCP 链接 sidecar value: tcp://localhost:2376 - name: DOCKER_TLS_VERIFY # 校验 TLS value: "1" - name: DOCKER_CERT_PATH # 使用 sidecar 守护进程生成的证书 value: /certs/client workingDir: $(resources.inputs.source.path) script: | # docker 构建命令 docker login $(params.registry_url) cd tekton/go docker build \ $(params.build_extra_args) \ --no-cache \ -f $(params.dockerfile) -t $(params.image) $(params.context) volumeMounts: # 声明挂载证书目录 - mountPath: /certs/client name: dind-certs - name: docker-push # image: $(params.builder_image) env: - name: DOCKER_HOST value: tcp://localhost:2376 - name: DOCKER_TLS_VERIFY value: "1" - name: DOCKER_CERT_PATH value: /certs/client workingDir: $(resources.inputs.source.path) script: | # 推送 docker 镜像 docker login $(params.registry_url) echo $(params.image) docker push $(params.push_extra_args) $(params.image) volumeMounts: - mountPath: /certs/client name: dind-certs sidecars: # sidecar 模式，提供 docker daemon服务，实现真正的 DinD 模式 - image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/docker:20.10.16-dind name: server args: - --storage-driver=vfs - --userland-proxy=false - --debug - --insecure-registry=$(params.insecure_registry) - --registry-mirror=$(params.registry_mirror) securityContext: privileged: true env: - name: DOCKER_TLS_CERTDIR # 将生成的证书写入与客户端共享的路径 value: /certs volumeMounts: - mountPath: /certs/client name: dind-certs readinessProbe: # 等待 dind daemon 生成它与客户端共享的证书 periodSeconds: 1 exec: command: ["ls", "/certs/client/ca.pem"] volumes: # 使用 emptyDir 的形式即可 - name: dind-certs emptyDir: {}上面的 Task 使用了一个 registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/docker:20.10.16-dind 镜像来提供 docker 服务端，在 sidecar 模式容器是共享 network namespace 的，通过 tcp://localhost:2376 和 docker 服务端进行通信，由于还使用的是 TLS 证书模式，所以需要将证书目录进行声明挂载。在dockerfile的细节部分需要进入dockerfile目录进行构建4.pipeline我门需要指定params参数最终给每个task传递使用apiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: test-sidecar-pipeline spec: resources: # 为 Tasks 提供输入和输出资源声明 - name: git-res type: git params: - name: image type: string - name: image-tag type: string default: "v0.4.0" - name: registry_url type: string default: "uhub.service.ucloud.cn" - name: registry_mirror type: string default: "https://ot2k4d59.mirror.aliyuncs.com/" - name: insecure_registry type: string default: "uhub.service.ucloud.cn" tasks: # 添加task到流水线中 # 运行应用测试 - name: test taskRef: name: test resources: inputs: - name: repo # Task 输入名称 resource: git-res # Pipeline 资源名称 - name: get-build-id taskRef: name: generate-build-id params: - name: base-version value: $(params.image-tag) # 构建并推送 Docker 镜像 - name: build-and-push taskRef: name: docker-build-push # 使用上面定义的镜像构建任务 runAfter: - test # 测试任务执行之后 resources: inputs: - name: source # 指定输入的git仓库资源 resource: git-res params: - name: image value: "$(params.image):$(tasks.get-build-id.results.build-id)" - name: registry_url value: $(params.registry_url) - name: insecure_registry value: $(params.insecure_registry) - name: registry_mirror value: $(params.registry_mirror)5.pipelinerun这些参数其中包括tag就会通过pipelinerun来进行传递，或者使用默认值apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: test-sidecar-pipelinerun spec: serviceAccountName: build-sa pipelineRef: name: test-sidecar-pipeline resources: - name: git-res # 指定输入的git仓库资源 resourceRef: name: git-res params: - name: image value: uhub.service.ucloud.cn/linuxea/golang - name: image-tag # 传入版本号 value: "v0.4.1"apply[root@master1 Sidecar]# kubectl apply -f ./ task.tekton.dev/generate-build-id created task.tekton.dev/docker-build-push created task.tekton.dev/test created pipeline.tekton.dev/test-sidecar-pipeline created pipelinerun.tekton.dev/test-sidecar-pipelinerun created我门仍然通过命令查看构建pipelinerun[root@master1 Sidecar]# tkn pipelinerun describe test-sidecar-pipelinerun Name: test-sidecar-pipelinerun Namespace: default Pipeline Ref: test-sidecar-pipeline Service Account: build-sa Labels: tekton.dev/pipeline=test-sidecar-pipeline Status STARTED DURATION STATUS 12 seconds ago --- Running Timeouts Pipeline: 1h0m0s Params NAME VALUE ∙ image uhub.service.ucloud.cn/linuxea/golang ∙ image-tag v0.4.1查看taskrun[root@master1 Sidecar]# tkn taskrun list NAME STARTED DURATION STATUS test-sidecar-pipelinerun-build-and-push 4 minutes ago 2m1s Succeeded test-sidecar-pipelinerun-get-build-id 4 minutes ago 12s Succeeded test-sidecar-pipelinerun-test 4 minutes ago 10s Succeeded test-pipelinerun-build-and-push-test 22 hours ago 11s Succeeded test-pipelinerun-get-build-id 22 hours ago 9s Succeeded test-pipelinerun-test 22 hours ago 9s Succeeded build-and-push 1 day ago 12m13s Succeeded testrun 4 days ago 2m45s Succeeded查看taskrun日志[root@master1 Sidecar]# tkn taskrun logs test-sidecar-pipelinerun-get-build-id [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/main/x86_64/APKINDEX.tar.gz [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/community/x86_64/APKINDEX.tar.gz [get-timestamp] (1/1) Installing tzdata (2023c-r1) [get-timestamp] OK: 11 MiB in 19 packages [get-timestamp] Current Timestamp: 20230613-151739 [get-timestamp] 20230613-151739 [get-buildid] v0.4.1-20230613-151739 [root@master1 Sidecar]# tkn taskrun logs test-sidecar-pipelinerun-test [git-source-repo-lgfhp] {"level":"info","ts":1686640658.3913343,"caller":"git/git.go:178","msg":"Successfully cloned https://gitee.com/marksugar/argocd-example @ d270fd8931fb059485622edb2ef1aa1209b7d42c (grafted, HEAD, origin/master) in path /workspace/repo"} [git-source-repo-lgfhp] {"level":"info","ts":1686640658.4037006,"caller":"git/git.go:217","msg":"Successfully initialized and updated submodules in path /workspace/repo"} [run-test] PASS [run-test] ok test 0.002s[root@master1 Sidecar]# tkn taskrun logs test-sidecar-pipelinerun-build-and-push -f [git-source-source-vx28b] {"level":"info","ts":1686640670.9329953,"caller":"git/git.go:178","msg":"Successfully cloned https://gitee.com/marksugar/argocd-example @ d270fd8931fb059485622edb2ef1aa1209b7d42c (grafted, HEAD, origin/master) in path /workspace/source"} [git-source-source-vx28b] {"level":"info","ts":1686640670.948619,"caller":"git/git.go:217","msg":"Successfully initialized and updated submodules in path /workspace/source"} [docker-build] Authenticating with existing credentials... [docker-build] WARNING! Your password will be stored unencrypted in /root/.docker/config.json. [docker-build] Configure a credential helper to remove this warning. See [docker-build] https://docs.docker.com/engine/reference/commandline/login/#credentials-store [docker-build] [docker-build] Login Succeeded [docker-build] Sending build context to Docker daemon 5.12kB [docker-build] Step 1/5 : FROM registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/golang:1.18.10-alpine3.17 [docker-build] 1.18.10-alpine3.17: Pulling from marksugar-k8s/golang [docker-build] 8921db27df28: Pulling fs layer [docker-build] a2f8637abd91: Pulling fs layer [docker-build] 4ba80a8cd2c7: Pulling fs layer [docker-build] dbc2308a4587: Pulling fs layer [docker-build] dbc2308a4587: Waiting [docker-build] a2f8637abd91: Verifying Checksum [docker-build] a2f8637abd91: Download complete [docker-build] dbc2308a4587: Verifying Checksum [docker-build] dbc2308a4587: Download complete [docker-build] 8921db27df28: Download complete [docker-build] 8921db27df28: Pull complete [docker-build] a2f8637abd91: Pull complete [docker-build] 4ba80a8cd2c7: Verifying Checksum [docker-build] 4ba80a8cd2c7: Download complete [docker-build] 4ba80a8cd2c7: Pull complete [docker-build] dbc2308a4587: Pull complete [docker-build] Digest: sha256:ab5685692564e027aa84e2980855775b2e48f8fc82c1590c0e1e8cbc2e716542 [docker-build] Status: Downloaded newer image for registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/golang:1.18.10-alpine3.17 [docker-build] ---> a77f45e5f987 [docker-build] Step 2/5 : RUN mkdir /test -p [docker-build] ---> Running in 7924d227b939 [docker-build] Removing intermediate container 7924d227b939 [docker-build] ---> 12e24160d708 [docker-build] Step 3/5 : WORKDIR /test [docker-build] ---> Running in 14f7ab09a508 [docker-build] Removing intermediate container 14f7ab09a508 [docker-build] ---> 0faa68db85f7 [docker-build] Step 4/5 : COPY . . [docker-build] ---> 9411b3cf535a [docker-build] Step 5/5 : CMD ["go test"] [docker-build] ---> Running in 8356b0c1c4ba [docker-build] Removing intermediate container 8356b0c1c4ba [docker-build] ---> 921c8de056ef [docker-build] Successfully built 921c8de056ef [docker-build] Successfully tagged uhub.service.ucloud.cn/linuxea/golang:v0.4.1-20230613-151739 [docker-push] Authenticating with existing credentials... [docker-push] WARNING! Your password will be stored unencrypted in /root/.docker/config.json. [docker-push] Configure a credential helper to remove this warning. See [docker-push] https://docs.docker.com/engine/reference/commandline/login/#credentials-store [docker-push] [docker-push] Login Succeeded [docker-push] uhub.service.ucloud.cn/linuxea/golang:v0.4.1-20230613-151739 [docker-push] The push refers to repository [uhub.service.ucloud.cn/linuxea/golang] [docker-push] a13db5615e44: Preparing [docker-push] 55da38d80f35: Preparing [docker-push] 4ad1c2ef216c: Preparing [docker-push] c23db623ee98: Preparing [docker-push] c1bfd5512d71: Preparing [docker-push] 8e012198eea1: Preparing [docker-push] 8e012198eea1: Waiting [docker-push] c23db623ee98: Layer already exists [docker-push] 4ad1c2ef216c: Layer already exists [docker-push] c1bfd5512d71: Layer already exists [docker-push] 8e012198eea1: Layer already exists [docker-push] 55da38d80f35: Pushed [docker-push] a13db5615e44: Pushed [docker-push] v0.4.1-20230613-151739: digest: sha256:a6ad200509fe8776b5ae2aaaf2ddf8387b0af30ae0667bfb67883bffefe7a962 size: 1571 [sidecar-server] 2023/06/13 07:19:44 Exiting...回到界面查看镜像仓库此时已经push了最新的

linuxea: 使用tekton构建镜像(2) 现在我门将这些组合起来形成一个流水线，使用的 CRD 对象是 Pipeline 。假如是一个新的环境，我门仍然需要配置此前的凭据，比如git认证，镜像仓库认证，必要的配置和版本号的传递等首先配置一个使用的凭据在清单中1.配置凭据和SAapiVersion: v1 kind: Secret metadata: name: ucloud-auth annotations: tekton.dev/docker-0: https://uhub.service.ucloud.cn type: kubernetes.io/basic-auth stringData: username: username password: password --- apiVersion: v1 kind: ServiceAccount metadata: name: build-sa secrets: - name: ucloud-auth --- apiVersion: tekton.dev/v1alpha1 kind: PipelineResource metadata: name: git-res namespace: default spec: params: - name: url value: https://gitee.com/marksugar/argocd-example - name: revision value: master type: git --- apiVersion: tekton.dev/v1alpha1 kind: PipelineResource metadata: name: ucloud-image-go spec: type: image params: - name: url value: uhub.service.ucloud.cn/linuxea/golang #构建完的镜像名称 PipelineResource的ucloud-image-go中并没有写入版本号，这是因为在task会进行生成2.配置task1task1作为一个假定的测试apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: test spec: resources: inputs: - name: repo type: git steps: - name: run-test image: registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/golang:1.18.10-alpine3.17 workingDir: /workspace/repo script: | #!/usr/bin/env sh cd tekton/go && go test3.生成版本号我门需要提供一个版本号，使用task生成一个由环境变量的传入-年月日时间的版本号apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: generate-build-id spec: description: >- Given a base version, this task generates a unique build id by appending the base-version to the current timestamp. params: - name: base-version description: Base product version type: string default: "1.0" results: - name: timestamp description: Current timestamp - name: build-id description: ID of the current build steps: - name: get-timestamp image: uhub.service.ucloud.cn/marksugar-k8s/bash:5.0.18 script: | #!/usr/bin/env bash sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/g' /etc/apk/repositories apk add -U tzdata cp /usr/share/zoneinfo/Asia/Shanghai /etc/localtime ts=`date "+%Y%m%d-%H%M%S"` echo "Current Timestamp: ${ts}" echo ${ts} | tr -d "\n" | tee $(results.timestamp.path) - name: get-buildid image: uhub.service.ucloud.cn/marksugar-k8s/bash:5.0.18 script: | #!/usr/bin/env bash ts=`cat $(results.timestamp.path)` buildId=$(inputs.params.base-version)-${ts} echo ${buildId} | tr -d "\n" | tee $(results.build-id.path)3.配置镜像构建我门需要修改构建时候的命令，以便于添加传入的版本号# task-build-push.yaml apiVersion: tekton.dev/v1beta1 kind: Task metadata: name: build-and-push-test spec: resources: inputs: # 定义输入资源 - name: repo #输入资源，就是gitee的那个仓库 type: git outputs: # 定义输出资源 - name: linuxea # 输出镜像名字 type: image params: # 定义参数 - name: DockerfileURL #指明 dockerfile 在仓库中的哪个位置 type: string default: $(resources.inputs.repo.path)/tekton/go/Dockerfile # repo资源的路径 description: The path to the dockerfile to build - name: pathToContext #指明构建上下文的路径 type: string default: $(resources.inputs.repo.path) # repo资源的路径 description: the build context used by docker daemon - name: imageTag type: string default: "v0.2.0" description: the docker image tag steps: - name: build-and-push image: docker:stable script: | #!/usr/bin/env sh docker login uhub.service.ucloud.cn cd /workspace/repo/tekton/go docker build -t $(resources.outputs.linuxea.url):$(params.imageTag) . docker push $(resources.outputs.linuxea.url):$(params.imageTag) # 这边的参数都是在 input 和 output 中定义的 env: - name: DOCKER_HOST value: tcp://docker-dind.tekton-pipelines:23754.配置pipeline管道中，我门需要注意每个task运行的顺序是由runAfter来决定的，在build-and-push-test阶段，传入了环境变量imageTagapiVersion: tekton.dev/v1beta1 kind: Pipeline metadata: name: test-pipeline spec: resources: # 为 Tasks 提供输入和输出资源声明 - name: git-res type: git - name: ucloud-image-go type: image params: - name: image-tag type: string tasks: # 添加task到流水线中 - name: get-build-id taskRef: name: generate-build-id # 引入generate-build-id task params: - name: base-version value: $(params.image-tag) # 运行应用测试 - name: test taskRef: name: test resources: inputs: - name: repo # Task 输入名称 resource: git-res # Pipeline 资源名称 # 构建并推送 Docker 镜像 - name: build-and-push-test taskRef: name: build-and-push-test runAfter: - test # 测试任务执行之后 - get-build-id resources: inputs: - name: repo # 指定输入的git仓库资源 resource: git-res outputs: # 指定输出的镜像资源 - name: linuxea resource: ucloud-image-go params: - name: imageTag value: "$(tasks.get-build-id.results.build-id)"5.pipelinerun接着在spec中引入build-sa的sa，和各个资源和管道apiVersion: tekton.dev/v1beta1 kind: PipelineRun metadata: name: test-pipelinerun spec: serviceAccountName: build-sa # 关联带有认证信息的 ServiceAccount pipelineRef: name: test-pipeline resources: - name: git-res # 指定输入的git仓库资源 resourceRef: name: git-res - name: ucloud-image-go # 指定输出的镜像资源 resourceRef: name: ucloud-image-go params: - name: image-tag # 传入版本号 value: "v0.3.0"注意，这里的image-tag的变量是传递给管道中名称为 get-build-id的task, 而这个task指向的是generate-build-id这个实际的task定义，而generate-build-id这个task在接收到image-tag的参数后返回了一个 buildId的环境变量，这个变量被获取后赋值给imageTag，而在构建镜像之前这个变量已经获取完成并且被传入到build-and-push-test的构建步骤中。6.构建查看创建完成后，通过命令查看[root@master1 pipeline]# tkn pipelinerun describe test-pipelinerun Name: test-pipelinerun Namespace: default Pipeline Ref: test-pipeline Service Account: build-sa Labels: tekton.dev/pipeline=test-pipeline Status STARTED DURATION STATUS 6 seconds ago --- Running ⏱ Timeouts Pipeline: 1h0m0s查看当前的taskrun[root@master1 pipeline]# tkn taskrun list NAME STARTED DURATION STATUS test-pipelinerun-build-and-push-test 15 minutes ago 11s Succeeded test-pipelinerun-get-build-id 15 minutes ago 9s Succeeded test-pipelinerun-test 15 minutes ago 9s Succeeded build-and-push 7 hours ago 12m13s Succeeded testrun 3 days ago 2m45s Succeeded根据name查看日志[root@master1 pipeline]# tkn taskrun logs test-pipelinerun-test [git-source-repo-gknf2] {"level":"info","ts":1686559991.0322871,"caller":"git/git.go:178","msg":"Successfully cloned https://gitee.com/marksugar/argocd-example @ d270fd8931fb059485622edb2ef1aa1209b7d42c (grafted, HEAD, origin/master) in path /workspace/repo"} [git-source-repo-gknf2] {"level":"info","ts":1686559991.0433397,"caller":"git/git.go:217","msg":"Successfully initialized and updated submodules in path /workspace/repo"} [run-test] PASS [run-test] ok test 0.002s [root@master1 pipeline]# tkn taskrun logs test-pipelinerun-get-build-id [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/main/x86_64/APKINDEX.tar.gz [get-timestamp] fetch https://mirrors.ustc.edu.cn/alpine/v3.18/community/x86_64/APKINDEX.tar.gz [get-timestamp] (1/1) Installing tzdata (2023c-r1) [get-timestamp] OK: 11 MiB in 19 packages [get-timestamp] Current Timestamp: 20230612-165311 [get-timestamp] 20230612-165311 [get-buildid] v0.3.0-20230612-165311 [root@master1 pipeline]# tkn taskrun logs test-pipelinerun-build-and-push-test [create-dir-linuxea-q6w6n] 2023/06/12 08:53:16 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/home/nonroot": unable to create destination directory: mkdir /home/nonroot: permission denied [git-source-repo-w5dcj] {"level":"info","ts":1686559998.4194562,"caller":"git/git.go:178","msg":"Successfully cloned https://gitee.com/marksugar/argocd-example @ d270fd8931fb059485622edb2ef1aa1209b7d42c (grafted, HEAD, origin/master) in path /workspace/repo"} [git-source-repo-w5dcj] {"level":"info","ts":1686559998.4337826,"caller":"git/git.go:217","msg":"Successfully initialized and updated submodules in path /workspace/repo"} [build-and-push] Authenticating with existing credentials... [build-and-push] WARNING! Your password will be stored unencrypted in /root/.docker/config.json. [build-and-push] Configure a credential helper to remove this warning. See [build-and-push] https://docs.docker.com/engine/reference/commandline/login/#credentials-store [build-and-push] [build-and-push] Login Succeeded [build-and-push] Sending build context to Docker daemon 5.12kB [build-and-push] Step 1/5 : FROM registry.cn-zhangjiakou.aliyuncs.com/marksugar-k8s/golang:1.18.10-alpine3.17 [build-and-push] ---> a77f45e5f987 [build-and-push] Step 2/5 : RUN mkdir /test -p [build-and-push] ---> Using cache [build-and-push] ---> 48d724b29eff [build-and-push] Step 3/5 : WORKDIR /test [build-and-push] ---> Using cache [build-and-push] ---> 9b6eda13d6c1 [build-and-push] Step 4/5 : COPY . . [build-and-push] ---> Using cache [build-and-push] ---> a5c71d579512 [build-and-push] Step 5/5 : CMD ["go test"] [build-and-push] ---> Using cache [build-and-push] ---> 2f377c99476e [build-and-push] Successfully built 2f377c99476e [build-and-push] Successfully tagged uhub.service.ucloud.cn/linuxea/golang:v0.3.0-20230612-165311 [build-and-push] The push refers to repository [uhub.service.ucloud.cn/linuxea/golang] [build-and-push] 6a7300559c98: Preparing [build-and-push] 17600296503b: Preparing [build-and-push] 4ad1c2ef216c: Preparing [build-and-push] c23db623ee98: Preparing [build-and-push] c1bfd5512d71: Preparing [build-and-push] 8e012198eea1: Preparing [build-and-push] 8e012198eea1: Waiting [build-and-push] 17600296503b: Layer already exists [build-and-push] 6a7300559c98: Layer already exists [build-and-push] 4ad1c2ef216c: Layer already exists [build-and-push] c1bfd5512d71: Layer already exists [build-and-push] c23db623ee98: Layer already exists [build-and-push] 8e012198eea1: Layer already exists [build-and-push] v0.3.0-20230612-165311: digest: sha256:b649a5f42e753e36562b574e26018301e8e839f129f92e3f87fd29f4b30734fc size: 1572 [image-digest-exporter-htxn8] {"severity":"INFO","timestamp":"2023-06-12T08:53:22.548887491Z","caller":"logging/config.go:116","message":"Successfully created the logger."} [image-digest-exporter-htxn8] {"severity":"INFO","timestamp":"2023-06-12T08:53:22.548992849Z","caller":"logging/config.go:117","message":"Logging level set to: info"} [image-digest-exporter-htxn8] {"severity":"INFO","timestamp":"2023-06-12T08:53:22.549269268Z","caller":"imagedigestexporter/main.go:59","message":"No index.json found for: linuxea","commit":"68f2a66"}此时pod构建结束[root@master1 build]# kubectl get pods -w NAME READY STATUS RESTARTS AGE build-and-push-pod 0/4 Completed 0 7h26m test-pipelinerun-build-and-push-test-pod 0/4 Completed 0 8m18s test-pipelinerun-get-build-id-pod 0/2 Completed 0 8m33s test-pipelinerun-test-pod 0/2 Completed 0 8m33s testrun-pod 0/2 Completed 0 3d7h test-pipelinerun-get-build-id-pod 0/2 Terminating 0 8m37s test-pipelinerun-test-pod 0/2 Terminating 0 8m37s test-pipelinerun-build-and-push-test-pod 0/4 Terminating 0 8m22s test-pipelinerun-get-build-id-pod 0/2 Terminating 0 8m37s test-pipelinerun-test-pod 0/2 Terminating 0 8m37s test-pipelinerun-build-and-push-test-pod 0/4 Terminating 0 8m22s test-pipelinerun-get-build-id-pod 0/2 Pending 0 0s test-pipelinerun-test-pod 0/2 Pending 0 0s test-pipelinerun-get-build-id-pod 0/2 Pending 0 0s test-pipelinerun-test-pod 0/2 Pending 0 0s test-pipelinerun-get-build-id-pod 0/2 Init:0/3 0 0s test-pipelinerun-test-pod 0/2 Init:0/4 0 0s test-pipelinerun-test-pod 0/2 Init:1/4 0 1s test-pipelinerun-get-build-id-pod 0/2 Init:1/3 0 1s test-pipelinerun-test-pod 0/2 Init:2/4 0 2s test-pipelinerun-get-build-id-pod 0/2 Init:2/3 0 2s test-pipelinerun-test-pod 0/2 Init:3/4 0 3s test-pipelinerun-get-build-id-pod 0/2 PodInitializing 0 3s test-pipelinerun-get-build-id-pod 2/2 Running 0 4s test-pipelinerun-test-pod 0/2 PodInitializing 0 4s test-pipelinerun-get-build-id-pod 2/2 Running 0 4s test-pipelinerun-test-pod 2/2 Running 0 5s test-pipelinerun-test-pod 2/2 Running 0 5s test-pipelinerun-test-pod 1/2 NotReady 0 8s test-pipelinerun-get-build-id-pod 1/2 NotReady 0 8s test-pipelinerun-test-pod 0/2 Completed 0 9s test-pipelinerun-get-build-id-pod 0/2 Completed 0 9s test-pipelinerun-build-and-push-test-pod 0/4 Pending 0 0s test-pipelinerun-build-and-push-test-pod 0/4 Pending 0 0s test-pipelinerun-build-and-push-test-pod 0/4 Init:0/3 0 0s test-pipelinerun-build-and-push-test-pod 0/4 Init:1/3 0 0s test-pipelinerun-build-and-push-test-pod 0/4 Init:2/3 0 2s test-pipelinerun-build-and-push-test-pod 0/4 PodInitializing 0 3s test-pipelinerun-build-and-push-test-pod 4/4 Running 0 4s test-pipelinerun-build-and-push-test-pod 4/4 Running 0 4s test-pipelinerun-build-and-push-test-pod 3/4 NotReady 0 5s test-pipelinerun-build-and-push-test-pod 2/4 NotReady 0 6s test-pipelinerun-build-and-push-test-pod 1/4 NotReady 0 9s test-pipelinerun-build-and-push-test-pod 0/4 Completed 0 10s任务成功[root@master1 pipeline]# tkn pipelinerun describe test-pipelinerun Name: test-pipelinerun Namespace: default Pipeline Ref: test-pipeline Service Account: build-sa Labels: tekton.dev/pipeline=test-pipeline STARTED DURATION STATUS 1 minute ago 20s Succeeded Timeouts Pipeline: 1h0m0s Params NAME VALUE ∙ image-tag v0.3.0镜像也推送完成在tekton中看到详细的细节，其中test的task1做了go test，build-and-push进行了镜像打包和推送。