linuxea:gitlab-ci/cd Gitlab和Gitlab Prometheus监控(十二) gitlab和Prometheus监控前面一篇中，有提到过使用Hygieia，Hygieia更善于集中式的展示更多的CI套件的状态。倘若我所有的CI都是自动化(特别是使用gitlab-ci)，那我们可能只需要健康容器的运行状态了，如果是jenkins那可能有些不同，这取决你使用的方式。gitlab的套包中的Prometheus自带了gitlab套件的所有的exporter，基本上开箱即用。并且在gitlab中已经完成了部分的，在之前章节的Hygieia所能暂时的部分功能(仅仅是gitlab本身)本篇简单的叙述下，gitlab和gitlab Prometheus的监控，其中涉及到Grafana，Grafana在后面会逐渐提到的更多Hygieia参考：Hygieia dashboard简单配置(十一) 简述：如果你装的是gitlab官网的包的话，Prometheus将会捆绑在软件包中，我们只需要打开它，并且重新gitlab-ctl reconfigur即可打开来进行使用，它和Prometheus单独安装所差不多，但是还是建议另外安装，当然，如过另外安装你可能需要安装其他的几个exporter几个exporter：https://github.com/prometheus/prometheus/wiki/Default-port-allocationsPrometheus监控修改配置文件[root@linuxea-VM-Node146 ~]# cat /etc/gitlab/gitlab.rb主要修改这两项prometheus_monitoring['enable'] = true 这里改成0.0.0.0比较妥当prometheus['listen_address'] = '0.0.0.0:9090'而后gitlab-ctl reconfigur，并且restart[root@linuxea-VM-Node146 ~]# gitlab-ctl reconfigure [root@linuxea-VM-Node146 ~]# gitlab-ctl restart它的配置文件在/var/opt/gitlab/prometheus下，可以在Status界面中看到配置项[root@linuxea-VM-Node146 ~]# cat /var/opt/gitlab/prometheus/prometheus.yml 在打开之前，放行端口，而后就通过IP：PORT打开Prometheus[root@linuxea-VM-Node146 ~]# iptables -I INPUT 5 -p tcp --dport 9090 -j ACCEPT官网提供的集中查询示例：％使用的内存： (1 - ((node_memory_MemFree + node_memory_Cached) / node_memory_MemTotal)) * 100％CPU负载： 1 - rate(node_cpu{mode="idle"}[5m])传输的数据： irate(node_network_transmit_bytes[5m])收到的数据： irate(node_network_receive_bytes[5m])还提供了,postgres_exporter,redis_exporter,以及gitlab的gitlab-monitorGrafana+Prometheusso，我们安装一个grafana来展示Prometheus的信息它呈现的效果大概是这样的(Grafana安装和配置实在是太简单了，就不叙述了)我已上传json文件到github上，upload即可json地址：https://raw.githubusercontent.com/LinuxEA-Mark/jenkins_gitlab_Docker/master/gitlab_monitor_linuxea_com.jsonGitlab监控另外可以通过gitlab项目中的自带的监控来看，例如，项目中的流水线CI/CD Charts，如下：也可以查看项目的提交信息，如下：以及commits信息，如下：参考：https://docs.gitlab.com/ee/administration/monitoring/prometheus/ https://gitlab.com/gitlab-org/gitlab-monitor https://docs.gitlab.com/ee/administration/monitoring/prometheus/gitlab_metrics.html#metrics-shared-directory

linuxea:gitlab-ci/cd Hygieia dashboard简单配置(十一) Hygieia他的发音hi-gee-ya，在希腊语和罗马神话中她是健康的女神/化身在Hygieia中分为五层，UI,API,DEVOPS TOOLS，数据收集，数据存储，其中：UI层(用户界面) : 是Hygieia的前端，包含用户可以查看的所有图形用户界面（GUI）元素。用户也可以在此处配置仪表板。API层 : Hygieia API层包含Hygieia API和Audit API。Hygieia API包含与源系统数据（由服务任务收集）和Internet一起使用的所有典型REST API服务。Hygieia审计API是API端点的集合，用于审计Hygieia收集器收集的CI / CD数据。该层是本地数据层和源系统数据层的抽象。DEVOPS TOOLS层 ：该层需要CI / CD管道中的大量DevOps工具。在图中，Jira，Git，Sonar和XLDeploy作为示例列出。数据收集层：收集器层从您的DevOps工具中获取数据。反过来，这些数据会显示在您的Hygieia仪表板上。您可以选择从Hygieia收集器清单中安装适用于您的DevOps工具集的收集器。数据存储层 ：Hygieia使用MongoDB作为数据存储和检索的数据库架构概述如下图：Hygieia可以描述CI/CD管道，从本质上讲Hygieia就是一个聚合器，它从团队的CI/CD管道中使用各种devops工具提取数据，使其在仪表板中更易于理解,坦白的说，便是将交付流水全过程反馈到可视化的界面中来Hygieia的仪表板简化了近乎实时查看CICD管道的能力。仪表板使DevOps工程师和管理人员能够监控代码提交到最终生产中的部署情况。在这两点之间开始（提交）到完成（prod） - 仪表板还提供有关软件操作的整体活力和性能指标的重要信息。在其中包括：组合视图，流水线视图，产品试图，云环境视图等，阅读参考：https://capitalone.github.io/Hygieia/getting_started.html本章节是对gitlab和Hygieia，相比较jenkins，还是后者更全面简单做了安装和配置实践，需要大量的开发套件才能完成，因此与gitlab-ci关系并不大，gitlab本身已经有视图，并不是如此好用而已，本篇仅提供思路，不具备任何参考价值Hygieia与gitlab安装mvn和nodeinstall mvncd /usr/local && curl -Lk https://mirror.rise.ph/apache/maven/maven-3/3.5.4/binaries/apache-maven-3.5.4-bin.tar.gz|tar xz -C ./ && ln -s apache-maven-3.5.4 apache-maven && echo "export PATH=/usr/local/apache-maven/bin:\$PATH" >> /etc/profile && source /etc/profile && echo -e "\033[32m`mvn -version` \033[0m"install nodecd /usr/local && curl -Lk https://nodejs.org/dist/v8.11.3/node-v8.11.3-linux-x64.tar.xz|tar xJ -C ./ && ln -s node-v8.11.3-linux-x64 node && echo "export PATH=/usr/local/node/bin:\$PATH" >> /etc/profile && source /etc/profile && echo -e "\033[32m`node -v` \033[0m"Hygieia install创建一个普通用户并且切换到普通用户，克隆代码并执行mvn clean install package[root@LinuxEA-VM-Node202 ~]# useradd Hygieia [root@LinuxEA-VM-Node202 ~]# su - Hygieia [Hygieia@LinuxEA-VM-Node202 ~]$ git clone https://github.com/capitalone/Hygieia.git 正克隆到 'Hygieia'... remote: Counting objects: 34942, done. remote: Compressing objects: 100% (74/74), done. remote: Total 34942 (delta 8), reused 44 (delta 3), pack-reused 34849 接收对象中: 100% (34942/34942), 71.17 MiB | 231.00 KiB/s, done. 处理 delta 中: 100% (15509/15509), done.执行mvn clean install package，这个过程可能会很长[Hygieia@LinuxEA-VM-Node202 ~]$ cd Hygieia [Hygieia@LinuxEA-VM-Node202 ~/Hygieia]$ mvn clean install package [INFO] Scanning for projects... Downloading from central: https://repo.maven.apache.org/maven2/org/ 省略一万字 [INFO] Reactor Summary: [INFO] [INFO] com.capitalone.dashboard:Hygieia 2.0.5-SNAPSHOT .... SUCCESS [ 49.924 s] [INFO] com.capitalone.dashboard:core ...................... SUCCESS [03:03 min] [INFO] com.capitalone.dashboard:api ....................... SUCCESS [ 45.683 s] [INFO] com.capitalone.dashboard:api-audit ................. SUCCESS [ 13.169 s] [INFO] com.capitalone.dashboard:rally-collector ........... SUCCESS [ 6.604 s] [INFO] com.capitalone.dashboard:artifactory-artifact-collector SUCCESS [ 4.012 s] [INFO] com.capitalone.dashboard:bamboo-build-collector .... SUCCESS [ 3.544 s] [INFO] com.capitalone.dashboard:jenkins-build-collector ... SUCCESS [ 3.855 s] [INFO] com.capitalone.dashboard:jenkins-cucumber-test-collector SUCCESS [ 2.881 s] [INFO] com.capitalone.dashboard:jenkins-codequality ....... SUCCESS [ 7.999 s] [INFO] com.capitalone.dashboard:sonar-codequality-collector SUCCESS [ 4.993 s] [INFO] com.capitalone.dashboard:aws-cloud-collector ....... SUCCESS [ 36.819 s] [INFO] com.capitalone.dashboard:udeploy-deployment-collector SUCCESS [ 2.942 s] [INFO] com.capitalone.dashboard:xldeploy-deployment-collector SUCCESS [ 2.776 s] [INFO] com.capitalone.dashboard:jira-feature-collector .... SUCCESS [ 58.628 s] [INFO] com.capitalone.dashboard:versionone-feature-collector SUCCESS [ 11.287 s] [INFO] com.capitalone.dashboard:gitlab-feature-collector .. SUCCESS [ 3.761 s] [INFO] com.capitalone.dashboard:chat-ops-collector ........ SUCCESS [ 0.929 s] [INFO] com.capitalone.dashboard:appdynamics-performance-collector SUCCESS [ 2.725 s] [INFO] com.capitalone.dashboard:bitbucket-scm-collector ... SUCCESS [ 3.123 s] [INFO] com.capitalone.dashboard:github-scm-collector ...... SUCCESS [ 2.988 s] [INFO] com.capitalone.dashboard:github-graphql-scm-collector SUCCESS [ 4.384 s] [INFO] com.capitalone.dashboard:subversion-collector ...... SUCCESS [ 21.260 s] [INFO] com.capitalone.dashboard:gitlab-scm-collector ...... SUCCESS [ 3.083 s] [INFO] com.capitalone.dashboard:hpsm-cmdb-collector ....... SUCCESS [ 3.074 s] [INFO] com.capitalone.dashboard:nexus-iq-collector ........ SUCCESS [ 3.144 s] [INFO] com.capitalone.dashboard:score-collector ........... SUCCESS [ 4.487 s] [INFO] Hygieia Publisher Plugin ........................... SUCCESS [08:25 min] [INFO] com.capitalone.dashboard:UI ........................ SUCCESS [02:40 min] [INFO] com.capitalone.dashboard:ui-tests 2.0.5-SNAPSHOT ... SUCCESS [01:44 min] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 25:09 min [INFO] Finished at: 2018-07-11T09:59:48+08:00 [INFO] ------------------------------------------------------------------------生成认证密钥生成一个core的key[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/core/target]$ java -jar core-2.0.5-SNAPSHOT.jar com.capitalone.dashboard.util.Encryption SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder". SLF4J: Defaulting to no-operation (NOP) logger implementation SLF4J: See https://www.slf4j.org/codes.html#StaticLoggerBinder for further details. Your secret key is: xIleAiYTyCy1McKXZz2dL6s4+FtiaML+ Sample encrypted string with the above key for 'thisIsMyPassword' is: 0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/k [Hygieia@LinuxEA-VM-Node202 ~/Hygieia/core/target]$ gitlab tokens在gitlab上生产一个 Access Tokens点击create pesonal access token将生成的Access Token保存：NsDKbGL2P9NYXFQbtMpPmongodb授权mongodb安装配置参考：https://www.linuxea.com/1848.html创建admin账户> db.createUser({"user":"admin","pwd":"admin","roles":["root"]}) Successfully added user: { "user" : "admin", "roles" : [ "root" ] }到admin库创建用户和库的授权信息> use admin switched to db admin > db.auth("admin","admin") 1开始创建linuxeacom> db.createUser({user: "linuxeacom",pwd: "123456",roles: [{role: "readWrite", db: "linuxeacom"}]}) Successfully added user: { "user" : "linuxeacom", "roles" : [ { "role" : "readWrite", "db" : "linuxeacom" } ] } > 进入linuxeacom授权linuxeacom用户> use linuxeacom switched to db linuxeacom > db.createUser({user: "linuxeacom",pwd: "123456",roles: [{role: "readWrite", db: "linuxeacom"}]}) Successfully added user: { "user" : "linuxeacom", "roles" : [ { "role" : "readWrite", "db" : "linuxeacom" } ] } > 验证> use linuxeacom switched to db linuxeacom > db.auth("linuxeacom","123456") 1 > 配置文件api部分将key加入到api配置中[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/api/target]$ cat dashboard.properties dbname=linuxeacom dbhost=10.10.240.203 dbusername=linuxeacom dbpassword=123456 dbhostport=10.10.240.203:27017 dbport=27017 dbreplicaset=false server.contextPath=/api auth.secret = hygsecret server.port=8080 key=0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/kgitlab配置文件gitlab生成的Access Token填写进去，key也填写进去[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/collectors/feature/gitlab/target]$ cat gitlab-application.properties # Database Name dbname=linuxeacom # Database HostName - default is localhost dbhost=10.10.240.203 # Database Port - default is 27017 dbport=27017 # MongoDB replicaset #dbreplicaset=[false if you are not using MongoDB replicaset] dbreplicaset=false dbhostport=10.10.240.203:27017,127.0.0.1:27017 # Database Username - default is blank dbusername=linuxeacom # Database Password - default is blank dbpassword=123456 # Logging File location logging.file=./logs/gitlab.log #Collector schedule (required) gitlab.cron=0 0/5 * * * * #Gitlab host (optional, defaults to 'gitlab.com') gitlab.host=10.10.240.146 #Gitlab protocol (optional, defaults to 'http') gitlab.protocol=http #Gitlab port (optional, defaults to protocol default port) gitlab.port=80 #Gitlab path (optional, if your instance of gitlab requires a path) #gitlab.path= gitlab.apiToken=NsDKbGL2P9NYXFQbtMpP gitlab.commitThresholdDays=20 gitlab.key=0kY/r1UAMiedT2XQCWFQwhJzAWMGiA/k gitlab.apiVersion=4UI切换到root用户下，安装gulp[root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]# npm install -g gulp npm WARN deprecated gulp-util@3.0.8: gulp-util is deprecated - replace it, following the guidelines at https://medium.com/gulpjs/gulp-util-ca3b1f9f9ac5 npm WARN deprecated graceful-fs@3.0.11: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js npm WARN deprecated minimatch@2.0.10: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated minimatch@0.2.14: Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue npm WARN deprecated graceful-fs@1.2.3: please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js npm WARN notice [SECURITY] minimatch has the following vulnerability: 1 high. Go here for more details: https://nodesecurity.io/advisories?search=minimatch&version=0.2.14 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. npm WARN notice [SECURITY] minimatch has the following vulnerability: 1 high. Go here for more details: https://nodesecurity.io/advisories?search=minimatch&version=2.0.10 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. npm WARN notice [SECURITY] lodash has the following vulnerability: 1 low. Go here for more details: https://nodesecurity.io/advisories?search=lodash&version=1.0.2 - Run `npm i npm@latest -g` to upgrade your npm version, and then `npm audit` to get more info. /usr/local/node-v8.11.3-linux-x64/bin/gulp -> /usr/local/node-v8.11.3-linux-x64/lib/node_modules/gulp/bin/gulp.jsgulp@3.9.1added 253 packages in 7.074s[root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]#### 启动 在启动api之前，加密属性的配置[Hygieia@LinuxEA-VM-Node202 ~]$ java -cp ~/.m2/repository/org/jasypt/jasypt/1.9.2/jasypt-1.9.2.jar org.jasypt.intf.cli.JasyptPBEStringEncryptionCLI input="dbpassword" password=hygieiasecret algorithm=PBEWithMD5AndDES----ENVIRONMENT-----------------Runtime: Oracle Corporation Java HotSpot(TM) 64-Bit Server VM 25.171-b11 ----ARGUMENTS-------------------algorithm: PBEWithMD5AndDESinput: dbpasswordpassword: hygieiasecret----OUTPUT----------------------6f/7fe3Mky72zulhSUu6PyujTNtHdQmy[Hygieia@LinuxEA-VM-Node202 ~]$启动api[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/api/target]$ exit 登出 [root@LinuxEA-VM-Node202 ~]# cd /home/Hygieia/Hygieia/api/target/ [root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/api/target]# [root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/api/target]# java -jar api.jar --spring.config.location=./dashboard.properties -Djasypt.encryptor.password=hygieiasecret启动scm插件gitlab[Hygieia@LinuxEA-VM-Node202 ~/Hygieia/collectors/scm/gitlab/target]$ java -jar gitlab-scm-collector-2.0.5-SNAPSHOT.jar --spring.config.name=gitlab --spring.config.location=./gitlab-application.properties启动UI[root@LinuxEA-VM-Node202 /home/Hygieia/Hygieia/UI]# gulp serve [11:07:05] Using gulpfile /home/Hygieia/Hygieia/UI/gulpfile.js [11:07:05] Starting 'build'... [11:07:05] Starting 'clean'... [11:07:05] Finished 'clean' after 57 ms [11:07:05] Starting 'assets'... [11:07:05] Starting 'themes'... [11:07:05] Starting 'fonts'... [11:07:05] Starting 'js'... [11:07:05] Starting 'views'... [11:07:05] Starting 'test-data'... [11:07:13] Finished 'themes' after 8.05 s [11:07:13] Finished 'assets' after 8.05 s [11:07:13] Finished 'test-data' after 8.1 s [11:07:13] Finished 'views' after 8.23 s [11:07:13] Finished 'js' after 8.36 s [11:07:14] Finished 'fonts' after 9.29 s [11:07:14] Starting 'html'... [11:07:14] gulp-inject 1 files into index.html. [11:07:14] gulp-inject 155 files into index.html. [11:07:14] Finished 'html' after 219 ms [11:07:14] Finished 'build' after 9.59 s [11:07:14] Starting 'serve'... [11:07:14] Finished 'serve' after 162 ms [BS] Local URL: https://localhost:3000 [BS] External URL: https://10.10.240.202:3000 [BS] Serving files from: dist/添加防火墙放行[root@LinuxEA-VM-Node202 ~]# iptables -I INPUT 6 -p tcp --dport 3000 -j ACCEPT界面配置点击右上角的Login，在弹出来的界面上点击Sign UP，注册一个即可创建一个项目 create a new dashboard，在widget management中选择添加我这里测试repo填写完成后保存选中一个查看sonar与Hygieia启动一个sonarqube快速的run一个sonarqube[root@DS-VM-Node_10_10_240_145 ~]$ docker run -d --name sonarqube --net=host -e SONARQUBE_JDBC_USERNAME=linuxeacom -e SONARQUBE_JDBC_PASSWORD=123 -e SONARQUBE_JDBC_URL=jdbc:postgresql://10.10.240.202/linuxeacom sonarqube:6.7.4通过IP和端口能够打开并且使用sonar-scanner检测一次[gitlab-runner@DS-VM-Node_10_10_240_145 ~/builds/d7f8c868/0/Hygieia_user/linuxea_app]$ docker run -v $(pwd):/root/src -v /var/run/docker.sock:/var/run/docker.sock "newtmitch/sonar-scanner" sonar-scanner -Dsonar.host.url=https://10.10.240.145:9000 -Dsonar.projectKey=linuxea_app -Dsonar.projectName=linuxea_app -Dsonar.projectBaseDir=/root/src -Dsonar.sources=./ -Dsonar.java.binaries=. 配置sonar api切换到$PATH/Hygieia/collectors/build/sonar/target/目录下创建配置文件配置文件详情请关注备注[root@DS-VM-Node202 ~]# cd /home/Hygieia/Hygieia/collectors/build/sonar/target/ [root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# cat sonar-application.properties # Database Name dbname=linuxeacom # Database HostName - default is localhost dbhost=10.10.240.203 # Database Port - default is 27017 dbport=27017 # MongoDB replicaset dbreplicaset=false dbhostport=[10.10.240.203:27017] # Database Username - default is blank dbusername=linuxeacom # Database Password - default is blank dbpassword=123456 # Collector schedule (required) sonar.cron=0 0/5 * * * * # Sonar server(s) (required) - Can provide multiple sonar.servers[0]=https://10.10.240.145:9000 # Sonar version, match array index to the server. If not set, will default to version prior to 6.3. sonar.versions[0]=6.7 # Sonar Metrics - Required. #Sonar versions lesser than 6.3 sonar.metrics[0]=ncloc,line_coverage,violations,critical_violations,major_violations,blocker_violations,violations_density,sqale_index,test_success_density,test_failures,test_errors,tests # For Sonar version 6.3 and above sonar.metrics[0]=ncloc,violations,new_vulnerabilities,critical_violations,major_violations,blocker_violations,tests,test_success_density,test_errors,test_failures,coverage,line_coverage,sqale_index,alert_status,quality_gate_details # Sonar login credentials sonar.username=admin sonar.password=admin [root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# 而后就在当前目录启动[root@DS-VM-Node202 /home/Hygieia/Hygieia/collectors/build/sonar/target]# java -jar sonar-codequality-collector-2.0.5-SNAPSHOT.jar --spring.config.name=sonar --spring.config.location=./sonar-application.properties 你可能会看到这样的日志回到UI添加添加一个code analysis ,在这个下拉菜单中必须存在sonar中的项目，否则则是失败的，如果失败请检查日志保存

linuxea:gitlab-ci/cd htpps配置和gitlab-runner配置(十) gitlab https配置自己签名一个证书给gitlab使用事实上我并不推荐使用https自签证书，如果还使用了runner那你肯定会明白的根证书自签开始根证书私钥文件创建[root@Linuxea-VM-Node146 /etc/gitlab/trusted-certs]# cd /etc/pki/CA/ [root@Linuxea-VM-Node146 /etc/pki/CA]#(umask 077; openssl genrsa -out /etc/pki/CA/private/cakey.pem 2048) Generating RSA private key, 2048 bit long modulus ....+++ ..................................................................+++ e is 65537 (0x10001)生成CA自己的私钥文件，cakey.pem。默认权限为600；[root@Linuxea-VM-Node146 /etc/pki/CA]# openssl req -new -x509 -key /etc/pki/CA/private/cakey.pem -out /etc/pki/CA/cacert.pem -days 36 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:cn #国家 State or Province Name (full name) []:shanghai # 身份 Locality Name (eg, city) [Default City]:shanghai # 城市 Organization Name (eg, company) [Default Company Ltd]:ca #机构i Organizational Unit Name (eg, section) []:ops # 部门名称 Common Name (eg, your name or your server's hostname) []:linuxea-gitlab.ds.com # 主机名称 Email Address []:user@163.com # 邮箱根据私钥文件生成CA自签证书。CA给自己签发一个证书；来关注下以下几个文件index.txt 创建索引文件serial 证书编号位置echo 01 > serial 自定义开始证书编号touch crlnumber 已吊销证书编号文件[root@Linuxea-VM-Node146 /etc/pki/CA]# touch index.txt serial [root@Linuxea-VM-Node146 /etc/pki/CA]# echo 01 > serial [root@Linuxea-VM-Node146 /etc/pki/CA]# touch crlnumber 以上的三个文件，是CA配置文件中定义的文件名，必须手动创建出来；以上简单自建CA配置完毕；签发gitlab证书到gitlab目录在创建一个证书，而后用上面的根证书来进行签发[root@Linuxea-VM-Node146 /etc/pki/CA]# mkdir /etc/gitlab/ssl [root@Linuxea-VM-Node146 /etc/pki/CA]# cd /etc/gitlab/ssl [root@Linuxea-VM-Node146 /etc/gitlab/ssl]# (umask 077; openssl genrsa -out linuxea-gitlab.ds.com.key 1024) Generating RSA private key, 1024 bit long modulus .......................++++++ ....................++++++ e is 65537 (0x10001)[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# ll 总用量 4 -rw------- 1 root root 891 7月 9 13:10 linuxea-gitlab.ds.com.key创建申请[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl req -new -key ./linuxea-gitlab.ds.com.key -out ./linuxea-gitlab.ds.com.csr -days 365 You are about to be asked to enter information that will be incorporated into your certificate request. What you are about to enter is what is called a Distinguished Name or a DN. There are quite a few fields but you can leave some blank For some fields there will be a default value, If you enter '.', the field will be left blank. ----- Country Name (2 letter code) [XX]:cn State or Province Name (full name) []:shanghai Locality Name (eg, city) [Default City]:shanghai Organization Name (eg, company) [Default Company Ltd]:gitlab-linuxea Organizational Unit Name (eg, section) []:linuxea Common Name (eg, your name or your server's hostname) []:linuxea-gitlab.ds.com Email Address []:user@163.com Please enter the following 'extra' attributes to be sent with your certificate request A challenge password []: An optional company name []:CA服务器上操作开始签发[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl ca -in linuxea-gitlab.ds.com.csr -out ./linuxea-gitlab.ds.com.crt -days 365 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok The organizationName field needed to be the same in the CA certificate (ca) and the request (gitlab-linuxea)解决报错，修改配置文件[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# vim /etc/pki/tls/openssl.cnf 84 [ policy_match ] 85 countryName|| = optional 86 stateOrProvinceName|= optional 87 organizationName| = optional 88 organizationalUnitName| = optional 89 commonName| | = supplied 90 emailAddress| | = optional在颁发一次[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# openssl ca -in linuxea-gitlab.ds.com.csr -out ./linuxea-gitlab.ds.com.crt -days 365 Using configuration from /etc/pki/tls/openssl.cnf Check that the request matches the signature Signature ok Certificate Details: Serial Number: 1 (0x1) Validity Not Before: Jul 9 05:19:07 2018 GMT Not After : Jul 9 05:19:07 2019 GMT Subject: countryName = cn stateOrProvinceName = shanghai organizationName = gitlab-linuxea organizationalUnitName = linuxea commonName = linuxea-gitlab.ds.com emailAddress = user@163.com X509v3 extensions: X509v3 Basic Constraints: CA:FALSE Netscape Comment: OpenSSL Generated Certificate X509v3 Subject Key Identifier: DE:FF:FD:A3:6E:96:9A:F8:D4:9A:1E:F2:FE:1B:99:FB:A3:A3:07:4F X509v3 Authority Key Identifier: keyid:00:8A:F1:19:5D:42:7E:CD:14:10:56:62:6C:C1:D6:00:36:6C:B0:29 Certificate is to be certified until Jul 9 05:19:07 2019 GMT (365 days) Sign the certificate? [y/n]:y out of 1 certificate requests certified, commit? [y/n]yWrite out database with 1 new entriesData Base Updated查看[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# cat /etc/pki/CA/serial02[root@Linuxea-VM-Node146 /etc/gitlab/ssl]# ll /etc/pki/CA/newcerts/01.pem -rw-r--r-- 1 root root 3917 7月 9 13:19 /etc/pki/CA/newcerts/01.pem修改gitlab文件[root@Linuxea-VM-Node146 ~]# vim /etc/gitlab/gitlab.rbexternal_url 'https://linuxea-gitlab.ds.com'nginx['redirect_http_to_https']=truenginx['ssl_certificate'] = "/etc/pki/CA/newcerts/01.pem"nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/linuxea-gitlab.ds.com.key"[root@Linuxea-VM-Node146 ~]# gitlab-ctl reconfigure![20180709-6.png][1] gitlab https已经配置完成 ### gitlab runner配置 gitlab一旦用来https,这可不当紧，但是runner也就需要了，不然会报错[root@Linuxea-VM-Node_10_10_240_145 ~]$ gitlab-runner verifyRunning in system-mode. ERROR: Verifying runner... failed runner=awSshzhi status=couldn't execute POST against https://linuxea-gitlab.ds.com/api/v4/runners/verify: Post https://linuxea-gitlab.ds.com/api/v4/runners/verify: x509: certificate signed by unknown authorityERROR: Verifying runner... failed runner=fc4a03c9 status=couldn't execute POST against https://linuxea-gitlab.ds.com/api/v4/runners/verify: Post https://linuxea-gitlab.ds.com/api/v4/runners/verify: x509: certificate signed by unknown authority`--tls-ca-file`来解决[root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner]$ gitlab-runner register --tls-ca-file ~/01.pem Running in system-mode.Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):https://linuxea-gitlab.ds.com/Please enter the gitlab-ci token for this runner:awSshzhiuXp1KbxmXbk3Please enter the gitlab-ci description for this runner:Please enter the gitlab-ci tags for this runner (comma separated): 145-runner Whether to run untagged builds [true/false]:Whether to lock the Runner to current project [true/false]:Registering runner... succeeded runner=awSshzhiPlease enter the executor: docker, parallels, shell, docker-ssh+machine, kubernetes, docker-ssh, ssh, virtualbox, docker+machine:shellRunner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded!### 正确的方式 这样操作完成后,可能还是不行的,将之前的文件都拿到runner机器上来 创建目录后，使用`tls_cert_path=/etc/gitlab-runner/certs`指定即可[root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$ ll总用量 12-rw-r--r-- 1 root root 3917 7月 26 22:14 01.pem-rw-r--r-- 1 root root 3917 7月 26 22:04 linuxea-gitlab.ds.com.crt-rw------- 1 root root 891 7月 26 22:13 linuxea-gitlab.ds.com.key[root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$ gitlab-runner register tls_cert_path=/etc/gitlab-runner/certsRunning in system-mode.Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):https://linuxea-gitlab.ds.com/Please enter the gitlab-ci token for this runner:awSshzhiuXp1KbxmXbk3Please enter the gitlab-ci description for this runner:Please enter the gitlab-ci tags for this runner (comma separated):testWhether to run untagged builds [true/false]:Whether to lock the Runner to current project [true/false]:Registering runner... succeeded runner=awSshzhiPlease enter the executor: docker+machine, kubernetes, ssh, virtualbox, parallels, shell, docker-ssh+machine, docker, docker-ssh:shellRunner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! [root@Linuxea-VM-Node_10_10_240_145 /etc/gitlab-runner/certs]$参考:https://docs.gitlab.com/runner/configuration/advanced-configuration.html

linuxea:gitlab-ci/cd Gmial邮件通知简单配置(九) configure email如果希望通过SMTP服务器而不是通过Sendmail发送应用程序电子邮件，添加以下配置信息 /etc/gitlab/gitlab.rb并运行gitlab-ctl reconfigure本地主机上的SMTP这种简单地启用SMTP并使用默认设置的配置可用于在本地主机上运行的MTA，该主机不提供sendmail接口或提供sendmail与GitLab不兼容的接口，如Exim。gitlab_rails['smtp_enable'] = true没有SSL的SMTP默认情况下，SSL为SMTP启用。如果SMTP服务器不支持通过SSL进行通信，使用以下设置：gitlab_rails['smtp_enable'] = true; gitlab_rails['smtp_address'] = 'localhost'; gitlab_rails['smtp_port'] = 25; gitlab_rails['smtp_domain'] = 'localhost'; gitlab_rails['smtp_tls'] = false; gitlab_rails['smtp_openssl_verify_mode'] = 'none' gitlab_rails['smtp_enable_starttls_auto'] = false gitlab_rails['smtp_ssl'] = false gitlab_rails['smtp_force_ssl'] = falseGmailgitlab_rails['smtp_enable'] = true gitlab_rails['smtp_address'] = "smtp.gmail.com" gitlab_rails['smtp_port'] = 587 gitlab_rails['smtp_user_name'] = "my.email@gmail.com" gitlab_rails['smtp_password'] = "my-gmail-password" gitlab_rails['smtp_domain'] = "smtp.gmail.com" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = false gitlab_rails['smtp_openssl_verify_mode'] = 'peer'访问权限打开要用的话先启用较低应用的访问权限config change而后修改配置文件[root@LinuxEA-VM-Node146 ~]# cat /etc/gitlab/gitlab.rb |egrep "^[^#]*gitlab_rails\['smtp_" gitlab_rails['smtp_enable'] = true gitlab_rails['smtp_address'] = "smtp.gmail.com" gitlab_rails['smtp_port'] = 587 gitlab_rails['smtp_user_name'] = "userlinuxea@gmail.com" gitlab_rails['smtp_password'] = "password.gzc.ooo" gitlab_rails['smtp_domain'] = "smtp.gmail.com" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = false gitlab_rails['smtp_openssl_verify_mode'] = 'peer'重启[root@LinuxEA-VM-Node146 /etc/gitlab]# gitlab-ctl restart ok: run: alertmanager: (pid 6301) 0s ok: run: gitaly: (pid 6320) 1s ok: run: gitlab-monitor: (pid 6333) 0s ok: run: gitlab-workhorse: (pid 6349) 1s ok: run: logrotate: (pid 6365) 0s ok: run: nginx: (pid 6373) 0s ok: run: node-exporter: (pid 6381) 1s ok: run: postgres-exporter: (pid 6390) 0s ok: run: postgresql: (pid 6400) 1s ok: run: prometheus: (pid 6484) 0s ok: run: redis: (pid 6496) 1s ok: run: redis-exporter: (pid 6500) 0s ok: run: sidekiq: (pid 6510) 0s ok: run: unicorn: (pid 6522) 0sTest Email发送测试邮件Notify.test_email('userlinuxea@gmail.com', 'Message Subject', 'Message Body And Linuxea.com').deliver_now[root@LinuxEA-VM-Node146 /etc/gitlab]# gitlab-rails console ------------------------------------------------------------------------------------- Gitlab: 10.8.4 (2268d0c) Gitlab Shell: 7.1.2 postgresql: 9.6.8 ------------------------------------------------------------------------------------- Loading production environment (Rails 4.2.10) irb(main):001:0> irb(main):004:0> Notify.test_email('userlinuxea@gmail.com', 'Message Subject', 'Message Body And Linuxea.com').deliver_now Notify#test_email: processed outbound mail in 1.1ms Sent mail to userlinuxea@gmail.com (3516.5ms) Date: Tue, 03 Jul 2018 14:02:28 +0800 From: GitLab <gitlab@10.10.240.146> Reply-To: GitLab <noreply@10.10.240.146> To: usertzc@gmail.com Message-ID: <5b3b1174199c3_19b23f9c467db1184948@LinuxEA-VM-Node146.cluster.com.mail> Subject: Message Subject Mime-Version: 1.0 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit Auto-Submitted: auto-generated X-Auto-Response-Suppress: All <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "https://www.w3.org/TR/REC-html40/loose.dtd"> <html><body><p>Message Body</p></body></html> => #<Mail::Message:69940213947220, Multipart: false, Headers: <Date: Tue, 03 Jul 2018 14:02:28 +0800>, <From: GitLab <gitlab@10.10.240.146>>, <Reply-To: GitLab <noreply@10.10.240.146>>, <To: userlinuxea@gmail.com>, <Message-ID: <5b3b1174199c3_19b23f9c467db1184948@LinuxEA-VM-Node146.cluster.com.mail>>, <Subject: Message Subject>, <Mime-Version: 1.0>, <Content-Type: text/html; charset=UTF-8>, <Content-Transfer-Encoding: 7bit>, <Auto-Submitted: auto-generated>, <X-Auto-Response-Suppress: All>>登陆查看pipeline email设置Admin area(小板手) ---> service templates---> pipelines emails而后添加邮件在运行一次运行完成，就能收到邮件腾讯企业邮箱（QQ exmail）gitlab_rails['smtp_enable'] = true gitlab_rails['smtp_address'] = "smtp.exmail.qq.com" gitlab_rails['smtp_port'] = 465 gitlab_rails['smtp_user_name'] = "xxxx@xx.com" gitlab_rails['smtp_password'] = "password" gitlab_rails['smtp_authentication'] = "login" gitlab_rails['smtp_enable_starttls_auto'] = true gitlab_rails['smtp_tls'] = true gitlab_rails['gitlab_email_from'] = 'xxxx@xx.com' gitlab_rails['smtp_domain'] = "exmail.qq.com"参考：https://docs.gitlab.com/omnibus/settings/smtp.html#more-examples-are-welcome

linuxea:gitlab-ci/cd sitespeed.io WEB性能测试(八) sitespeed.ioSitespeed.io是一组开源工具，可以轻松监控和衡量您网站的性能。衡量并不是很难：https://www.sitespeed.io/documentation/sitespeed.io/Sitespeed.io参考:https://docs.gitlab.com/ee/ci/examples/browser_performance.html https://www.sitespeed.io/documentation/sitespeed.io/configuration/ https://www.sitespeed.io/documentation/sitespeed.io/continuous-integration/pull images[gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ docker pull sitespeedio/sitespeed.io:7.1.3简单测试一下[gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:7.1.3 https://www.linuxea.com/ -b chrome --speedIndex --video Google Chrome 67.0.3396.99 Mozilla Firefox 61.0 [2018-06-29 12:37:45] INFO: Versions OS: linux 4.16.13-1.el7.elrepo.x86_64 nodejs: v8.11.1 sitespeed.io: 7.1.3 browsertime: 3.1.4 coach: 2.0.4 [2018-06-29 12:37:46] INFO: Starting chrome for analysing https://www.linuxea.com/ 3 time(s) [2018-06-29 12:37:46] INFO: Testing url https://www.linuxea.com/ iteration 1 [2018-06-29 12:38:32] INFO: BackEndTime: 3242 DomInteractiveTime: 15760 DomContentLoadedTime: 15760 FirstPaint: 4224 PageLoadTime: 20314 [2018-06-29 12:38:32] INFO: VisualMetrics FirstVisualChange: 4400 SpeedIndex: 11630 PerceptualSpeedIndex: 9421 LastVisualChange: 22133 [2018-06-29 12:38:32] INFO: Testing url https://www.linuxea.com/ iteration 2 [2018-06-29 12:39:04] INFO: BackEndTime: 2668 DomInteractiveTime: 5797 DomContentLoadedTime: 5797 FirstPaint: 3672 PageLoadTime: 12645 [2018-06-29 12:39:04] INFO: VisualMetrics FirstVisualChange: 3833 SpeedIndex: 6462 PerceptualSpeedIndex: 5649 LastVisualChange: 14500 [2018-06-29 12:39:04] INFO: Testing url https://www.linuxea.com/ iteration 3 [2018-06-29 12:39:35] INFO: BackEndTime: 2638 DomInteractiveTime: 6498 DomContentLoadedTime: 6499 FirstPaint: 3700 PageLoadTime: 12690 [2018-06-29 12:39:35] INFO: VisualMetrics FirstVisualChange: 3833 SpeedIndex: 7681 PerceptualSpeedIndex: 5652 LastVisualChange: 14833 [2018-06-29 12:39:35] INFO: 36 requests, 1224.04 kb, backEndTime: 2.85s (±160.46ms), firstPaint: 3.87s (±146.57ms), firstVisualChange: 4.02s (±154.32ms), DOMContentLoaded: 9.35s (±2.62s), Load: 15.22s (±2.08s), speedIndex: 8591 (±1273.50), visualComplete85: 13.06s (±3.14s), lastVisualChange: 17.16s (±2.03s), rumSpeedIndex: 11685 (±1103.72) (3 runs) [2018-06-29 12:39:37] INFO: HTML stored in /sitespeed.io/sitespeed-result/www.linuxea.com/2018-06-29-12-37-45 [2018-06-29 12:39:37] INFO: Finished analysing https://www.linuxea.com/ [gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ 产生一个目录。[gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ ls sitespeed-result/www.linuxea.com/2018-06-29-12-37-45/ assets.html css detailed.html domains.html font help.html img index.html js logs pages pages.html toplist.html [gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ 我们打开它既能看到页面的状态集成GitLab写入到.gitlab.yml中$PWD当前目录挂在进去将JSON文件复制到当前目录，并且让sitespeed-results和JSON文件提供下载#Sitespeed Sitespeed.io: <<: *performance script: - mkdir gitlab-exporter - wget -O ./gitlab-exporter/index.js https://gitlab.com/gitlab-org/gl-performance/raw/master/index.js - mkdir sitespeed-results - docker run --shm-size=1g --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:7.1.3 --plugins.add ./gitlab-exporter --outputFolder sitespeed-results $website || true - cp sitespeed-results/data/performance.json $PWD/performance.json - sudo chown -R gitlab-runner.gitlab-runner $PWD artifacts: paths: - performance.json - sitespeed-results/ 运行完成后的报告在页面中下载打开页面index.html即可查看

linuxea:gitlab-ci/cd DAST(动态应用程序安全测试)-ZAP(七) DASTDAST全称Dynamic Application Security Testing，使用流行的开源工具OWASP ZAProxy 对正在运行的Web应用程序执行分析 OWASP Zed Attack Proxy（ZAP）是全球最受欢迎的免费安全工具之一 ，它可以在测试应用程序自动查找web应用程序中的安全漏洞，也可以用作手动安全测试gitlab官网集成的按照官网https://docs.gitlab.com/ee/ci/examples/dast.html这里来操作，结果失败了，来往下看官网示例如下：dast: image: registry.gitlab.com/gitlab-org/security-products/zaproxy variables: website: "https://example.com" allow_failure: true script: - mkdir /zap/wrk/ - /zap/zap-baseline.py -J gl-dast-report.json -t $website || true - cp /zap/wrk/gl-dast-report.json . artifacts: paths: [gl-dast-report.json]在执行DAST检查之前，还可以对用户进行身份验证：dast: image: registry.gitlab.com/gitlab-org/security-products/zaproxy variables: website: "https://example.com" login_url: "https://example.com/sign-in" username: "john.doe@example.com" password: "john-doe-password" allow_failure: true script: - mkdir /zap/wrk/ - /zap/zap-baseline.py -J gl-dast-report.json -t $website --auth-url $login_url --auth-username $username --auth-password $password || true - cp /zap/wrk/gl-dast-report.json . artifacts: paths: [gl-dast-report.json]写yml文件写一个非常简短的测试，如下stages: - dast dast: stage: dast image: registry.gitlab.com/gitlab-org/security-products/zaproxy variables: website: "https://www.linuxea.com" allow_failure: true script: - mkdir /zap/wrk/ -p - /zap/zap-baseline.py -J gl-dast-report.json -t $website || true - cp /zap/wrk/gl-dast-report.json . artifacts: paths: [gl-dast-report.json] 运行第一次，报文件已存在，因为我之前手动本地创建过(我与我的顽皮)，我加了-p在跑了一次(我与我的顽皮2)跑第二次告诉我没有那个文件这很明显，这条命令被放在本地运行了，并没有在容器内运行，才会报这个错，so，我就改了命令如下开始如果是我手动运行的问题，或者是我用法问题，请留言告诉我，感激不尽优化后集成GitLabpull images(当然，你前面的步骤也需要pull这个image)[gitlab-runner@LinuxEA-VM-Node_10_10_240_145 ~]$ docker pull registry.gitlab.com/gitlab-org/security-products/zaproxy也可以使用官网的镜像：owasp/zap2docker-stable和每周的owasp/zap2docker-weekly YML的JSON部分安全测试Dynamic Application Security Testing这里有两部分，分别是json和html的产生文件，html相对来说更直观，所以有了两个，任选其一即可其中 $website是一个URL，在后面的部分会提到(以上部分已经提到了一次)，这里主要展示zaproxy其中的$PWD部分，是把当前目录挂载到容器内，JSON和HTML文件会生成在$PWD目录中(当然，也可以是其他目录)||true用法在这里主要是返回一个文件(如果运行完成的话)，且不论结果如何都会为真，则会进行下一步，也就是返回文件，因为要的就是那份JSON或者HTML文件JSON部分如下：1/2 DAST_ZAP_JSON: stage: code-check image: docker:stable variables: DOCKER_DRIVER: overlay2 PATHD: /home/gitlab-runner/Increment/ allow_failure: true services: - docker:stable-dind script: - if [ `docker ps -a|egrep "owasp"|wc -l` -gt 0 ];then echo "this $(docker ps -a|awk '/owasp/{print $2}') been deleted" && docker ps -a|docker rm -f $(egrep "owasp"|awk -F' ' 'END{print $NF}'); else echo "Nothing owasp/.... Runing"; fi - docker run --rm --volume /etc/localtime:/etc/localtime:ro --volume $PWD:/zap/wrk/:rw -t "registry.gitlab.com/gitlab-org/security-products/zaproxy" zap-baseline.py -t $website -g gen.conf -J gl-dast-report.json || true - date dependencies: - deploy artifacts: paths: [gl-dast-report.json] YML的HTML部分2/2 DAST_ZAP_HTML: stage: code-check image: docker:stable variables: DOCKER_DRIVER: overlay2 PATHD: /home/gitlab-runner/Increment/ allow_failure: true services: - docker:stable-dind script: - if [ `docker ps -a|egrep "owasp"|wc -l` -gt 0 ];then echo "this $(docker ps -a|awk '/owasp/{print $2}') been deleted" && docker ps -a|docker rm -f $(egrep "owasp"|awk -F' ' 'END{print $NF}'); else echo "Nothing owasp/.... Runing"; fi - docker run --rm --volume /etc/localtime:/etc/localtime:ro --volume $PWD:/zap/wrk/:rw -t "registry.gitlab.com/gitlab-org/security-products/zaproxy" zap-baseline.py -t $website -g gen.conf -r testreport.html || true - date artifacts: paths: [testreport.html] 当运行完成后，我们可以下载查看文件的结果直接打开html它可能是这样的在上面用的是gitlab官网的镜像，也可以直接用zaproxy的镜像来做DAST集成参考：https://docs.gitlab.com/ee/ci/examples/dast.html https://gitlab.com/gitlab-org/security-products/zaproxyZAP参考：https://github.com/zaproxy/zaproxy https://github.com/zaproxy/zaproxy/wiki/Packaged-Scans https://github.com/zaproxy/zaproxy/wiki/Docker

linuxea:gitlab-ci/cd license_management许可证管理 (六) license_managementGitLab工具，用于检测由提供的源使用的依赖关系的许可证。它目前仅基于许可证查找器，但将来可能会发生变化。关于这一项可以不加入到自动化里面支持的语言和包管理器JavaScript --->> Bower, npmGo --->> Godep, go get Java --->> Gradle, Maven.NET --->> NugetPython --->> pipRuby --->> gem集成Gitlab这里的小细节是，需要判断"dependency-scanning"容器是否有搁置的，如果有就删除，这一点很重要因为在运行的过程中，会出现很多问题，比如中断，失败等，如果没有删除则会出问题--rm非常重要5/8 license_management: stage: code-check image: docker:stable variables: DOCKER_DRIVER: overlay2 PATHD: /home/gitlab-runner/Increment/ allow_failure: true services: - docker:stable-dind script: - if [ `docker ps -a|egrep "dependency-scanning"|wc -l` -gt 0 ];then docker ps -a|docker rm -f $(egrep "dependency-scanning"|awk -F' ' 'END{print $NF}'); else echo "Nothing dependency-scanning Runing"; fi - export LICENSE_MANAGEMENT_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') - docker run --rm --volume /etc/localtime:/etc/localtime:ro --volume "$PWD:/code" "registry.gitlab.com/gitlab-org/security-products/license-management:latest" analyze /code - date artifacts: paths: [gl-license-management-report.json] 在 开始之前，我们可以测试下：[gitlab-runner@LinuxEA-VM-Node_10_10_240_145 linuxea]$ docker run --rm registry.gitlab.com/gitlab-org/security-products/license-management /test/test.sh [INFO] Installing /code/java-maven/target/java-maven-1.0-SNAPSHOT.jar to /root/.m2/repository/com/gitlab/security_products/tests/java-maven/1.0-SNAPSHOT/java-maven-1.0-SNAPSHOT.jar [INFO] Installing /code/java-maven/pom.xml to /root/.m2/repository/com/gitlab/security_products/tests/java-maven/1.0-SNAPSHOT/java-maven-1.0-SNAPSHOT.pom [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 02:54 min [INFO] Finished at: 2018-07-02T02:41:15Z [INFO] Final Memory: 20M/133M [INFO] ------------------------------------------------------------------------ Running license_finder in /code/java-maven LicenseFinder::Maven: is active /code/java-maven All tests are OK.那么在使用前需要判断一下是否存在license_management，如果没有则跳过打印License management is not available in your subscription如果引用的话，那大概是这样的，如下： function license_management() { export LICENSE_MANAGEMENT_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') if [ `docker ps -a|egrep "license_management"|wc -l` -gt 0 ];then echo "this $(docker ps -a|awk '/license_management/{print $2}') Will be deleted" docker ps -a|docker rm -f $(egrep "license_management"|awk -F' ' 'END{print $NF}'); else echo "Nothing license_management Runing"; fi if echo $GITLAB_FEATURES |grep license_management > /dev/null ; then # Extract "MAJOR.MINOR" from CI_SERVER_VERSION and generate "MAJOR-MINOR-stable" LICENSE_MANAGEMENT_VERSION=$(echo "$CI_SERVER_VERSION" | sed 's/^\([0-9]*\)\.\([0-9]*\).*/\1-\2-stable/') echo "license_management Start running now ...." docker run --volume "$PWD:/code" \ "registry.gitlab.com/gitlab-org/security-products/license-management:$LICENSE_MANAGEMENT_VERSION" analyze /code else echo "License management is not available in your subscription" fi }License management，因为没有license_management，报License management is not available in your subscription后结束