一、基于Docker配置构建资源池¶
1.1 整体架构¶

1.2 Docker安装Jenkins Slave节点(静态)¶
首先我们在Jenkins的节点管理中,添加节点。输入节点的名称和类型。
配置节点信息:自定义目录 启动方式: Launch agent by connecting it to the master 。
docker-agent# of executors: 5/home/vagrant/workspace- Labels:
docker-agent


获取JNLP方式运行slave所需要的秘钥信息。
秘钥信息获取方式,点击下载slave-agent.jnlp,打开文件,及获取秘钥信息
<jnlp codebase="http://192.168.33.11:8080/computer/docker-agent/" spec="1.0+"><information><title>Agent for docker-agent</title><vendor>Jenkins project</vendor><homepage href="https://jenkins-ci.org/"></homepage></information><security><all-permissions></all-permissions></security><resources><j2se version="1.8+"></j2se><jar href="http://192.168.33.11:8080/jnlpJars/remoting.jar"></jar></resources><application-desc main-class="hudson.remoting.jnlp.Main"><argument>857f47abd55cb5c13567f8cc8c8ddfb77c5af04260efc2436b50a8b896b522e6</argument><argument>docker-agent</argument><argument>-workDir</argument><argument>/home/vagrant/workspace</argument><argument>-internalDir</argument><argument>remoting</argument><argument>-url</argument><argument>http://192.168.33.11:8080/</argument></application-desc></jnlp>
- 857f47abd55cb5c13567f8cc8c8ddfb77c5af04260efc2436b50a8b896b522e6
获取jnlp slave的docker镜像: https://hub.docker.com/r/jenkins/inbound-agent/
docker pull jenkins/inbound-agent
docker run --init -itd jenkins/inbound-agent -url http://192.168.33.11:8080 857f47abd55cb5c13567f8cc8c8ddfb77c5af04260efc2436b50a8b896b522e6 docker-agent
启动slave测试,出现以下日志表示成功连接。这个部分容易出现问题,原因很可能是因为网络权限导致的。
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main createEngine
INFO: Setting up agent: docker-agent
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener <init>
INFO: Jenkins agent is running in headless mode.
Aug 19, 2020 12:48:11 PM hudson.remoting.Engine startEngine
INFO: Using Remoting version: 4.3
Aug 19, 2020 12:48:11 PM hudson.remoting.Engine startEngine
WARNING: No Working Directory. Using the legacy JAR Cache location: /home/jenkins/.jenkins/cache/jars
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Locating server among [http://192.168.33.11:8080]
Aug 19, 2020 12:48:11 PM org.jenkinsci.remoting.engine.JnlpAgentEndpointResolver resolve
INFO: Remoting server accepts the following protocols: [JNLP4-connect, Ping]
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Agent discovery successful
Agent address: 192.168.33.11
Agent port: 50000
Identity: 50:9b:15:1c:3b:3f:ab:bf:86:0b:82:fb:56:56:e5:f6
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Handshaking
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connecting to 192.168.33.11:50000
Aug 19, 2020 12:48:11 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Trying protocol: JNLP4-connect
Aug 19, 2020 12:48:12 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Remote identity confirmed: 50:9b:15:1c:3b:3f:ab:bf:86:0b:82:fb:56:56:e5:f6
Aug 19, 2020 12:48:12 PM hudson.remoting.jnlp.Main$CuiListener status
INFO: Connected
查看效果

pipeline {
agent { node { label "docker-agent" }}
stages {
stage('Test') {
steps {
sleep(20)
echo "Hello world!"
}
}
}
}
console output
Running on docker-agent in /home/vagrant/workspace/workspace/docker-agent2
...

1.3 基于Docker配置Jenkins Slave节点(动态)¶
docker插件: https://plugins.jenkins.io/docker-plugin/
项目地址: https://github.com/jenkinsci/docker-plugin
对与Jenkins动态slave的配置,其实就是Jenkins调用Docker的接口完成的。我们需要开启Docker远程访问。
docker 开启API远程访问 (mac) 参考文档: https://juejin.im/entry/5bdf04b06fb9a049e41223f1
docker pull bobrik/socat
docker run -d -v /var/run/docker.sock:/var/run/docker.sock -p 2376:2375 bobrik/socat TCP4-LISTEN:2375,fork,reuseaddr UNIX-CONNECT:/var/run/docker.sock
如果你的docker环境是使用的centos系统,可以做如下配置。编辑/usr/lib/systemd/system/docker.service
$ sudo vim /usr/lib/systemd/system/docker.service
...
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix://var/run/docker.sock
sudo systemctl daemon-reload
sudo systemctl restart docker
当我们把上面的配置完成后,可以通过Curl命令进行基本的测试API。出现一下信息表示成功开启。
$ curl -XGET http://127.0.0.1:2375/version
{"Platform":{"Name":"Docker Engine - Community"},"Components":[{"Name":"Engine","Version":"19.03.12","Details":{"ApiVersion":"1.40","Arch":"amd64","BuildTime":"2020-06-22T15:45:28.000000000+00:00","Experimental":"false","GitCommit":"48a66213fe","GoVersion":"go1.13.10","KernelVersion":"3.10.0-957.12.2.el7.x86_64","MinAPIVersion":"1.12","Os":"linux"}},{"Name":"containerd","Version":"1.2.13","Details":{"GitCommit":"7ad184331fa3e55e52b890ea95e65ba581ae3429"}},{"Name":"runc","Version":"1.0.0-rc10","Details":{"GitCommit":"dc9208a3303feef5b3839f4323d9beb36df0a9dd"}},{"Name":"docker-init","Version":"0.18.0","Details":{"GitCommit":"fec3683"}}],"Version":"19.03.12","ApiVersion":"1.40","MinAPIVersion":"1.12","GitCommit":"48a66213fe","GoVersion":"go1.13.10","Os":"linux","Arch":"amd64","KernelVersion":"3.10.0-957.12.2.el7.x86_64","BuildTime":"2020-06-22T15:45:28.000000000+00:00"}
Jenkins配置
我们需要安装插件 Docker plugin
1.3.1 方式1: 启动镜像进行构建(无需连接master)¶
dockerNode(dockerHost: 'tcp://192.168.33.11:2375', image: 'jenkins/inbound-agent') {
// some block
sh "sleep 50"
}

[Pipeline] Start of Pipeline
[Pipeline] dockerNode
Launching new docker node based on jenkins/inbound-agent
Pulling from jenkins/inbound-agent
Digest: sha256:9392c06eaef92cebe60deae0581541e8ef19197533cd628d792374c4a32199c9
Status: Image is up to date for jenkins/inbound-agent:latest
Waiting for node to be online ...
Node docker-0001vtsr0hv0n is online.
[Pipeline] {
[Pipeline] sh
+ sleep 50
[Pipeline] }
Terminating docker node ...
Disconnected computer for node 'docker-0001vtsr0hv0n'.
Removed Node for node 'docker-0001vtsr0hv0n'.
1.3.2 方式2: 使用CLoud¶
Manage Jenkins -> Manage Nodes and Clouds -> Configure Clouds
Docker Cloud details
- Docker Host URI Docker主机信息(需要开启Docker配置)
- tcp://192.168.33.11:2375
- unix:///var/run/docker.sock
- Connection Timeout 连接超时时间
- Read Timeout 读操作超时时间 (调大些,容易出现超时的情况)
- Enabled 是否启用?默认否
-
Error Duration 错误的持续时间 默认300 5分钟
-
Container Cap 容器数量 负值或零,或2147483647都意味着“无限制” ,默认值100。
- 10

Docker Agent templates
- Labels 节点标签:
jenkins-docker-agent - Enabled 是否启动 默认否
- Name 节点名称
jenkins-docker-agent - Docker Image 镜像标签:
jenkins/inbound-agent - Remote File System Root 远程文件系统根目录
/home/jenkins - 用法 自定义指定项目运行
- Connect method 连接方式
- JNLP 推荐
- User 运行用户 :
jenkins - Jenkins URL jenkins地址
http://192.168.33.11:8080 - Idle timeout 空闲时间多少秒后杀死slave
- SSH
- Docker Container
- Pull strategy 镜像下载策略
- Pull timeout 镜像下载超时时间 单位秒
- Instance Capacity 实例数量

测试
node("jenkins-docker-agent"){
sh "sleep 30"
echo "Hello world!"
}

[Pipeline] node
Running on jenkins-docker-agent-0001w25yyyc8a on docker in /home/jenkins/workspace/dokcer-agent3
[Pipeline] {
[Pipeline] sh
二、基于Docker的pipeline流水线¶
2.1 容器中编译流水线作业¶
2.1.1 准备工作¶
配置JenkinsMaster挂载Docker
docker run -d -p 8080:8080 -p 50000:50000 --env=JAVA_OPTS=-Djenkins.install.runSetupWizard=false -v /var/lib/jenkins:/var/jenkins_home -v /var/run/docker.sock:/var/run/docker.sock -v /usr/bin/docker:/usr/bin/docker jenkins:v20200816
解决权限问题/以root用户运行
docker exec -it 005a7714ecc9 bash
jenkins@005a7714ecc9:/$ docker ps
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.40/containers/json: dial unix /var/run/docker.sock: connect: permission denied
$ docker exec -it -u root 005a7714ecc9 bash
root@005a7714ecc9:/# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
005a7714ecc9 jenkins:v20200816 "/sbin/tini -- /usr/…" 4 minutes ago Up 4 minutes 0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp hopeful_pascal
384516ab8579 jenkins/inbound-agent "jenkins-agent -url …" 12 minutes ago Up 12 minutes blissful_nash
root@005a7714ecc9:/# usermod -aG root jenkins
root@005a7714ecc9:/# exit
$ docker restart 005a7714ecc9
$ id jenkins
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins),0(root)
我的问题
我在主机上创建了docker的group, 根据这个文章 https://docs.docker.com/engine/install/linux-postinstall/
sudo groupadd docker
sudo usermod -aG docker $USER
$ cat /etc/group
...
docker:x:993:vagrant
$ docker exec -it -u root 005a7714ecc9 bash
$ groupadd -g 993 docker-test
root@005a7714ecc9:/# usermod -aG docker-test jenkins
root@005a7714ecc9:/# id jenkins
uid=1000(jenkins) gid=1000(jenkins) groups=1000(jenkins),0(root),1001(docker),993(docker-test)
root@005a7714ecc9:/# cat /etc/group
...
docker-test:x:993:jenkins
$ docker exec -it 005a7714ecc9 bash
jenkins@005a7714ecc9:/$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
005a7714ecc9 jenkins:v20200816 "/sbin/tini -- /usr/…" 5 hours ago Up 3 hours 0.0.0.0:8080->8080/tcp, 0.0.0.0:50000->50000/tcp hopeful_pascal
jenkins should have permission now.
Getting “Permission Denied” error when pulling a docker image in Jenkins docker container on Mac
2.1.2 测试流水线¶
pipeline {
agent {
docker {
image 'maven:3.6.3-jdk-8'
args '-v $HOME/.m2:/root/.m2'
}
}
stages {
stage('Build') {
steps {
sh 'mvn -v'
}
}
}
}
Console Output
Running on Jenkins in /var/jenkins_home/workspace/docker-pipeline1
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . maven:3.6.3-jdk-8
.
[Pipeline] withDockerContainer
Jenkins seems to be running inside container 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3
$ docker run -t -d -u 1000:1000 -v $HOME/.m2:/root/.m2 -w /var/jenkins_home/workspace/docker-pipeline1 --volumes-from 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3 -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** maven:3.6.3-jdk-8 cat
$ docker top c8aec18483a36a66c120975c09b667ba0292a7abebbe3b4409a5bbc1d23469ae -eo pid,comm
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Build)
[Pipeline] sh
+ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/share/maven
Java version: 1.8.0_265, vendor: Oracle Corporation, runtime: /usr/local/openjdk-8/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.12.2.el7.x86_64", arch: "amd64", family: "unix"
[Pipeline] sleep
Sleeping for 30 sec
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
$ docker stop --time=1 c8aec18483a36a66c120975c09b667ba0292a7abebbe3b4409a5bbc1d23469ae
$ docker rm -f c8aec18483a36a66c120975c09b667ba0292a7abebbe3b4409a5bbc1d23469ae
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
2.1.3 前后端未分离项目¶
对于代码库中既包含前端项目又包含后端项目的配置。可以启动多个容器。
pipeline {
agent none
stages {
stage('ServiceBuild') {
agent {
docker {
image 'maven:3.6.3-jdk-8'
args '-v $HOME/.m2:/root/.m2'
}
}
steps {
sh 'mvn -v && sleep 15'
}
}
stage('WebBuild') {
agent {
docker {
image 'node:7-alpine'
args '-v $HOME/.npm:/root/.npm'
}
}
steps {
sh 'node -v && sleep 15'
}
}
}
}
Console output
Running on Jenkins in /var/jenkins_home/workspace/docker-pipeline2
[Pipeline] {
[Pipeline] isUnix (hide)
[Pipeline] sh
+ docker inspect -f . maven:3.6.3-jdk-8
.
[Pipeline] withDockerContainer
Jenkins seems to be running inside container 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3
$ docker run -t -d -u 1000:1000 -v $HOME/.m2:/root/.m2 -w /var/jenkins_home/workspace/docker-pipeline2 --volumes-from 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3 -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** maven:3.6.3-jdk-8 cat
$ docker top 9f15b56640e3d65660876a4f854918103f7f099150dbb5f99dff43ace1296392 -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/share/maven
Java version: 1.8.0_265, vendor: Oracle Corporation, runtime: /usr/local/openjdk-8/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.12.2.el7.x86_64", arch: "amd64", family: "unix"
+ sleep 15
[Pipeline] }
$ docker stop --time=1 9f15b56640e3d65660876a4f854918103f7f099150dbb5f99dff43ace1296392
$ docker rm -f 9f15b56640e3d65660876a4f854918103f7f099150dbb5f99dff43ace1296392
[Pipeline] // withDockerContainer
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (WebBuild)
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/docker-pipeline2
[Pipeline] {
[Pipeline] isUnix
[Pipeline] sh
+ docker inspect -f . node:7-alpine
.
[Pipeline] withDockerContainer
Jenkins seems to be running inside container 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3
$ docker run -t -d -u 1000:1000 -v $HOME/.npm:/root/.npm -w /var/jenkins_home/workspace/docker-pipeline2 --volumes-from 005a7714ecc980f673c3083030ff79a82687681de55d9cd9875625a5aa0520f3 -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** node:7-alpine cat
$ docker top 33be7fc88b0f15f3cd3f947250e3b0d6d57f05096b6bac2bee7006cf0e380680 -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ node -v
v7.10.1
+ sleep 15
[Pipeline] }
$ docker stop --time=1 33be7fc88b0f15f3cd3f947250e3b0d6d57f05096b6bac2bee7006cf0e380680
$ docker rm -f 33be7fc88b0f15f3cd3f947250e3b0d6d57f05096b6bac2bee7006cf0e380680
[Pipeline] // withDockerContainer
2.1.4 前端项目流水线¶
$ cd /home/vagrant/workspace/workspace/demo-pipeline3
// 安装 vue-cli
$ sudo chown -R 1000:1000 "/home/vagrant/.npm"
$ npm install -g @vue/cli-init
npm WARN deprecated vue-cli@2.9.6: This package has been deprecated in favour of @vue/cli
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
npm WARN deprecated coffee-script@1.12.7: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
+ @vue/cli-init@4.5.4
added 251 packages from 206 contributors in 17.774s
$ vue --version
@vue/cli 4.5.4
$ cd /home/vagrant/workspace/workspace/demo-pipeline3
$ $ vue init webpack demo
? Project name demo
? Project description A Vue.js project
? Author
? Vue build standalone
? Install vue-router? Yes
? Use ESLint to lint your code? Yes
? Pick an ESLint preset Standard
? Set up unit tests Yes
? Pick a test runner jest
? Setup e2e tests with Nightwatch? No
? Should we run `npm install` for you after the project has been created? (recommended) npm
vue-cli · Generated "demo".
## Installing project dependencies ...
编写jenkinsfile
pipeline {
agent {node {label "hostmachine"}}
stages {
stage('WebBuild') {
steps {
script {
docker.image('node:10.19.0-alpine').inside('-u 0:0 -v /var/jenkins_home/.npm:/root/.npm') {
sh """
id
ls /root/.npm
ls /root/ -a
npm config set unsafe-perm=true
npm config list
npm config set cache /root/.npm
#npm config set registry https://registry.npm.taobao.org
npm config list
ls
cd demo && npm install --unsafe-perm=true && npm run build && ls -l dist/ && sleep 15
"""
}
}
}
}
}
}
Console output
$ docker run -t -d -u 1000:1000 -u 0:0 -v /var/jenkins_home/.npm:/root/.npm -w /home/vagrant/workspace/workspace/demo-pipeline3 -v /home/vagrant/workspace/workspace/demo-pipeline3:/home/vagrant/workspace/workspace/demo-pipeline3:rw,z -v /home/vagrant/workspace/workspace/demo-pipeline3@tmp:/home/vagrant/workspace/workspace/demo-pipeline3@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** node:10.19.0-alpine cat
$ docker top ea767dafdc3ba98baf9399ebcde3fd491911d77a0c993f92a8a3b6f44a51fc7a -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ id
uid=0(root) gid=0(root)
+ ls /root/.npm
_cacache
_locks
_logs
anonymous-cli-metrics.json
+ npm config set 'unsafe-perm=true'
+ npm config list
; cli configs
metrics-registry = "https://registry.npmjs.org/"
scope = ""
user-agent = "npm/6.13.4 node/v10.19.0 linux x64 ci/jenkins"
; userconfig /root/.npmrc
unsafe-perm = true
; node bin location = /usr/local/bin/node
; cwd = /home/vagrant/workspace/workspace/demo-pipeline3
; HOME = /root
; "npm config ls -l" to show all defaults.
+ npm config set cache /root/.npm
+ npm config list
; cli configs
metrics-registry = "https://registry.npmjs.org/"
scope = ""
user-agent = "npm/6.13.4 node/v10.19.0 linux x64 ci/jenkins"
; userconfig /root/.npmrc
cache = "/root/.npm"
unsafe-perm = true
; node bin location = /usr/local/bin/node
; cwd = /home/vagrant/workspace/workspace/demo-pipeline3
; HOME = /root
; "npm config ls -l" to show all defaults.
+ ls
demo
package-lock.json
+ cd demo
+ npm install '--unsafe-perm=true'
Cannot contact hostmachine: java.lang.InterruptedException
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.3 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.2.13 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
audited 1734 packages in 10.906s
28 packages are looking for funding
run `npm fund` for details
found 89 vulnerabilities (73 low, 9 moderate, 7 high)
run `npm audit fix` to fix them, or `npm audit` for details
+ npm run build
> demo@1.0.0 build /home/vagrant/workspace/workspace/demo-pipeline3/demo
> node build/build.js
Hash: [1mc4e4fdef7b4fbb5efe81[39m[22m
Version: webpack [1m3.12.0[39m[22m
Time: [1m12173[39m[22mms
[1mAsset[39m[22m [1mSize[39m[22m [1mChunks[39m[22m [1m[39m[22m [1m[39m[22m[1mChunk Names[39m[22m
[1m[32mstatic/js/vendor.936b7041a764ab1c3f2c.js[39m[22m 123 kB [1m0[39m[22m [1m[32m[emitted][39m[22m vendor
[1m[32mstatic/js/app.b22ce679862c47a75225.js[39m[22m 11.6 kB [1m1[39m[22m [1m[32m[emitted][39m[22m app
[1m[32mstatic/js/manifest.2ae2e69a05c33dfc65f8.js[39m[22m 857 bytes [1m2[39m[22m [1m[32m[emitted][39m[22m manifest
[1m[32mstatic/css/app.30790115300ab27614ce176899523b62.css[39m[22m 432 bytes [1m1[39m[22m [1m[32m[emitted][39m[22m app
[1m[32mstatic/css/app.30790115300ab27614ce176899523b62.css.map[39m[22m 797 bytes [1m[39m[22m [1m[32m[emitted][39m[22m
[1m[32mstatic/js/vendor.936b7041a764ab1c3f2c.js.map[39m[22m 619 kB [1m0[39m[22m [1m[32m[emitted][39m[22m vendor
[1m[32mstatic/js/app.b22ce679862c47a75225.js.map[39m[22m 22.2 kB [1m1[39m[22m [1m[32m[emitted][39m[22m app
[1m[32mstatic/js/manifest.2ae2e69a05c33dfc65f8.js.map[39m[22m 4.97 kB [1m2[39m[22m [1m[32m[emitted][39m[22m manifest
[1m[32mindex.html[39m[22m 506 bytes [1m[39m[22m [1m[32m[emitted][39m[22m
Build complete.
Tip: built files are meant to be served over an HTTP server.
Opening index.html over file:// won't work.
+ ls -l dist/
total 4
-rw-r--r-- 1 root root 506 Aug 20 01:34 index.html
drwxr-xr-x 4 root root 27 Aug 20 01:34 static
+ sleep 15
[Pipeline] }
$ docker stop --time=1 ea767dafdc3ba98baf9399ebcde3fd491911d77a0c993f92a8a3b6f44a51fc7a
$ docker rm -f ea767dafdc3ba98baf9399ebcde3fd491911d77a0c993f92a8a3b6f44a51fc7a
2.1.5 FAQ¶
npm构建权限问题:使用root用户构建。设置容器运行用户 -u 0:0
三、构建应用镜像到镜像仓库管理¶
3.1 构建应用镜像到镜像仓库管理¶
将应用构建成镜像、将镜像上传到镜像仓库非常简单。通过命令就能解决。镜像仓库巨大爆满如何解决? 我们需要在开始使用前就应该设置好镜像的构建策略。(每个公司中管理不一样,具体可根据分支策略决定)。
3.2 分支开发策略¶
- 主干分支
master - 特性分支
f1 f2 - 版本分支
release
你会发现:
- 特性分支需要构建发布到特性环境验证。
- 版本分支需要构建发布到
UAT/STAG/PROD环境验证。
考虑到:
- 特性分支产生的镜像会很多,而且并不重要可以定时清理掉。
- 版本分支产生相对较少(考虑到版本修复情况),每个版本只有一个镜像。(上线发布完成后清除掉其他)

3.3 镜像管理规范¶
3.3.1 命名规范¶
- 仓库类型
- snapshot : 开发版本仓库
- release : 生产正式版本仓库
- 仓库命名
- snapshot : 业务/项目名称-snapshot demo-snapshot
- release : 业务/项目名称-release demo-release
- 镜像命名
- (业务/项目名称)/应用名称/标签
- DEV :
demo-snapshot/demo-devops-service:branch_commitid - PRD:
demo-release/demo-devops-service:version_commitid
- 标签命名
分支名_提交ID版本号_提交ID
提交ID的意义: 减少重复构建
3.3.2 镜像清理策略¶

随着镜像越来越多,频繁更新导致Harbor镜像仓库容量很快爆满。
- snapshot仓库: 每天晚上定时清理前一天的镜像(注意风险)
-
release仓库: 版本发布完成后,清除版本其他镜像
-
1.1.1_xxxxxxxxxx11
-
1.1.1_xxxxxxxxxx22
3.4 构建应用镜像¶
3.4.1 编写应用Dockerfile¶
$ cd demo
$ pwd
/home/vagrant/workspace/workspace/demo-pipeline3/demo
$ touch Dockerfile
$ vim Dockerfile
FROM nginx:latest
COPY dist /usr/share/nginx/html
3.4.2 配置流水线构建镜像¶
docker build -t demo-web-app:1.1.1 .
3.4.3 上传镜像¶
docker push demo-web-app:1.1.1
Jenkinsfile
pipeline {
agent {node {label "hostmachine"}}
stages {
stage('WebBuild') {
steps {
script {
docker.image('node:10.19.0-alpine').inside('-u 0:0 -v /var/jenkins_home/.npm:/root/.npm') {
sh """
id
ls /root/.npm
ls /root/ -a
npm config set unsafe-perm=true
npm config list
npm config set cache /root/.npm
#npm config set registry https://registry.npm.taobao.org
npm config list
ls
cd demo && npm install --unsafe-perm=true && npm run build && ls -l dist/ && sleep 15
"""
}
}
}
}
stage("BuildImage"){
steps {
script{
sh """
#构建镜像
cd demo
docker build -t demo/demo-web-app:1.1.1_xxxxxxxx1 .
#docker push demo/demo-web-app:1.1.1_xxxxxxxx1
"""
}
}
}
}
}
console output
+ cd demo
+ docker build -t demo/demo-web-app:1.1.1_xxxxxxxx1 .
Sending build context to Docker daemon 203MB
..
Successfully built 3149ee3602a7
Successfully tagged demo/demo-web-app:1.1.1_xxxxxxxx1
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
demo/demo-web-app 1.1.1_xxxxxxxx1 3149ee3602a7 6 minutes ago 133MB
运行镜像
docker run -itd -p 8088:80 --name nginx-server demo/demo-web-app:1.1.1_xxxxxxxx1
http://192.168.33.11:8088/#/

3.5 镜像清理(Harbor) 注意风险 扩展研究¶
获取标签
- https://registry.demo.com/api/repositories/${registryName}/${serviceName}/tags
- 方式: GET
删除标签
- https://registry.demo.com/api/repositories/${registryName}/${serviceName}/tags/${tag}
- 方式: DELETE
#!groovy
@Library('jenkinslib@master') _
def tools = new org.devops.tools()
String registryName = "${env.registryName}"
String serviceName = "${env.serviceName}"
String tagName = "${env.tagName}"
def harborProjects = []
currentBuild.description = "Trigger by ${serviceName} ${tagName}"
pipeline {
agent { node { label "build"} }
stages{
stage("GetHarborTags"){
steps{
timeout(time:5, unit:"MINUTES"){
script{
tools.PrintMes("获取Harbor仓库中的项目信息","green")
println(serviceName)
try {
response = httpRequest authentication: 'harbor-admin,
url: "https://registry.demo.com/api/repositories/${registryName}/${serviceName}/tags",
ignoreSslErrors: true
//println(response.content)
response = readJSON text: """${response.content}"""
} catch(e){
response = ['name':'']
println(e)
println("Harbor镜像不存在此标签!")
}
/*println(tagName)
for (tagname in response){
//println(response)
harborProjects << tagname['name']
}
println(harborProjects)*/
}
}
}
}
stage("DeleteHarborTags"){
steps{
timeout(time:20, unit:"MINUTES"){
script{
tools.PrintMes("总共找到 ${harborProjects.size()} 个标签","green")
sumImageNum = harborProjects.size()
for (tag in harborProjects){
sumImageNum -= 1
tools.PrintMes(" ${sumImageNum} Delete Tags ---> ${registryName} --> ${serviceName} --> ${tag} ","green")
httpRequest httpMode: 'DELETE',
authentication: 'c016027e-0573-4246-93cf-f4a55b08a86a',
url: "https://registry.demo.com/api/repositories/${registryName}/${serviceName}/tags/${tag}",
ignoreSslErrors: true
sleep 1
}
}
}
}
}
}
post {
always{
script{
cleanWs notFailBuild: true
}
}
}
}
四、使用Groovy代码初始化Docker配置¶
4.1 解析官方提供的groovy代码¶
插件地址: https://plugins.jenkins.io/docker-plugin/
- https://github.com/jenkinsci/docker-plugin/blob/master/docs/attachments/docker-plugin-configuration-script.groovy
import com.nirima.jenkins.plugins.docker.DockerCloud
import com.nirima.jenkins.plugins.docker.DockerTemplate
import com.nirima.jenkins.plugins.docker.DockerTemplateBase
import com.nirima.jenkins.plugins.docker.launcher.AttachedDockerComputerLauncher
import io.jenkins.docker.connector.DockerComputerAttachConnector
import jenkins.model.Jenkins
src/main/java/com/nirima/jenkins/plugins/docker目录可以看到这些源码文件。

src/main/java/com/nirima/jenkins/plugins/docker/launcher 目录下可以看多支持的启动类型。

src/main/java/io/jenkins/docker/connector 目录下定义的是每个启动类型对应的连接器。

dockerTemplateBaseParameters定义容器的模板所需要的基本参数,实例化新的对象dockerTemplateBase。

//Docker容器的配置参数
def dockerTemplateBaseParameters = [
bindAllPorts: false,
bindPorts: '',
cpuShares: null,
dnsString: '',
dockerCommand: '',
environmentsString: '',
extraHostsString: '',
hostname: '',
image: 'jenkins/inbound-agent:latest',
macAddress: '',
memoryLimit: null,
memorySwap: null,
network: '',
privileged: false,
pullCredentialsId: '',
sharedMemorySize: null,
tty: true,
volumesFromString: '',
volumesString: ''
]
//Docker容器的配置参数
def DockerTemplateParameters = [
instanceCapStr: '4', //实例数量
labelString: 'docker.local.jenkins.slave', //节点标签
remoteFs: '' //根目录
]
//src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java
//实例化参数
DockerTemplateBase dockerTemplateBase = new DockerTemplateBase(
dockerTemplateBaseParameters.image,
dockerTemplateBaseParameters.pullCredentialsId,
dockerTemplateBaseParameters.dnsString,
dockerTemplateBaseParameters.network,
dockerTemplateBaseParameters.dockerCommand,
dockerTemplateBaseParameters.volumesString,
dockerTemplateBaseParameters.volumesFromString,
dockerTemplateBaseParameters.environmentsString,
dockerTemplateBaseParameters.hostname,
dockerTemplateBaseParameters.memoryLimit,
dockerTemplateBaseParameters.memorySwap,
dockerTemplateBaseParameters.cpuShares,
dockerTemplateBaseParameters.sharedMemorySize,
dockerTemplateBaseParameters.bindPorts,
dockerTemplateBaseParameters.bindAllPorts,
dockerTemplateBaseParameters.privileged,
dockerTemplateBaseParameters.tty,
dockerTemplateBaseParameters.macAddress,
dockerTemplateBaseParameters.extraHostsString
)
根据Docker容器的配置参数,创建一个DockerComputerAttachConnector类型的容器模板。
//创建容器模板
// src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplate.java
DockerTemplate dockerTemplate = new DockerTemplate(
dockerTemplateBase,
new DockerComputerAttachConnector(),
DockerTemplateParameters.labelString,
DockerTemplateParameters.remoteFs,
DockerTemplateParameters.instanceCapStr
)
声明DockerCLoud的配置信息:连接超时时间、容器实例数量、认证ID、主机名、读超时时间、服务远程地址、Docker版本信息。
//Cloud配置信息
def dockerCloudParameters = [
connectTimeout: 3,
containerCapStr: '4',
credentialsId: '',
dockerHostname: '',
name: 'docker.local',
readTimeout: 60,
serverUrl: 'unix:///var/run/docker.sock',
version: ''
]
将Docker容器的配置模板加入到DockerCloud模板中,通过Jenkins.clouds添加的jenkins配置。
//配置DockerCloud
// src/main/java/com/nirima/jenkins/plugins/docker/DockerCloud.java
DockerCloud dockerCloud = new DockerCloud(
dockerCloudParameters.name,
[dockerTemplate], //docker容器模板
dockerCloudParameters.serverUrl,
dockerCloudParameters.containerCapStr,
dockerCloudParameters.connectTimeout,
dockerCloudParameters.readTimeout,
dockerCloudParameters.credentialsId,
dockerCloudParameters.version,
dockerCloudParameters.dockerHostname
)
// get Jenkins instance
Jenkins jenkins = Jenkins.getInstance()
// add cloud configuration to Jenkins
jenkins.clouds.add(dockerCloud)
// save current Jenkins state to disk
jenkins.save()
复制写好的代码,在Jenkins脚本命令行运行。
import com.nirima.jenkins.plugins.docker.DockerCloud
import com.nirima.jenkins.plugins.docker.DockerTemplate
import com.nirima.jenkins.plugins.docker.DockerTemplateBase
import com.nirima.jenkins.plugins.docker.launcher.DockerComputerJNLPLauncher
import io.jenkins.docker.connector.DockerComputerJNLPConnector
import jenkins.model.Jenkins
// parameters
def dockerTemplateBaseParameters = [
bindAllPorts: false,
bindPorts: '',
cpuPeriod: null,
cpuQuota: null,
cpuShares: null,
dnsString: '',
dockerCommand: '',
environmentsString: '',
extraHostsString: '',
hostname: '',
image: 'jenkins/inbound-agent:latest',
macAddress: '',
memoryLimit: null,
memorySwap: null,
network: '',
privileged: false,
pullCredentialsId: '',
sharedMemorySize: null,
tty: true,
volumesFromString: '',
volumesString: ''
]
def DockerTemplateParameters = [
instanceCapStr: '4',
labelString: 'docker.local.jenkins.agent',
remoteFs: ''
]
def dockerCloudParameters = [
connectTimeout: 3,
containerCapStr: '4',
credentialsId: '',
dockerHostname: '',
name: 'docker.local',
readTimeout: 60,
serverUrl: 'unix:///var/run/docker.sock',
version: ''
]
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java
DockerTemplateBase dockerTemplateBase = new DockerTemplateBase(
dockerTemplateBaseParameters.image,
dockerTemplateBaseParameters.pullCredentialsId,
dockerTemplateBaseParameters.dnsString,
dockerTemplateBaseParameters.network,
dockerTemplateBaseParameters.dockerCommand,
dockerTemplateBaseParameters.volumesString,
dockerTemplateBaseParameters.volumesFromString,
dockerTemplateBaseParameters.environmentsString,
dockerTemplateBaseParameters.hostname,
dockerTemplateBaseParameters.memoryLimit,
dockerTemplateBaseParameters.memorySwap,
dockerTemplateBaseParameters.cpuPeriod,
dockerTemplateBaseParameters.cpuQuota,
dockerTemplateBaseParameters.cpuShares,
dockerTemplateBaseParameters.sharedMemorySize,
dockerTemplateBaseParameters.bindPorts,
dockerTemplateBaseParameters.bindAllPorts,
dockerTemplateBaseParameters.privileged,
dockerTemplateBaseParameters.tty,
dockerTemplateBaseParameters.macAddress,
dockerTemplateBaseParameters.extraHostsString
)
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplate.java
DockerTemplate dockerTemplate = new DockerTemplate(
dockerTemplateBase,
new DockerComputerAttachConnector(),
DockerTemplateParameters.labelString,
DockerTemplateParameters.remoteFs,
DockerTemplateParameters.instanceCapStr
)
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerCloud.java
DockerCloud dockerCloud = new DockerCloud(
dockerCloudParameters.name,
[dockerTemplate],
dockerCloudParameters.serverUrl,
dockerCloudParameters.containerCapStr,
dockerCloudParameters.connectTimeout,
dockerCloudParameters.readTimeout,
dockerCloudParameters.credentialsId,
dockerCloudParameters.version,
dockerCloudParameters.dockerHostname
)
// get Jenkins instance
Jenkins jenkins = Jenkins.getInstance()
// add cloud configuration to Jenkins
jenkins.clouds.add(dockerCloud)
// save current Jenkins state to disk
jenkins.save()

4.2 实例: 添加一个JNLP类型的Docker Cloud配置¶
不同点1: 需要引入DockerComputerJNLPLauncher、DockerComputerJNLPConnector、hudson.slaves.JNLPLauncher。
import com.nirima.jenkins.plugins.docker.DockerCloud
import com.nirima.jenkins.plugins.docker.DockerTemplate
import com.nirima.jenkins.plugins.docker.DockerTemplateBase
//import com.nirima.jenkins.plugins.docker.launcher.AttachedDockerComputerLauncher
import com.nirima.jenkins.plugins.docker.launcher.DockerComputerJNLPLauncher
//import io.jenkins.docker.connector.DockerComputerAttachConnector
import io.jenkins.docker.connector.DockerComputerJNLPConnector
import jenkins.model.Jenkins
import hudson.slaves.JNLPLauncher;
不同点2: JNLP类型的启动方式需要添加两个变量,分别是Jenkinsuser和Jenkins服务器地址。
// src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplate.java
DockerTemplate dockerTemplate = new DockerTemplate(
dockerTemplateBase,
//new DockerComputerAttachConnector(),
new DockerComputerJNLPConnector(new JNLPLauncher(null, null)).withUser("jenkins")
.withJenkinsUrl("http://123.123.123:8080"),
DockerTemplateParameters.labelString,
DockerTemplateParameters.remoteFs,
DockerTemplateParameters.instanceCapStr
)
参考测试文件,获取JNLP方式的容器模板创建方式。
final DockerTemplate template = new DockerTemplate(
new DockerTemplateBase(JNLP_SLAVE_IMAGE_IMAGENAME),
new DockerComputerJNLPConnector(new JNLPLauncher(null, null)).withUser(COMMON_IMAGE_USERNAME)
.withJenkinsUrl(uri.toString()),
LABEL, COMMON_IMAGE_HOMEDIR, INSTANCE_CAP
);
完整源码
import com.nirima.jenkins.plugins.docker.DockerCloud
import com.nirima.jenkins.plugins.docker.DockerTemplate
import com.nirima.jenkins.plugins.docker.DockerTemplateBase
//import com.nirima.jenkins.plugins.docker.launcher.AttachedDockerComputerLauncher
import com.nirima.jenkins.plugins.docker.launcher.DockerComputerJNLPLauncher
//import io.jenkins.docker.connector.DockerComputerAttachConnector
import io.jenkins.docker.connector.DockerComputerJNLPConnector
import jenkins.model.Jenkins
import hudson.slaves.JNLPLauncher;
// parameters
//Docker Agent 基本参数
def dockerTemplateBaseParameters = [
bindAllPorts: false,
bindPorts: '',
cpuShares: null,
dnsString: '',
dockerCommand: '',
environmentsString: '',
extraHostsString: '',
hostname: '',
image: 'jenkins/inbound-agent:latest', //镜像
macAddress: '',
memoryLimit: null,
memorySwap: null,
network: '',
privileged: false,
pullCredentialsId: '',
sharedMemorySize: null,
tty: true,
volumesFromString: '',
volumesString: ''
]
//Docker Agnet 参数
def DockerTemplateParameters = [
instanceCapStr: '10', //实例数
labelString: 'jenkins-agent-test', //节点标签
remoteFs: '/home/jenkins' //主目录
]
def dockerCloudParameters = [
connectTimeout: 60, //链接超时
containerCapStr: '10', //实例数量
credentialsId: '',
dockerHostname: '',
name: 'docker.local.test',
readTimeout: 60,
serverUrl: 'tcp://192.168.33.11:2375',
version: ''
]
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplateBase.java
DockerTemplateBase dockerTemplateBase = new DockerTemplateBase(
dockerTemplateBaseParameters.image,
dockerTemplateBaseParameters.pullCredentialsId,
dockerTemplateBaseParameters.dnsString,
dockerTemplateBaseParameters.network,
dockerTemplateBaseParameters.dockerCommand,
dockerTemplateBaseParameters.volumesString,
dockerTemplateBaseParameters.volumesFromString,
dockerTemplateBaseParameters.environmentsString,
dockerTemplateBaseParameters.hostname,
dockerTemplateBaseParameters.memoryLimit,
dockerTemplateBaseParameters.memorySwap,
dockerTemplateBaseParameters.cpuPeriod,
dockerTemplateBaseParameters.cpuQuota,
dockerTemplateBaseParameters.cpuShares,
dockerTemplateBaseParameters.sharedMemorySize,
dockerTemplateBaseParameters.bindPorts,
dockerTemplateBaseParameters.bindAllPorts,
dockerTemplateBaseParameters.privileged,
dockerTemplateBaseParameters.tty,
dockerTemplateBaseParameters.macAddress,
dockerTemplateBaseParameters.extraHostsString
)
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerTemplate.java
DockerTemplate dockerTemplate = new DockerTemplate(
dockerTemplateBase,
//new DockerComputerAttachConnector(),
new DockerComputerJNLPConnector(new JNLPLauncher(null, null)).withUser("jenkins")
.withJenkinsUrl("http://192.168.33.11:8080"),
DockerTemplateParameters.labelString,
DockerTemplateParameters.remoteFs,
DockerTemplateParameters.instanceCapStr
)
// https://github.com/jenkinsci/docker-plugin/blob/docker-plugin-1.1.2/src/main/java/com/nirima/jenkins/plugins/docker/DockerCloud.java
DockerCloud dockerCloud = new DockerCloud(
dockerCloudParameters.name,
[dockerTemplate],
dockerCloudParameters.serverUrl,
dockerCloudParameters.containerCapStr,
dockerCloudParameters.connectTimeout,
dockerCloudParameters.readTimeout,
dockerCloudParameters.credentialsId,
dockerCloudParameters.version,
dockerCloudParameters.dockerHostname
)
// get Jenkins instance
Jenkins jenkins = Jenkins.getInstance()
// add cloud configuration to Jenkins
jenkins.clouds.add(dockerCloud)
// save current Jenkins state to disk
jenkins.save()
