三、实战练习¶
登录http://gitlab.example.com/，账号为
root,密码为<gitlab-password>。登录成功后，参考前面内容创建名为Argocd Example Apps的项目，标识字符串为argocd-example-apps
创建一个 Rollout 资源和一个针对该资源的 Kubernetes Service 对象，这里的 Rollout 采用了蓝绿部署：
[root@master01 ~]# cd /root/17/argo-rollouts
[root@master01 argo-rollouts]# mkdir -p argorollout-examples/bluegreen
[root@master01 argo-rollouts]# cd argorollout-examples/bluegreen
[root@master01 bluegreen]# vim rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: newapp-rollout-0902
namespace: demo
spec:
replicas: 3
strategy:
blueGreen:
activeService: bluegreen-appv1
previewService: bluegreen-appv2
autoPromotionEnabled: false
revisionHistoryLimit: 2
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web
image: registry.cn-hangzhou.aliyuncs.com/abroad_images/kubernetes-bootcamp:v1
ports:
- name: http
containerPort: 8080
protocol: TCP
web-app 应用的 appv1-svc：
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/bluegreen
[root@master01 bluegreen]# vim web-app-appv1.yaml
apiVersion: v1
kind: Service
metadata:
name: bluegreen-appv1
namespace: demo
spec:
selector:
app: web-app
ports:
- port: 8080
protocol: TCP
targetPort: 8080
web-app 应用的 appv2-svc：
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/bluegreen
[root@master01 bluegreen]# vim web-app-appv2.yaml
apiVersion: v1
kind: Service
metadata:
name: bluegreen-appv2
namespace: demo
spec:
selector:
app: web-app
ports:
- port: 8080
protocol: TCP
targetPort: 8080
把上述3个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git的argorollout-examples/bluegreen 目录下
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# git init
# 添加远端仓库
[root@master01 argorollout-examples]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git
# 验证查看
[root@master01 argorollout-examples]# git remote -v
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (push)
# 添加到暂存区
[root@master01 argorollout-examples]# git add .
# 提交到本地仓库
[root@master01 argorollout-examples]# git commit -m "first for argocd-example-apps"
# 切换到main分支
[root@master01 argorollout-examples]# git branch -M main
# 上传到main分支
[root@master01 argorollout-examples]# git push -uf origin main
Username for 'http://gitlab.example.com': root
Password for 'http://root@gitlab.example.com': <gitlab-password>
添加gitlab仓库
# 先使用初始密码登录
[root@master01 17]# argocd login argocd.example.com
Username: admin
Password: <new-admin-password>
# 添加前需要保证私有gitlab仓库不为空
[root@master01 argo-rollouts]# argocd repo add http://gitlab.example.com/demoteam/argocd-example-apps.git --username root --password <gitlab-password> --insecure-skip-server-verification
# 验证查看
[root@master01 argorollout-examples]# argocd repo list | grep argocd-example-apps
git http://gitlab.example.com/demoteam/argocd-example-apps.git true false false true Successful
后续在 Argo CD 上使用 YAML 资源清单创建服务：
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-blue-green.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argorollout-blue-green
namespace: argocd
spec:
destination:
name: ''
namespace: demo
server: 'https://kubernetes.default.svc'
source:
path: bluegreen
repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
targetRevision: main
sources: []
project: default
# 应用
[root@master01 argorollout-examples]# kubectl apply -f argorollout-blue-green.yaml
最终目录文件结构：
[root@master01 argorollout-examples]# tree ../argorollout-examples/
../argorollout-examples/
├── argorollout-blue-green.yaml
└── bluegreen
├── rollout.yaml
├── web-app-appv1.yaml
└── web-app-appv2.yaml
1 directory, 4 files
web页面手动实现同步
依次点击【SYNC】-【SYNCHEONIZE】
web 页面查看状态：
查看网络流向：
Argo Rollouts 查看服务：
# 查看服务
[root@master01 argorollout-examples]# kubectl argo rollouts list rollouts -ndemo
NAME STRATEGY STATUS STEP SET-WEIGHT READY DESIRED UP-TO-DATE AVAILABLE
newapp-rollout-0902 BlueGreen Healthy - - 3/3 3 3 3
# 查看节点状态
[root@master01 argorollout-examples]# kubectl argo rollouts status newapp-rollout-0902 -ndemo
Healthy
获取滚动更新的信息：
[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo
Name: newapp-rollout-0902
Namespace: demo
Status: ✔ Healthy
Strategy: BlueGreen
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable, active)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ newapp-rollout-0902 Rollout ✔ Healthy 2m33s
└──# revision:1
└──⧉ newapp-rollout-0902-746fbdc85 ReplicaSet ✔ Healthy 2m33s stable,active
├──□ newapp-rollout-0902-746fbdc85-h77np Pod ✔ Running 2m33s ready:1/1
├──□ newapp-rollout-0902-746fbdc85-kvn98 Pod ✔ Running 2m33s ready:1/1
└──□ newapp-rollout-0902-746fbdc85-md5j7 Pod ✔ Running 2m33s ready:1/1
测试验证：均为 v1 版本
# 查看服务
[root@master01 argorollout-examples]# kgs -ndemo | grep bluegreen
bluegreen-appv1 ClusterIP <cluster-ip> <none> 8080/TCP 10m
bluegreen-appv2 ClusterIP <cluster-ip> <none> 8080/TCP 10m
# 查看bluegreen-appv1对应的版本为v1
[root@master01 argorollout-examples]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-746fbdc85-kvn98 | v=1
# 查看bluegreen-appv2对应的版本为v1
[root@master01 argorollout-examples]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-746fbdc85-kvn98 | v=1
更新镜像：
1）更新 git 仓库配置文件，修改镜像版本从 v1 --> v2
2)通过命令行的方式更新：
- 更新镜像
#语法格式
kubectl argo rollouts set image ROLLOUT_NAME CONTAINER=IMAGE [flags]
# 实际替换镜像为v2
[root@master01 argorollout-examples]# kubectl argo rollouts set image newapp-rollout-0902 web=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo
- 动态观察服务切换信息
[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch
Name: newapp-rollout-0902
Namespace: demo
Status: ॥ Paused
Message: BlueGreenPause
Strategy: BlueGreen
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable, active)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (preview)
Replicas:
Desired: 3
Current: 6
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ newapp-rollout-0902 Rollout ॥ Paused 12m
├──# revision:2
│ └──⧉ newapp-rollout-0902-7466b89448 ReplicaSet ✔ Healthy 41s preview
│ ├──□ newapp-rollout-0902-7466b89448-75x4s Pod ✔ Running 41s ready:1/1
│ ├──□ newapp-rollout-0902-7466b89448-8wh8l Pod ✔ Running 41s ready:1/1
│ └──□ newapp-rollout-0902-7466b89448-nn84z Pod ✔ Running 41s ready:1/1
└──# revision:1
└──⧉ newapp-rollout-0902-746fbdc85 ReplicaSet ✔ Healthy 12m stable,active
├──□ newapp-rollout-0902-746fbdc85-h77np Pod ✔ Running 12m ready:1/1
├──□ newapp-rollout-0902-746fbdc85-kvn98 Pod ✔ Running 12m ready:1/1
└──□ newapp-rollout-0902-746fbdc85-md5j7 Pod ✔ Running 12m ready:1/1
- 手动执行版本切换
# 手动进行切换，等待大概10s后，v1版本的pod会被删除
[root@master01 argorollout-examples]# kubectl argo rollouts promote newapp-rollout-0902 -ndemo
# 实时查看，观察到目前只剩下v2版本的pod
[root@master01 argorollout-examples]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch
Name: newapp-rollout-0902
Namespace: demo
Status: ✔ Healthy
Strategy: BlueGreen
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable, active)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ newapp-rollout-0902 Rollout ✔ Healthy 18m
├──# revision:2
│ └──⧉ newapp-rollout-0902-7466b89448 ReplicaSet ✔ Healthy 6m55s stable,active
│ ├──□ newapp-rollout-0902-7466b89448-75x4s Pod ✔ Running 6m55s ready:1/1
│ ├──□ newapp-rollout-0902-7466b89448-8wh8l Pod ✔ Running 6m55s ready:1/1
│ └──□ newapp-rollout-0902-7466b89448-nn84z Pod ✔ Running 6m55s ready:1/1
└──# revision:1
└──⧉ newapp-rollout-0902-746fbdc85 ReplicaSet • ScaledDown 18m
测试验证：均为 v2 版本
# 查看服务
[root@master01 argorollout-examples]# kgs -ndemo | grep blue
bluegreen-appv1 ClusterIP <cluster-ip> <none> 8080/TCP 19m
bluegreen-appv2 ClusterIP <cluster-ip> <none> 8080/TCP 19m
# 测试访问v1
[root@master01 argorollout-examples]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-nn84z | v=2
# 测试访问v2
[root@master01 argorollout-examples]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-nn84z | v=2
浏览器中输入http://argorollouts.example.com/，打开argorollouts Dashboard 界面
如何中断 Rollout ？
在生产过程中，如果新版本部署后测试有问题，如何在更新过程中手动中止 Rollout ？
首先，使用 set image 命令部署一个新的 V1 版本的容器，并等待 rollout 再次达到暂停的步骤。
[root@master01 ~]# kubectl argo rollouts set image newapp-rollout-0902 web=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 -ndemo
查看验证：
[root@master01 ~]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo --watch
中止更新，而不是将滚动切换到下一步，这样它就回到了 V2 版本，该插件同样提供了一个 abort 命令，可以在更新过程中的任何时候手动中止 Rollout 。
[root@master01 ~]# kubectl argo rollouts abort newapp-rollout-0902 -ndemo
当中止滚动时，它将扩大 ReplicaSet 的 stable 版本（在本例中是 V2 版本），并缩小任何其他版本。
尽管 ReplicaSet 的稳定版本可能正在运行，并且是健康的，但整个 Rollout 仍然被认为是退化的，因为期望的版本（ V1 版本）不是实际运行的版本。
[root@master01 ~]# kubectl argo rollouts get rollout newapp-rollout-0902 -ndemo
Name: newapp-rollout-0902
Namespace: demo
Status: ✖ Degraded
Message: RolloutAborted: Rollout aborted update to revision 3
Strategy: BlueGreen
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (preview)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable, active)
Replicas:
Desired: 3
Current: 6
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ newapp-rollout-0902 Rollout ✖ Degraded 32m
├──# revision:3
│ └──⧉ newapp-rollout-0902-746fbdc85 ReplicaSet ✔ Healthy 32m preview,delay:1s
│ ├──□ newapp-rollout-0902-746fbdc85-44hq9 Pod ✔ Running 3m47s ready:1/1
│ ├──□ newapp-rollout-0902-746fbdc85-sz5rv Pod ✔ Running 3m47s ready:1/1
│ └──□ newapp-rollout-0902-746fbdc85-z7cj4 Pod ✔ Running 3m47s ready:1/1
└──# revision:2
└──⧉ newapp-rollout-0902-7466b89448 ReplicaSet ✔ Healthy 21m stable,active
├──□ newapp-rollout-0902-7466b89448-75x4s Pod ✔ Running 21m ready:1/1
├──□ newapp-rollout-0902-7466b89448-8wh8l Pod ✔ Running 21m ready:1/1
└──□ newapp-rollout-0902-7466b89448-nn84z Pod ✔ Running 21m ready:1/1
模拟测试访问,发现实际访问的版本为v2，但是用户实际想访问的是v1
[root@master01 ~]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: newapp-rollout-0902-7466b89448-8wh8l | v=2
[root@master01 ~]# curl <cluster-ip>:8080
curl: (7) Failed connect to <cluster-ip>:8080; Connection refused
四、基于ArgoCD的金丝雀部署¶
4.1 传统金丝雀(灰度)流程¶
4.2 ArgoRollout 金丝雀(灰度)流程¶
实现原理：
金丝雀发布是一种部署新版本应用的方法，其中 新版本 先部署给一小部分用户，如果一切正常，则逐渐增加新版本的流量，直到完全替换旧版本。
4.3 基于 Replica Shifting(版本替换) 金丝雀发布¶
4.3.1 部署 Rollout¶
创建数据目录
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# mkdir canary-replica
在数据目录中创建文件
[root@master01 argorollout-examples]# cd canary-replica/
[root@master01 canary-replica]# vim canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: canary-rollouts-demo
namespace: demo
spec:
replicas: 3 # 定义3个副本
strategy: # 定义升级策略
canary: # 金丝雀发布
steps: # 发布的节奏
- setWeight: 20
- pause: {} # 会一直暂停,需要手动 promote
- setWeight: 40
- pause: { duration: 10 } # 暂停10s
- setWeight: 60
- pause: { duration: 10 }
- setWeight: 80
- pause: { duration: 10 }
revisionHistoryLimit: 2 # 下面部分其实是和 Deployment 兼容的
selector:
matchLabels:
app: rollouts-demo
template:
metadata:
labels:
app: rollouts-demo
spec:
containers:
- name: rollouts-demo
image: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1
ports:
- name: http
containerPort: 8080
protocol: TCP
可以看到除了 apiVersion，kind 以及 strategy 之外，其他和 Deployment 无异。
strategy 字段定义的是发布策略，其中：
-
setWeight：设置流量的权重
-
pause：暂停，如果里面没有跟duration: 10则表示需要手动更新，如果跟了表示等待多长时间会自动更新。
4.3.2 定义 service¶
[root@master01 argorollout-examples]# cd canary-replica/
[root@master01 canary-replica]# vim canary-rollout-svc.yaml
apiVersion: v1
kind: Service
metadata:
name: rollouts-demo
namespace: demo
spec:
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http
selector:
app: rollouts-demo
4.3.3 创建 argorollout-canary-replica¶
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-canary-replica.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argorollout-canary-replica
namespace: argocd
spec:
destination:
name: ''
namespace: demo
server: 'https://kubernetes.default.svc'
source:
path: canary-replica
repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
targetRevision: main
sources: []
project: default
把上述2个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git canary-replica 目录下，后续在 Argo CD 上使用 YAML 资源清单创建服务。
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# git init
# 添加远端仓库
[root@master01 argorollout-examples]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git
# 验证查看
[root@master01 argorollout-examples]# git remote -v
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (push)
# 添加到暂存区
[root@master01 argorollout-examples]# git add .
# 提交到本地仓库
[root@master01 argorollout-examples]# git commit -m "first for argocd-example-apps"
# 切换到main分支
[root@master01 argorollout-examples]# git branch -M main
# 上传到main分支
[root@master01 argorollout-examples]# git push -uf origin main
Username for 'http://gitlab.example.com': root
Password for 'http://root@gitlab.example.com': <gitlab-password>
创建相关资源
[root@master01 argorollout-examples]# kubectl apply -f argorollout-canary-replica.yaml
4.3.4 手动 sync¶
点击【SYNC】-【SYNCHRONIZE】进行手动同步
底层命令观察：
# 查看回滚列表
[root@master01 ~]# kubectl argo rollouts list rollouts -ndemo
NAME STRATEGY STATUS STEP SET-WEIGHT READY DESIRED UP-TO-DATE AVAILABLE
canary-rollouts-demo Canary Healthy 8/8 100 3/3 3 3 3
newapp-rollout-0902 BlueGreen Healthy - - 3/3 3 3 3
# 获取滚动更新的信息
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo
Name: canary-rollouts-demo
Namespace: demo
Status: ✔ Healthy
Strategy: Canary
Step: 8/8
SetWeight: 100
ActualWeight: 100
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-demo Rollout ✔ Healthy 63s
└──# revision:1
└──⧉ canary-rollouts-demo-d7975f98c ReplicaSet ✔ Healthy 63s stable
├──□ canary-rollouts-demo-d7975f98c-sdsb8 Pod ✔ Running 62s ready:1/1
├──□ canary-rollouts-demo-d7975f98c-vlxr5 Pod ✔ Running 62s ready:1/1
└──□ canary-rollouts-demo-d7975f98c-wkwlg Pod ✔ Running 62s ready:1/1
打开浏览器输入http://argorollouts.example.com/打开argorollout dashboard，命名空间选择demo
4.3.5 更新 Rollout¶
先打开一个新的 xshell 窗口，便于动态观察：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo --watch
# 初始页面
Name: canary-rollouts-demo
Namespace: demo
Status: ✔ Healthy
Strategy: Canary
Step: 8/8
SetWeight: 100
ActualWeight: 100
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-demo Rollout ✔ Healthy 5m3s
└──# revision:1
└──⧉ canary-rollouts-demo-d7975f98c ReplicaSet ✔ Healthy 5m3s stable
├──□ canary-rollouts-demo-d7975f98c-sdsb8 Pod ✔ Running 5m2s ready:1/1
├──□ canary-rollouts-demo-d7975f98c-vlxr5 Pod ✔ Running 5m2s ready:1/1
└──□ canary-rollouts-demo-d7975f98c-wkwlg Pod ✔ Running 5m2s ready:1/1
使用 set image 的命令更新：
[root@master01 ~]# kubectl argo rollouts set image canary-rollouts-demo rollouts-demo=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo
动态观察数据：
Name: canary-rollouts-demo
Namespace: demo
Status: ॥ Paused #暂停，等待进一步promote
Message: CanaryPauseStep
Strategy: Canary
Step: 1/8
SetWeight: 20
ActualWeight: 25
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
Desired: 3
Current: 4
Updated: 1
Ready: 4
Available: 4
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-demo Rollout ॥ Paused 5m50s
├──# revision:2
│ └──⧉ canary-rollouts-demo-68bfb5ccbb ReplicaSet ✔ Healthy 15s canary # 版本被标记为canary
│ └──□ canary-rollouts-demo-68bfb5ccbb-zhh74 Pod ✔ Running 15s ready:1/1
└──# revision:1
└──⧉ canary-rollouts-demo-d7975f98c ReplicaSet ✔ Healthy 5m50s stable
├──□ canary-rollouts-demo-d7975f98c-sdsb8 Pod ✔ Running 5m49s ready:1/1
├──□ canary-rollouts-demo-d7975f98c-vlxr5 Pod ✔ Running 5m49s ready:1/1
└──□ canary-rollouts-demo-d7975f98c-wkwlg Pod ✔ Running 5m49s ready:1/1
当 demo rollout 到达第二步时，我们可以从插件中看到，Rollout 处于暂停状态，现在有 4 个副本中的1 个运行新版本的 pod，其余 3 个仍然运行旧版本，这相当于 setWeight: 20 步骤所定义的 20% 的金丝雀权重。
4.3.6 Promote Rollout¶
Rollout 现在处于暂停状态，要手动将 Rollout 切换到下一个步骤，运行插件的 promote 命令。
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-demo -ndemo
然后，每隔 10s 进行 v2 版本的逐步替换，最后全部替换为 v2 版本：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo
Name: canary-rollouts-demo
Namespace: demo
Status: ✔ Healthy
Strategy: Canary
Step: 8/8
SetWeight: 100
ActualWeight: 100
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-demo Rollout ✔ Healthy 9m21s
├──# revision:2
│ └──⧉ canary-rollouts-demo-68bfb5ccbb ReplicaSet ✔ Healthy 3m46s stable
│ ├──□ canary-rollouts-demo-68bfb5ccbb-zhh74 Pod ✔ Running 3m46s ready:1/1
│ ├──□ canary-rollouts-demo-68bfb5ccbb-n956r Pod ✔ Running 33s ready:1/1
│ └──□ canary-rollouts-demo-68bfb5ccbb-mb4fc Pod ✔ Running 22s ready:1/1
└──# revision:1
└──⧉ canary-rollouts-demo-d7975f98c ReplicaSet • ScaledDown 9m21s
└──□ canary-rollouts-demo-d7975f98c-wkwlg Pod ◌ Terminating 9m20s ready:1/1
也可以底层 svc 访问验证：
# 查看svc地址为<cluster-ip>
[root@master01 argorollout-examples]# kgs -ndemo | grep rollouts-demo
rollouts-demo ClusterIP <cluster-ip> <none> 8080/TCP 2d4h
# 测试访问服务，目前版本变为v2
[root@master01 ~]# curl <cluster-ip>:8080
Hello Kubernetes bootcamp! | Running on: canary-rollouts-demo-68bfb5ccbb-mb4fc | v=2
4.4 基于 Traffic Shifting(流量接入) 金丝雀发布¶
官方数据：https://argoproj.github.io/argo-rollouts/features/traffic-management/nginx/
上面我们并没有接入外部流量，仅仅是在内部使用展示了金丝雀部署过程，下面我们接入外部流量进行测试。
Argo-Rollout 主要集成了 Ingress 和 ServiceMesh 两种流量控制方法。
主要 Ingress 支持：（主要使用 Nginx-Ingress）
4.4.1 创建 Rollout¶
Rollout 里用 canaryService 和 stableService 分别定义了该应用：
-
灰度的 Service Name (rollouts-traffic-canary)
-
当前版本的 Service Name (rollouts-traffic-stable)
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# mkdir canary-traffic
[root@master01 argorollout-examples]# cd canary-traffic
[root@master01 canary-traffic]# vim canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: canary-rollouts-traffic-demo
namespace: demo
spec:
replicas: 3
strategy:
canary:
canaryService: rollouts-traffic-canary
stableService: rollouts-traffic-stable
trafficRouting:
nginx:
stableIngress: rollouts-traffic-stable
# 发布的节奏
steps:
- setWeight: 30
# 会一直暂停
- pause: {}
- setWeight: 60
- pause: {}
- setWeight: 100
- pause: {}
revisionHistoryLimit: 2
selector:
matchLabels:
app: rollouts-traffic-demo
template:
metadata:
labels:
app: rollouts-traffic-demo
spec:
containers:
- name: rollouts-traffic-demo
image: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1
ports:
- name: http
containerPort: 8080
protocol: TCP
resources:
requests:
memory: 512Mi
cpu: 512m
4.4.2 创建应用的 stable/canary SVC¶
Service rollouts-traffic-canary 和 rollouts-traffic-stable，二者内容一样。selector 中暂时没有填上pod-template-hash。
Argo-Rollout Controller 会根据实际的 ReplicaSet hash 来修改该值:
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/canary-traffic
[root@master01 canary-traffic]# vim rollouts-traffic-stable-canary.yaml
apiVersion: v1
kind: Service
metadata:
name: rollouts-traffic-canary
namespace: demo
spec:
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http
selector:
app: rollouts-traffic-demo
# This selector will be updated with the pod-template-hash of the canary ReplicaSet. e.g.:
# rollouts-pod-template-hash: 7bf84f9696
---
apiVersion: v1
kind: Service
metadata:
name: rollouts-traffic-stable
namespace: demo
spec:
ports:
- port: 8080
targetPort: http
protocol: TCP
name: http
selector:
app: rollouts-traffic-demo
4.4.3 创建应用的 ingress 路由¶
Ingress 则定义了规则，nginx 将 canary.example.com 域名的请求转发到当前版本的 Service(rollouts-traffic-stable)
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples/canary-traffic
[root@master01 canary-traffic]# vim canary-traffic-ing.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rollouts-traffic-stable
namespace: demo
spec:
ingressClassName: nginx
rules:
- host: canary.example.com
http:
paths:
- backend:
service:
name: rollouts-traffic-stable
port:
number: 8080
path: /
pathType: Prefix
4.4.4 创建 APP¶
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# vim argorollout-canary-traffic.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: argorollout-canary-traffic
namespace: argocd
spec:
destination:
name: ''
namespace: default
server: 'https://kubernetes.default.svc'
source:
path: canary-traffic
repoURL: 'http://gitlab.example.com/demoteam/argocd-example-apps.git'
targetRevision: main
sources: []
project: default
把上述 3 个文件放入代码仓库 http://gitlab.example.com/demoteam/argocd-example-apps.git canary-traffic 目录下，后续在 Argo CD 上使用 YAML 资源清单创建服务。
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
# 初始化
[root@master01 argorollout-examples]# git init
# 添加远端仓库
[root@master01 argocd]# git remote add origin http://gitlab.example.com/demoteam/argocd-example-apps.git
# 验证查看
[root@master01 argorollout-examples]# git remote -v
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (fetch)
origin http://gitlab.example.com/demoteam/argocd-example-apps.git (push)
# 添加到暂存区
[root@master01 argorollout-examples]# git add .
# 提交到本地仓库
[root@master01 argorollout-examples]# git commit -m "second for argorollout-examples"
# 切换到main分支
[root@master01 argorollout-examples]# git branch -M main
# 上传到main分支
[root@master01 argorollout-examples]# git push -uf origin main
Username for 'http://gitlab.example.com': root
Password for 'http://root@gitlab.example.com': <gitlab-password>
...
...
gitlab页面查看
应用创建
[root@master01 ~]# cd /root/17/argo-rollouts/argorollout-examples
[root@master01 argorollout-examples]# kubectl apply -f argorollout-canary-traffic.yaml
WEB UI 手动同步：
点击【SYNC】-【SYNCHRONIZE】
traffic 流向：
4.4.5 Canary Ingress¶
Rollout Controller 会根据 ingress rollouts-traffic-stable 内容，自动创建一个 ingress 用于灰度的流量，名字为 --canary，
所以这里多了一个 ingress canary-rollouts-traffic-demo-rollouts-traffic-stable-canary，将流量导向Canary Service (rollouts-traffic-canary):
[root@master01 ~]# kubectl get ing -ndemo | grep canary
canary-rollouts-traffic-demo-rollouts-traffic-stable-canary nginx canary.example.com <node-ip> 80 8m36s
rollouts-traffic-stable nginx canary.example.com <node-ip> 80 8m39s
4.4.6 触发更新¶
[root@master01 ~]# kubectl argo rollouts set image canary-rollouts-traffic-demo rollouts-traffic-demo=registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 -ndemo
动态观察，第一步：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显内容
Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 1/6
SetWeight: 30 #权重比例
ActualWeight: 30 #权重比例
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
Desired: 3
Current: 4
Updated: 1
Ready: 4
Available: 4
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-traffic-demo Rollout ॥ Paused 12m
├──# revision:2
│ └──⧉ canary-rollouts-traffic-demo-69f4dd6c8 ReplicaSet ✔ Healthy 2m27s canary
│ └──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd Pod ✔ Running 2m27s ready:1/1
└──# revision:1
└──⧉ canary-rollouts-traffic-demo-5c8c98d57f ReplicaSet ✔ Healthy 12m stable
├──□ canary-rollouts-traffic-demo-5c8c98d57f-6fwcj Pod ✔ Running 12m ready:1/1
├──□ canary-rollouts-traffic-demo-5c8c98d57f-jbqcv Pod ✔ Running 12m ready:1/1
└──□ canary-rollouts-traffic-demo-5c8c98d57f-mrqmx Pod ✔ Running 12m ready:1/1
curl 命令测试：
# 目前v2的权重占30%
[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-mrqmx | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-6fwcj | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-mrqmx | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-jbqcv | v=1
因为配置文件中 pause: {} ，需要在命令行手动执行灰度发布，命令执行完成后就能达到 60% 灰度
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
再次查看 滚动更新状态：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显
Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 3/6
SetWeight: 60 #权重比例
ActualWeight: 60 #权重比例
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
Desired: 3
Current: 5
Updated: 2
Ready: 5
Available: 5
查看 canary ingress 资源：
[root@master01 ~]# kgi -ndemo canary-rollouts-traffic-demo-rollouts-traffic-stable-canary -oyaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/canary: "true"
nginx.ingress.kubernetes.io/canary-weight: "60"
creationTimestamp: "2025-04-24T12:01:59Z"
generation: 1
name: canary-rollouts-traffic-demo-rollouts-traffic-stable-canary
namespace: demo
ownerReferences:
- apiVersion: argoproj.io/v1alpha1
blockOwnerDeletion: true
controller: true
kind: Rollout
name: canary-rollouts-traffic-demo
uid: a40a95e0-617a-4f40-b7ed-79f07e17cefa
resourceVersion: "2571984"
uid: 465ee9e2-71d1-45cb-92ad-ad1f5e7ffc27
spec:
ingressClassName: nginx
rules:
- host: canary.example.com
http:
paths:
- backend:
service:
name: rollouts-traffic-canary
port:
number: 8080
path: /
pathType: Prefix
status:
loadBalancer:
ingress:
- ip: <node-ip>
继续执行命令完成 100% 的灰度流程：
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
再次查看滚动更新状态：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-traffic-demo -ndemo --watch
# 回显
Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 5/6
SetWeight: 100
ActualWeight: 100
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
Desired: 3
Current: 6
Updated: 3
Ready: 6
Available: 6
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-traffic-demo Rollout ॥ Paused 16m
├──# revision:2
│ └──⧉ canary-rollouts-traffic-demo-69f4dd6c8 ReplicaSet ✔ Healthy 6m29s canary
│ ├──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd Pod ✔ Running 6m29s ready:1/1
│ ├──□ canary-rollouts-traffic-demo-69f4dd6c8-cw95h Pod ✔ Running 3m11s ready:1/1
│ └──□ canary-rollouts-traffic-demo-69f4dd6c8-npwvj Pod ✔ Running 62s ready:1/1
└──# revision:1
└──⧉ canary-rollouts-traffic-demo-5c8c98d57f ReplicaSet ✔ Healthy 16m stable
├──□ canary-rollouts-traffic-demo-5c8c98d57f-6fwcj Pod ✔ Running 16m ready:1/1
├──□ canary-rollouts-traffic-demo-5c8c98d57f-jbqcv Pod ✔ Running 16m ready:1/1
└──□ canary-rollouts-traffic-demo-5c8c98d57f-mrqmx Pod ✔ Running 16m ready:1/1
打开浏览器输入http://argorollouts.example.com/查看 argorollouts dashboard：
浏览器输入https://argocd.example.com/打开argocd dashboard
完成金丝雀发布及销毁 v1 版本：
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
等待一小段时间，再次查看底层数据，观察到v1版本的pod删除，目前只剩下v2版本的pod
Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ✔ Healthy
Strategy: Canary
Step: 6/6
SetWeight: 100
ActualWeight: 100
Images: registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (stable)
Replicas:
Desired: 3
Current: 3
Updated: 3
Ready: 3
Available: 3
NAME KIND STATUS AGE INFO
⟳ canary-rollouts-traffic-demo Rollout ✔ Healthy 19m
├──# revision:2
│ └──⧉ canary-rollouts-traffic-demo-69f4dd6c8 ReplicaSet ✔ Healthy 9m44s stable
│ ├──□ canary-rollouts-traffic-demo-69f4dd6c8-f5bqd Pod ✔ Running 9m44s ready:1/1
│ ├──□ canary-rollouts-traffic-demo-69f4dd6c8-cw95h Pod ✔ Running 6m26s ready:1/1
│ └──□ canary-rollouts-traffic-demo-69f4dd6c8-npwvj Pod ✔ Running 4m17s ready:1/1
└──# revision:1
└──⧉ canary-rollouts-traffic-demo-5c8c98d57f ReplicaSet • ScaledDown 19m
curl 数据验证：
# 因为版本切换为v2，所以版本全为v2
[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-f5bqd | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-cw95h | v=2
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-69f4dd6c8-npwvj | v=2
五、回退应用¶
5.1 命令行回退¶
有时候在应用上线过后，有些BUG并没有发现，这时候要回退怎么办呢？argo rollouts 有一个 undo 命令，可以进行回退。
比如我们要将版本回退到第一个版本，则执行一下命令：
[root@master01 ~]# kubectl argo rollouts undo canary-rollouts-demo -ndemo --to-revision=1
通过 watch 界面可以看到如下信息：
[root@master01 ~]# kubectl argo rollouts get rollout canary-rollouts-demo -ndemo
再次 promote 执行更多的 canary 过程：
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-demo -ndemo
最终数据：
Name: canary-rollouts-traffic-demo
Namespace: demo
Status: ॥ Paused
Message: CanaryPauseStep
Strategy: Canary
Step: 5/6
SetWeight: 100
ActualWeight: 100
Images:
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v1 (stable)
registry.cn-hangzhou.aliyuncs.com/zhdya/kubernetes-bootcamp:v2 (canary)
Replicas:
Desired: 3
Current: 6
Updated: 3
Ready: 6
Available: 6
5.2 argorollouts界面回退¶
浏览器中输入http://argorollouts.example.com/打开argorollouts界面后，点击【Rollback】
依次执行promote直至完全回退到上个版本
# 第一次回退，此时权重比例变为60%
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
# 第二次回退，此时权重比例变为100%
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
# 第三次回退，删除现有版本的pod，完全回滚到上一个版本
[root@master01 ~]# kubectl argo rollouts promote canary-rollouts-traffic-demo -ndemo
# 等待一小段时间进行测试访问，目前全部为v1,回退成功
[root@master01 ~]# for i in {1..10}; do curl canary.example.com; done
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-bxz2c | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-bxz2c | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-95snl | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1
Hello Kubernetes bootcamp! | Running on: canary-rollouts-traffic-demo-5c8c98d57f-8f9q2 | v=1
六、总结¶
Argo Rollout 可以认为是 Deployment 的扩展，增加了蓝绿发布和金丝雀发布策略配置，并且支持通过 自动测试实现服务发布或者回滚。此外，Argo Rollout 还提供了更精细的流量管理和监控能力，开发者 能够在部署过程中实施健康检查、A/B测试以及性能评估，从而确保新版本的稳定性和可靠性。