Kubernetes垂直扩缩容详解

前文已经介绍过，Kubernetes安装默认是带有HorizontalPodAutoscaler水平扩缩容组件的。然而垂直扩缩容组件以及集群扩缩容组件却是需要额外安装的。

安装

由于VerticalPodAutoscaler组件默认并没有安装，因此首先是安装

安装之前先clone下来autoscaler仓库的代码 

1、安装CRD

]# sudo kubectl apply -f  vpa-rbac.yaml 
clusterrole.rbac.authorization.k8s.io/system:metrics-reader unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-actor unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-status-actor unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-checkpoint-actor unchanged
clusterrole.rbac.authorization.k8s.io/system:evictioner unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:metrics-reader unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-actor unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-status-actor unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-checkpoint-actor unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-target-reader unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-target-reader-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-evictioner-binding unchanged
serviceaccount/vpa-admission-controller unchanged
serviceaccount/vpa-recommender unchanged
serviceaccount/vpa-updater unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-admission-controller unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-admission-controller unchanged
clusterrole.rbac.authorization.k8s.io/system:vpa-status-reader unchanged
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-status-reader-binding unchanged
]# sudo kubectl apply -f  vpa-v1-crd-gen.yaml
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalercheckpoints.autoscaling.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/verticalpodautoscalers.autoscaling.k8s.io configured

安装完成后，可以通过get api-resources去查看：

]# kubectl  api-resources | grep VerticalPodAutoscaler
verticalpodautoscalercheckpoints             vpacheckpoint                                   autoscaling.k8s.io/v1                  true         VerticalPodAutoscalerCheckpoint
verticalpodautoscalers                       vpa                                             autoscaling.k8s.io/v1                  true         VerticalPodAutoscaler

启动VPA相关的服务：

进入仓库目录，执行如下命令
./vertical-pod-autoscaler/hack/vpa-up.sh

如果要下线，则执行：

./vertical-pod-autoscaler/hack/vpa-down.sh

如果要看执行的yaml

./vertical-pod-autoscaler/hack/vpa-process-yamls.sh print

启动过程：

]# ./hack/vpa-up.sh                     
clusterrolebinding.rbac.authorization.k8s.io/system:vpa-evictionter-binding created
serviceaccount/vpa-admission-controller created
serviceaccount/vpa-updater created
deployment.apps/vpa-updater created
serviceaccount/vpa-recommender created
deployment.apps/vpa-recommender created
Generating certs for the VPA Admission Controller in /tmp/vpa-certs.
Generating RSA private key, 2048 bit long modulus
..+++
..............................................................................................+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
.............................+++
..........+++
e is 65537 (0x10001)
Signature ok
subject=/CN=vpa-webhook.kube-system.svc
Getting CA Private Key
Uploading certs to the cluster.
secret/vpa-tls-certs created
Deleting /tmp/vpa-certs.
deployment.apps/vpa-admission-controller created
service/vpa-webhook created

安装完成后，查看相关服务状态：

]# kubectl -n kube-system get po  | grep vpa
vpa-admission-controller-6dcb8db46c-rfds5   1/1     Running   0                 14m
vpa-recommender-6498f7d6f4-h7nfx            1/1     Running   0                 15m
vpa-updater-84b67fdff4-4bgjn                1/1     Running   0                 24m

接下来我们就可以进行vpa的演示。

VPA示例

1. 首先需要创建一个VerticalPodAutoscaler，用于规定达到多大资源时进行扩容和缩容以及最大能扩容的资源限制。

本示例使用的创建的VerticalPodAutoscaler yaml文件如下所示：

---
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: vpa-demo
  namespace: demo
spec:
  targetRef:
    apiVersion: "apps/v1"
    kind: Deployment
    name: vpa-demo
  updatePolicy:
    updateMode: "Auto"
  resourcePolicy:
    containerPolicies:
      - containerName: '*'
        minAllowed:
          cpu: 10m
          memory: 2Mi
        maxAllowed:
          cpu: 100m
          memory: 100Mi
        controlledResources: ["cpu", "memory"]

targetRef：表示本VerticalPodAutoscaler要生效的资源，同一个命名空间下的名字为vpa-demo的Deployment

updatePolicy：设定了触发资源限制时的更新策略，取值有：Off、Initial、Recreate、Auto；Off表示不会根据推荐值去主动更新，Initial表示只有在下次创建的时候去更改resource的requests；Recreate则是直接触发pod重新创建；Auto：则会根据K8S集群特性自动选择，如果集群支持原地更新requests，则直接更新，如果不支持则和Recreate一样，进行重建pod。

resourcePolicy：设定了触发后的资源上下限，下面containerPolicies可以根据容器进行设置。

本示例创建的vpa-demo的Deployment的yaml如下所示：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vpa-demo
  namespace: demo
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: vpa-demo
      app.kubernetes.io/name: vpa-demo
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app.kubernetes.io/instance: vpa-demo
        app.kubernetes.io/name: vpa-demo
    spec:
      containers:
      - image: mirror.ccs.tencentyun.com/library/nginx:stable-otel
        imagePullPolicy: IfNotPresent
        name: demo
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /
            port: http
            scheme: HTTP
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 10m
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/hostname: master2
      restartPolicy: Always

namespace是demo，资源类型是Deployment，名字是vpa-demo，启动服务

]# kubectl -n demo get po
NAME                        READY   STATUS    RESTARTS   AGE
vpa-demo-5658d9d7fd-b964t   1/1     Running   0          5s
vpa-demo-5658d9d7fd-rzqf4   1/1     Running   0          73s

启动压测：

]# wrk -c 5 -t 5 -d 360  xx.xx.xx.xx
Running 6m test @ xx.xx.xx.xx
  5 threads and 5 connections

如果updateMode是Off，vpa会显示recommendation，给出设置资源的推荐值。

  conditions:
  - lastTransitionTime: "2024-08-30T08:57:09Z"
    status: "True"
    type: RecommendationProvided
  recommendation:
    containerRecommendations:
    - containerName: demo
      lowerBound:
        cpu: 25m
        memory: 100Mi
      target:
        cpu: 100m
        memory: 100Mi
      uncappedTarget:
        cpu: 511m
        memory: 262144k
      upperBound:
        cpu: 100m
        memory: 100Mi

如果updateMode是Auto，创建新的Pod（本集群下Auto等同于Recreate）：

]# kubectl -n demo get po -w
NAME                        READY   STATUS    RESTARTS   AGE
vpa-demo-5658d9d7fd-4brzh   1/1     Running   0          47s
vpa-demo-5658d9d7fd-jkssc   1/1     Running   0          51s
vpa-demo-5658d9d7fd-jkssc   1/1     Terminating   0          64s
vpa-demo-5658d9d7fd-bdp2k   0/1     Pending       0          0s
vpa-demo-5658d9d7fd-bdp2k   0/1     Pending       0          0s
vpa-demo-5658d9d7fd-bdp2k   0/1     ContainerCreating   0          0s
vpa-demo-5658d9d7fd-jkssc   1/1     Terminating         0          65s
vpa-demo-5658d9d7fd-jkssc   0/1     Terminating         0          68s
vpa-demo-5658d9d7fd-jkssc   0/1     Terminating         0          68s
vpa-demo-5658d9d7fd-jkssc   0/1     Terminating         0          68s
vpa-demo-5658d9d7fd-bdp2k   0/1     ContainerCreating   0          4s
vpa-demo-5658d9d7fd-bdp2k   0/1     Running             0          5s
vpa-demo-5658d9d7fd-bdp2k   1/1     Running             0          6s

总结

本文针对Kubernetes的扩缩容概念进行了简单的介绍，随后又通过实践的方式对VerticalPodAutoscaler组件进行了安装，后续又通过示例对垂直扩容进行了操作实践。