利用crd进行资源编排

1. 需求说明

希望在云管平台中去展示集群中的组件的信息和组件之间的关系，而不需总是手动的进入进群敲命令来获得数据。这些数据都是实时跟新的，而不需手工维护。

希望这些组件的信息和组件之间的关系是可以在云管平台动态编辑的，而不需修改后段代码。

希望可以批量的对组件进行一些常用的操作，比如重启，更新镜像和连接pod。

考虑到是在k8s环境中进行组件管理因此我们自然的想到可以利用k8s自带的功能扩展方式crd进行开发。

2. 实现效果

2.1 组件列表

在组件列表中可展示组件实时的资源用量和镜像版本等信息。同时可以批量的对组件进行一些常用的操作。

点击某一个组件则会展示出该组件的pod信息。

2.2 组件拓扑图

组件拓扑图表示了组件之间的调用关系。（这里只是示例）

3. 原理解析

3.1 创建拓扑cr的流程图

3.2 创建组件操作cr的流程图

4. 资源定义

4.1 定义组件crd

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.10.0
  creationTimestamp: null
  name: topologies.ecstack.ctcdn.cn
spec:
  group: ecstack.ctcdn.cn
  names:
    kind: Topology
    listKind: TopologyList
    plural: topologies
    singular: topology
  scope: Namespaced
  versions:
  - name: v1
    schema:
      openAPIV3Schema:
        description: Topology is the Schema for the topologies API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TopologySpec defines the desired state of Topology
            properties:
              chsName:
                type: string
              links:
                items:
                  properties:
                    dest:
                      type: string
                    src:
                      type: string
                  required:
                  - dest
                  - src
                  type: object
                type: array
              msg:
                type: string
              nodes:
                description: Foo is an example field of Topology. Edit topology_types.go
                  to remove/update Foo     string `json:"foo,omitempty"`
                items:
                  properties:
                    kind:
                      type: string
                    msg:
                      type: string
                    name:
                      type: string
                    namespace:
                      type: string
                    property:
                      items:
                        description: 'EDIT THIS FILE!  THIS IS SCAFFOLDING FOR YOU
                          TO OWN! NOTE: json tags are required.  Any new fields you
                          add must have json tags for the fields to be serialized.'
                        properties:
                          type:
                            type: string
                          value:
                            format: byte
                            type: string
                        required:
                        - type
                        - value
                        type: object
                      type: array
                    resource:
                      type: string
                  required:
                  - kind
                  - msg
                  - name
                  - namespace
                  - resource
                  type: object
                type: array
              version:
                type: string
            required:
            - chsName
            - links
            - msg
            - nodes
            - version
            type: object
          status:
            description: TopologyStatus defines the observed state of Topology
            properties:
              phase:
                description: 'INSERT ADDITIONAL STATUS FIELD - define observed state
                  of cluster Important: Run "make" to regenerate code after modifying
                  this file'
                type: string
            required:
            - phase
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}

4.2 组件cr示例

下图是实现效果中的组件的cr。

主要要关注两个字段：links和nodes

Nodes: 表示了要去展示哪些组件的信息。

Links：表示这些组件之间的调用关系（这里只是示例）

apiVersion: ecstack.ctcdn.cn/v1
kind: Topology
metadata:
  annotations:
  generation: 9
  labels:
    app.kubernetes.io/created-by: kube-stat
    app.kubernetes.io/instance: topology-sample
    app.kubernetes.io/managed-by: kustomize
    app.kubernetes.io/name: topology
    app.kubernetes.io/part-of: kube-stat
spec:
  chsName: topology-demo
  links:
  - dest: patch-demo
    src: nginx-dep
  - dest: patch-demo
    src: nginx-ds
  - dest: kube-multus-ds-amd64
    src: patch-demo
  msg: 测试列表
  nodes:
  - kind: deployment
    msg: 一个测试用的nginx-deployment
    name: nginx-dep
    namespace: n10012342
    resource: nginx-dep
  - kind: deployment
    msg: 测试多容器pod
    name: patch-demo
    namespace: n10012342
    resource: patch-demo
  - kind: daemonset
    msg: 测试daemonset
    name: nginx-ds
    namespace: n10012342
    resource: nginx-ds
  - kind: daemonset
    msg: 测试daemonset
    name: kube-multus-ds-amd64
    namespace: kube-system
    resource: kube-multus-ds-amd64
  version: v0.0.0

4.3 定义组件操作crd

---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  annotations:
    controller-gen.kubebuilder.io/version: v0.10.0
  creationTimestamp: null
  name: topologyoperations.ecstack.ctcdn.cn
spec:
  group: ecstack.ctcdn.cn
  names:
    kind: TopologyOperation
    listKind: TopologyOperationList
    plural: topologyoperations
    singular: topologyoperation
  scope: Namespaced
  versions:
  - additionalPrinterColumns:
    - jsonPath: .spec.topologyName
      name: TopologyName
      type: string
    - jsonPath: .spec.operation.operationType
      name: OpType
      type: string
    - jsonPath: .status.jobStatus
      name: JobStatus
      type: string
    name: v1
    schema:
      openAPIV3Schema:
        description: TopologyOperation is the Schema for the topologyoperations API
        properties:
          apiVersion:
            description: 'APIVersion defines the versioned schema of this representation
              of an object. Servers should convert recognized schemas to the latest
              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
            type: string
          kind:
            description: 'Kind is a string value representing the REST resource this
              object represents. Servers may infer this from the endpoint the client
              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
            type: string
          metadata:
            type: object
          spec:
            description: TopologyOperationSpec defines the desired state of TopologyOperation
            properties:
              nodes:
                items:
                  properties:
                    imageInfos:
                      items:
                        properties:
                          image:
                            type: string
                          name:
                            type: string
                        required:
                        - image
                        - name
                        type: object
                      type: array
                    name:
                      type: string
                  required:
                  - imageInfos
                  - name
                  type: object
                type: array
              operation:
                properties:
                  operationType:
                    type: string
                required:
                - operationType
                type: object
              topologyName:
                description: Foo is an example field of TopologyOperation. Edit topologyoperation_types.go
                  to remove/update
                type: string
            required:
            - nodes
            - operation
            - topologyName
            type: object
          status:
            description: TopologyOperationStatus defines the observed state of TopologyOperation
            properties:
              currentNode:
                description: 'INSERT ADDITIONAL STATUS FIELD - define observed state
                  of cluster Important: Run "make" to regenerate code after modifying
                  this file'
                items:
                  properties:
                    name:
                      type: string
                    pod:
                      items:
                        type: string
                      type: array
                  required:
                  - name
                  type: object
                type: array
              doneNode:
                items:
                  type: string
                type: array
              jobMessage:
                type: string
              jobStatus:
                type: string
              nextNode:
                items:
                  type: string
                type: array
            type: object
        type: object
    served: true
    storage: true
    subresources:
      status: {}
​

4.4 组件操作cr示例

apiVersion: ecstack.ctcdn.cn/v1
kind: TopologyOperation
metadata:
  creationTimestamp: "2023-06-15T06:36:44Z"
  generation: 1
  name: images-1686811004893
  namespace: n10012342
  resourceVersion: "404466476"
  selfLink: /apis/ecstack.ctcdn.cn/v1/namespaces/n10012342/topologyoperations/images-1686811004893
  uid: e995304a-1c0b-4c3b-b9a9-65dd21aec73f
spec:
  nodes:
  - imageInfos:
    - image: nginx:alpine
      name: nginx-instance
    name: nginx-ds
  operation:
    operationType: updateImage
  topologyName: topology-test
status:
  currentNode:
  - name: nginx-ds
  doneNode:
  - nginx-ds
  jobStatus: Success