应用场景
CSI对接两个HBlock,集群版和单机版。
- clusterID为stor1,对应集群版。clusterID为stor2,对应单机版。
- 集群版的服务器IP和API为:192.168.0.192:1443、192.168.0.110:1443、192.168.0.102:1443。单机版的服务器IP和API端口为192.168.0.32:1443。
- 集群版的用户名和密码为:storuser、hblock12@。单机版的用户名和密码为:storuser、hblock12@。userKey源码可以参考配置HBlock访问用户名和密码示例中的步骤1、2。
- 集群版和单机版均不使用CHAP认证,如若使用,可以参考配置加密模式 。
- 启用样例blockVolumes的动态PV样例(dynamicPv),验证HBlock CSI是否已经可以正常启用。
操作步骤
修改配置文件,并应用配置文件
-
修改charts/csi-driver-stor/values.yaml配置文件,配置HBlock访问地址、访问用户名和密码、加密模式。
images: csiProvisioner: registry.aliyuncs.com/google_containers/csi-provisioner:v3.5.0 csiAttacher: registry.aliyuncs.com/google_containers/csi-attacher:v4.3.0 csiResizer: registry.aliyuncs.com/google_containers/csi-resizer:v1.8.0 csiDriverRegistrar: registry.aliyuncs.com/google_containers/csi-node-driver-registrar:v2.8.0 csiStorPlugin: stor-csi-driver:1.3.0 # stor 集群信息 storConfig: # ClusterId And EndPointList configJson: |- [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ] userKey: WwogICAgICB7CiAgICAgICAgImNsdXN0ZXJJRCI6ICJzdG9yMSIsICAKICAgICAgICAidXNlcm5hbWUiOiAic3RvcnVzZXIiLAogICAgICAgICJwYXNzd29yZCI6ICJoYmxvY2sxMkAiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiY2x1c3RlcklEIjogInN0b3IyIiwgIAogICAgICAgICJ1c2VybmFtZSI6ICJzdG9ydXNlciIsCiAgICAgICAgInBhc3N3b3JkIjogImhibG9jazEyQCIKICAgICAgfQogICAgXQo= chap: # Whether to enable chap encryption. true(dHJ1ZQ==), false(ZmFsc2U=) decryptFlag: ZmFsc2U= # Encrypt the key with base64 decryptData: c3RvcjAxMjM0NTY3ODkwMQ== # Samples of pod, pvc and storageclass example: # Mode of block blockVolumes: # Sample of dynamic pv. Path is templates/examples/block-volumes/dynamic-pv dynamicPv: enable: true clusterId: stor1 clusterMode: true # Sample of statefulset with dynamic pv. Path is templates/examples/block-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/block-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun05 # Mode of filesystem filesystemVolumes: dynamicPv: # Sample of dynamic pv. Path is templates/examples/filesystem-volumes/dynamic-pv/local local: enable: false clusterId: cluster1 clusterMode: true # Sample of dynamic pv with chap disabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap localChap: enable: false clusterId: cluster1 clusterMode: true chapUser: user chapPassword: skskdndD0dkfL # Sample of dynamic pv with chap enabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap-encrypt localChapDecrypt: enable: false clusterId: cluster1 clusterMode: true chapUser: SFFFzADcpZaUJKsYbPq54A== chapPassword: tbHWrBep3R0RhkFaW+f5Fw== # Sample of statefulset with dynamic pv. Path is templates/examples/filesystem-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/filesystem-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun04
-
应用配置文件,在charts/csi-driver-stor下执行更新配置命令。
[root@server csi-driver-stor]# helm upgrade stor ./ Release "stor" has been upgraded. Happy Helming! NAME: stor LAST DEPLOYED: Mon May 13 19:20:22 2024 NAMESPACE: default STATUS: deployed REVISION: 13 TEST SUITE: None NOTES: The Stor CSI Plugin has been published. The plugin image is: stor-csi-driver:1.3.0 The Stor Cluster Info is: [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ]
使用样例验证HBlock配置是否成功(可选)
说明此处HBlock CSI中配置了2个HBlock,则需要验证2个HBlock的样例。
验证clusterID为stor1的HBlock(集群版)
-
启用样例。
(1) 修改配置文件charts/csi-driver-stor/values.yaml中example的开关,打开样例blockVolumes中的动态PV样例,clusterID为stor1,clusterMode为true。images: # k8s官方维护镜像 csiProvisioner: registry.aliyuncs.com/google_containers/csi-provisioner:v3.5.0 csiAttacher: registry.aliyuncs.com/google_containers/csi-attacher:v4.3.0 csiResizer: registry.aliyuncs.com/google_containers/csi-resizer:v1.8.0 csiDriverRegistrar: registry.aliyuncs.com/google_containers/csi-node-driver-registrar:v2.8.0 # CSI 插件镜像 csiStorPlugin: stor-csi-driver:1.3.0 …… # Samples of pod, pvc and storageclass example: # Mode of block blockVolumes: # Sample of dynamic pv. Path is templates/examples/block-volumes/dynamic-pv dynamicPv: enable: true clusterId: stor1 clusterMode: true # Sample of statefulset with dynamic pv. Path is templates/examples/block-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/block-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun05 # Mode of filesystem filesystemVolumes: dynamicPv: # Sample of dynamic pv. Path is templates/examples/filesystem-volumes/dynamic-pv/local local: enable: false clusterId: cluster1 clusterMode: true # Sample of dynamic pv with chap disabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap localChap: enable: false clusterId: cluster1 clusterMode: true chapUser: user chapPassword: skskdndD0dkfL # Sample of dynamic pv with chap enabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap-encrypt localChapDecrypt: enable: false clusterId: cluster1 clusterMode: true chapUser: SFFFzADcpZaUJKsYbPq54A== chapPassword: tbHWrBep3R0RhkFaW+f5Fw== # Sample of statefulset with dynamic pv. Path is templates/examples/filesystem-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/filesystem-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun04
(2) 应用配置文件,在charts/csi-driver-stor下执行更新配置命令。
[root@server csi-driver-stor]# helm upgrade stor ./ Release "stor" has been upgraded. Happy Helming! NAME: stor LAST DEPLOYED: Tue May 14 08:45:46 2024 NAMESPACE: default STATUS: deployed REVISION: 14 TEST SUITE: None NOTES: The Stor CSI Plugin has been published. The plugin image is: stor-csi-driver:1.3.0 The Stor Cluster Info is: [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ]
(3) 验证HBlock CSI是否已经可以正常启用:如果启用的样例成功建立pod,则表示HBlock CSI可以正常启用。
[root@pm-001 csi-driver-stor]# kubectl get pod NAME READY STATUS RESTARTS AGE csi-storplugin-controller-6c789569b9-mfh2q 3/3 Running 0 13d csi-storplugin-node-fdqq7 2/2 Running 6 (5d13h ago) 13d csi-storplugin-node-tj7rf 2/2 Running 0 13d my-csi-app-block-dynamic 1/1 Running 0 6m41s
-
停用刚启动的样例:如果启用了样例,为了避免与正式的实例冲突,建议停用样例。
(1)在values.yaml中,将dynamicPv的开关设置为false。
images: csiProvisioner: registry.aliyuncs.com/google_containers/csi-provisioner:v3.5.0 csiAttacher: registry.aliyuncs.com/google_containers/csi-attacher:v4.3.0 csiResizer: registry.aliyuncs.com/google_containers/csi-resizer:v1.8.0 csiDriverRegistrar: registry.aliyuncs.com/google_containers/csi-node-driver-registrar:v2.8.0 csiStorPlugin: stor-csi-driver:1.3.0 …… # Samples of pod, pvc and storageclass example: # Mode of block blockVolumes: # Sample of dynamic pv. Path is templates/examples/block-volumes/dynamic-pv dynamicPv: enable: false clusterId: stor1 clusterMode: true # Sample of statefulset with dynamic pv. Path is templates/examples/block-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/block-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun05 # Mode of filesystem filesystemVolumes: dynamicPv: # Sample of dynamic pv. Path is templates/examples/filesystem-volumes/dynamic-pv/local local: enable: false clusterId: cluster1 clusterMode: true # Sample of dynamic pv with chap disabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap localChap: enable: false clusterId: cluster1 clusterMode: true chapUser: user chapPassword: skskdndD0dkfL # Sample of dynamic pv with chap enabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap-encrypt localChapDecrypt: enable: false clusterId: cluster1 clusterMode: true chapUser: SFFFzADcpZaUJKsYbPq54A== chapPassword: tbHWrBep3R0RhkFaW+f5Fw== # Sample of statefulset with dynamic pv. Path is templates/examples/filesystem-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/filesystem-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun04
(2)执行命令helm upgrade stor ./更新配置。
[root@server csi-driver-stor]# helm upgrade stor ./ Release "stor" has been upgraded. Happy Helming! NAME: stor LAST DEPLOYED: Tue May 14 09:09:05 2024 NAMESPACE: default STATUS: deployed REVISION: 15 TEST SUITE: None NOTES: The Stor CSI Plugin has been published. The plugin image is: stor-csi-driver:1.3.0 The Stor Cluster Info is: [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ]
(3)执行命令kubectl get pod检查样例是否停用,若刚创建的pod不存在,即说明样例已停用。
[root@server csi-driver-stor]# kubectl get pod NAME READY STATUS RESTARTS AGE csi-storplugin-controller-6c789569b9-mfh2q 3/3 Running 0 14d csi-storplugin-node-fdqq7 2/2 Running 6 (5d14h ago) 14d csi-storplugin-node-tj7rf 2/2 Running 0 14d
验证clusterID为stor2的HBlock(单机版)
-
启用样例。
(1)修改配置文件charts/csi-driver-stor/values.yaml中example的开关,将clusterMode的取值修改为false,clusterId为stor2,打开样例filesystem中的动态PV开关。…… # Samples of pod, pvc and storageclass example: # Mode of block blockVolumes: # Sample of dynamic pv. Path is templates/examples/block-volumes/dynamic-pv dynamicPv: enable: false clusterId: stor1 clusterMode: true # Sample of statefulset with dynamic pv. Path is templates/examples/block-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/block-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun05 # Mode of filesystem filesystemVolumes: dynamicPv: # Sample of dynamic pv. Path is templates/examples/filesystem-volumes/dynamic-pv/local local: enable: true clusterId: stor2 clusterMode: false # Sample of dynamic pv with chap disabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap localChap: enable: false clusterId: cluster1 clusterMode: true chapUser: user chapPassword: skskdndD0dkfL # Sample of dynamic pv with chap enabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap-encrypt localChapDecrypt: enable: false clusterId: cluster1 clusterMode: true chapUser: SFFFzADcpZaUJKsYbPq54A== chapPassword: tbHWrBep3R0RhkFaW+f5Fw== # Sample of statefulset with dynamic pv. Path is templates/examples/filesystem-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/filesystem-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun04
(2)应用配置文件,在charts/csi-driver-stor下执行更新配置命令。
[root@server csi-driver-stor]# helm upgrade stor ./ Release "stor" has been upgraded. Happy Helming! NAME: stor LAST DEPLOYED: Tue May 14 10:20:50 2024 NAMESPACE: default STATUS: deployed REVISION: 22 TEST SUITE: None NOTES: The Stor CSI Plugin has been published. The plugin image is: stor-csi-driver:1.3.0 The Stor Cluster Info is: [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ]
(3)验证HBlock CSI是否已经可以正常启用:如果启用的样例成功建立pod,则表示HBlock CSI可以正常启用。
[root@pm-001 csi-driver-stor]# kubectl get pod NAME READY STATUS RESTARTS AGE csi-storplugin-controller-6c789569b9-mfh2q 3/3 Running 0 14d csi-storplugin-node-fdqq7 2/2 Running 6 (5d15h ago) 14d csi-storplugin-node-tj7rf 2/2 Running 0 14d my-csi-app-filesystem-dynamic-local 1/1 Running 0 4m14s
-
停用刚启动的样例:如果启用了样例,为了避免与正式的实例冲突,建议停用样例。
(1)在values.yaml中,将clusterMode的取值修改为true,关闭样例filesystem中的动态PV开关。
images: csiProvisioner: registry.aliyuncs.com/google_containers/csi-provisioner:v3.5.0 csiAttacher: registry.aliyuncs.com/google_containers/csi-attacher:v4.3.0 csiResizer: registry.aliyuncs.com/google_containers/csi-resizer:v1.8.0 csiDriverRegistrar: registry.aliyuncs.com/google_containers/csi-node-driver-registrar:v2.8.0 csiStorPlugin: stor-csi-driver:1.3.0 …… # Samples of pod, pvc and storageclass example: # Mode of block blockVolumes: # Sample of dynamic pv. Path is templates/examples/block-volumes/dynamic-pv dynamicPv: enable: false clusterId: stor1 clusterMode: true # Sample of statefulset with dynamic pv. Path is templates/examples/block-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/block-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun05 # Mode of filesystem filesystemVolumes: dynamicPv: # Sample of dynamic pv. Path is templates/examples/filesystem-volumes/dynamic-pv/local local: enable: false clusterId: stor2 clusterMode: false # Sample of dynamic pv with chap disabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap localChap: enable: false clusterId: cluster1 clusterMode: true chapUser: user chapPassword: skskdndD0dkfL # Sample of dynamic pv with chap enabled. Path is templates/examples/filesystem-volumes/dynamic-pv/local-chap-encrypt localChapDecrypt: enable: false clusterId: cluster1 clusterMode: true chapUser: SFFFzADcpZaUJKsYbPq54A== chapPassword: tbHWrBep3R0RhkFaW+f5Fw== # Sample of statefulset with dynamic pv. Path is templates/examples/filesystem-volumes/statefulset statefulset: enable: false clusterId: cluster1 clusterMode: true # Sample of static pv. Path is templates/examples/filesystem-volumes/static-pv staticPv: enable: false clusterId: cluster1 clusterMode: true lunName: lun04
(2)执行命令helm upgrade stor ./更新配置。
[root@server csi-driver-stor]# helm upgrade stor ./ Release "stor" has been upgraded. Happy Helming! NAME: stor LAST DEPLOYED: Tue May 14 10:47:02 2024 NAMESPACE: default STATUS: deployed REVISION: 25 TEST SUITE: None NOTES: The Stor CSI Plugin has been published. The plugin image is: stor-csi-driver:1.3.0 The Stor Cluster Info is: [ { "clusterID": "stor1", "apiEndPointList": [ "https://192.168.0.192:1443", "https://192.168.0.110:1443", "https://192.168.0.102:1443" ], "storProvider": "HBlock" }, { "clusterID": "stor2", "apiEndPointList": [ "https://192.168.0.32:1443" ], "storProvider": "HBlock" } ]
(3)执行命令kubectl get pod检查样例是否停用,若刚创建的pod不存在,即说明样例已停用。
[root@server csi-driver-stor]# kubectl get pod NAME READY STATUS RESTARTS AGE csi-storplugin-controller-6c789569b9-mfh2q 3/3 Running 0 14d csi-storplugin-node-fdqq7 2/2 Running 6 (5d15h ago) 14d csi-storplugin-node-tj7rf 2/2 Running 0 14d