Kubernetes提供了原生审计功能,其审计是用于记录外部对 Kubernetes API 的访问和操作的。它记录了对集群资源的创建、修改和删除等操作,以及与这些操作相关的用户、时间戳和请求信息。
由于原生审计功能主要关注对 Kubernetes API 的访问和操作,它能够审计到对于容器的exec操作,具体操作如下:
1、定义审计策略,比如审计对象是pod相关的全部操作:
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Request
resources:
- group: ""
resources: ["pods/*"]
verbs: ["*"]
2、启用 API Server 的审计功能
设置kube-apiserver的两个启动参数audit-policy-file和audit-log-path:
- --audit-policy-file=/root/audit/policy.yaml
- --audit-log-path=/var/log/pods.audit
- --audit-log-maxage=7
- --audit-log-maxbackup=4
- --audit-log-maxsize=10
- --audit-log-format=json
3、kube-apiserver重启
4、执行pod exec操作,观察审计日志输出:
示例1:执行kubectl exec nginx-6947d66995-6wwm6 -it sh
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Request","auditID":"c8259450-a7da-406b-ab93-8950dfabe4aa","stage":"RequestReceived","requestURI":"/api/v1/namespaces/default/pods/nginx-6947d66995-6wwm6/exec?command=sh\u0026container=nginx\u0026stdin=true\u0026stdout=true\u0026tty=true","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["192.168.59.115"],"userAgent":"kubectl/v1.27.4 (linux/amd64) kubernetes/fa3d799","objectRef":{"resource":"pods","namespace":"default","name":"nginx-6947d66995-6wwm6","apiVersion":"v1","subresource":"exec"},"requestReceivedTimestamp":"2023-08-02T09:53:52.632620Z","stageTimestamp":"2023-08-02T09:53:52.632620Z"}
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Request","auditID":"c8259450-a7da-406b-ab93-8950dfabe4aa","stage":"ResponseStarted","requestURI":"/api/v1/namespaces/default/pods/nginx-6947d66995-6wwm6/exec?command=sh\u0026container=nginx\u0026stdin=true\u0026stdout=true\u0026tty=true","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["192.168.59.115"],"userAgent":"kubectl/v1.27.4 (linux/amd64) kubernetes/fa3d799","objectRef":{"resource":"pods","namespace":"default","name":"nginx-6947d66995-6wwm6","apiVersion":"v1","subresource":"exec"},"responseStatus":{"metadata":{},"code":101},"requestReceivedTimestamp":"2023-08-02T09:53:52.632620Z","stageTimestamp":"2023-08-02T09:53:52.651550Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Request","auditID":"c8259450-a7da-406b-ab93-8950dfabe4aa","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/default/pods/nginx-6947d66995-6wwm6/exec?command=sh\u0026container=nginx\u0026stdin=true\u0026stdout=true\u0026tty=true","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["192.168.59.115"],"userAgent":"kubectl/v1.27.4 (linux/amd64) kubernetes/fa3d799","objectRef":{"resource":"pods","namespace":"default","name":"nginx-6947d66995-6wwm6","apiVersion":"v1","subresource":"exec"},"responseStatus":{"metadata":{},"code":101},"requestReceivedTimestamp":"2023-08-02T09:53:52.632620Z","stageTimestamp":"2023-08-02T09:54:37.965628Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":""}}
示例2:执行kubectl exec nginx-6947d66995-6wwm6 -it -- mkdir /test
{"kind":"Event","apiVersion":"audit.k8s.io/v1","level":"Request","auditID":"3417527d-4ad3-4028-b6ad-5540c7076c48","stage":"RequestReceived","requestURI":"/api/v1/namespaces/default/pods/nginx-6947d66995-6wwm6/exec?command=mkdir\u0026command=%2Ftest\u0026container=nginx\u0026stdin=true\u0026stdout=true\u0026tty=true","verb":"create","user":{"username":"kubernetes-admin","groups":["system:masters","system:authenticated"]},"sourceIPs":["192.168.59.115"],"userAgent":"kubectl/v1.27.4 (linux/amd64) kubernetes/fa3d799","objectRef":{"resource":"pods","namespace":"default","name":"nginx-6947d66995-6wwm6","apiVersion":"v1","subresource":"exec"},"requestReceivedTimestamp":"2023-08-02T09:43:26.965583Z","stageTimestamp":"2023-08-02T09:43:26.965583Z"}
验证结果说明原生审计功能只能记录exec后面直接跟命令操作的场景,对于执行exec -it bash后进入容器的命令并不能记录。