一、应用场景

跨网络访问kafka集群

kafka集群部署在vpc，提供一个或多个公网ip，kafka sdk通过公网ip访问kafka集群

二、方案

kafka集群信息如下：

broker1：vpc ip1
broker2：vpc ip2
broker3：vpc ip3

1、同vpc访问

直接配置bootstrap.servers

bootstrap.servers = vpc ip1: port, vpc ip2: port, vpc ip3: port

2、公网多ip访问

每个kafka节点对应一个公网ip，集群暴露多个公网ip，同一个端口

bootstrap.servers = 公网ip1: port, 公网ip2: port, 公网ip3: port

转发关系

公网ip1 --- broker1

公网ip2 --- broker2

公网ip3 --- broker3

broker1：advertised.listeners=PLAINTEXT://公网ip1: port
broker2：advertised.listeners=PLAINTEXT://公网ip2: port
broker3：advertised.listeners=PLAINTEXT://公网ip3: port

3、公网多端口访问

每个kafka节点对应一个公网端口，集群暴露一个公网ip，多个端口

bootstrap.servers = 公网ip: port1, 公网ip: port2, 公网ip: port3

转发关系

port1 --- broker1

port2 --- broker2

port3 --- broker3

broker1：advertised.listeners=PLAINTEXT://公网ip: port1
broker2：advertised.listeners=PLAINTEXT://公网ip: port2
broker3：advertised.listeners=PLAINTEXT://公网ip: port3

4、反向代理实现端口映射节点

以nginx为例

stream {
    upstream broker1 {
        server vpc_ip1:port;
    }
    upstream broker2 {
        server vpc_ip2:port;
    }
    upstream broker3 {
        server vpc_ip3:port;
    }

    server {
        listen port1;
        proxy_pass broker1;
    }
    server {
        listen port2;
        proxy_pass broker2;
    }
    server {
        listen port3;
        proxy_pass broker3;
    }
}