1.Rocketmq异地双活是什么
Rocketmq是一款高性能,高可用,高可靠的消息中间件,具有可靠的消息存储,消息发送的高性能和低延迟,强大的消息堆积能力与消息处理能力,严格的顺序消息模式等核心优势。
通过rocketmq消息中间件,可以快速实现了服务之间的解耦,流程异步化,流量削峰等核心功能,随着业务越来越复杂,集群规模越来越大 ,服务数量越来越多,依赖关系越来越复杂,消息中间件已经成为不可或缺的重要基础组件之一。
从rocketmq的架构图中,我们可以看到nameserver,broker,producer,consumer都是支持集群部署的,因此实现了高可用,但是考虑到不同机房之间网络开销,稳定性以及部署成本,这样的架构只支持单机房部署,不同rocketmq集群,数据是彼此独立,也就是说,rocketmq原生架构并不支持异地多活。
Rokcetmq的异地双话,简单来说,就是指两个集群分别部署在两个机房,同时对外提供服务,当其中一个机房出现故障,另外一个的机房能接管全部的流量,同时对于故障机房尚未处理的数据,能够接手处理,机房的故障不影响服务的可用性。
2.Rocketmq异地双活需要解决什么问题
1.解决业务跨机房双写的问题;
2.解决基于消息的应用零侵入接入;
3.异地双活消息中间件的解决方案
3.1 异地双活消息中间件架构原理
模块功能说明:
msha-agent
异地多活客户端代理,提供路由传递和消息过滤服务,通过javaagent的方式植入用户应用程序,代码侵入。生产消息的时候为消息打上路由信息,消费消息的时候只消费有当前集群对应路由消息的数据。
msha-mq-sync
提供消息队列异地双向同步服务,以app的方式部署,可集群部署,支持高可用。使得两个broker集群都拥有全量的数据,互为灾备。
msha-manager
用于从整体上管控异地多活服务,主要是切流任务规划和实施,以app的方式部署,单点。基于nacos实现管控指令的下发。
说明:
nameserver,broker是rocketmq原生组件
msha-agent,msha-mq-sync,msha-manager是自研组件
3.2 接入RocketMQ双活能力应用需要做什么
(一)生产端:修改启动脚本,引入双活包
java -javaagent:lib/msha-agent-1.0.0.jar -Dmsha.commander.namespace=DemoNamespace -Dmsha.commander.addr=127.0.0.1:7810 -Dmsha.unit.name=unit-1 -jar rocketmq-lddl-client-demo-1.0.jar
(二)消费端:修改启动脚本,引入双活包
java -javaagent:lib/msha-agent-1.0.0.jar -Dmsha.commander.namespace=DemoNamespace -Dmsha.commander.addr=127.0.0.1:7810 -Dmsha.unit.name=unit-1 -jar rocketmq-lddl-client-demo-1.0.jar
(三)服务端:部署异地双活RocketMQ版本
部署msha-mq-sync集群,保证高可用,用于同步rocketmq broker数据。
(四)故障切换:集成RocketMQ的切流API
调用切流API
http://ip:port/switch?values=beijing
效果:将routerId为beijing的路由切换到另外一个机房,如果原本是unti-1,则切换到unit-2。
routerId由用户按业务需要配置。
4.以短线网关应用场景示例
4.1 业务场景介绍
在互联网服务,政企服务中有大量的发送短信场景,比如手机验证码,通知短信,营销短信等等,短信服务是广大企业客户快速触达手机用户所优先使用的通信能力。越大的平台,对短信服务的要求就越高,高吞吐量,低时延,可靠稳定,实时性高等特点是不可或缺的。
从短信网关需求来看,我们可以发现基于rocketmq实现短信网关是一个很好的选择。
4.2 基于rocketmq实现短信网关
业务架构图
模块简介:
MTR —— 接受用户请求,做必要的业务处理,号码检查,内容检查,通道分配等,封装成消息提交到rocketmq集群。
Channel —— 从rocketmq集群上拉取未消费的消息,按协议提交到运营商网关,完成短信的发送,因为channel是集群部署,可横向扩展,提高短信发送的总速度。
因为rocketmq的引入,短信网关很方便就实现了高吞吐量,低时延,可靠稳定,实时性高等要求。
4.3.短信网关实现异地双活
从上面短信网关的架构来看,我们可以看出这还是一个典型的单机房架构,即使在机房内实现了高可用,但是对于机房级别的故障,其容灾能力是不够的,还可以进一步提升。
基于自研的异地双活能力,我们可以很轻松实现短信网关的异地双活。
应用模块MTR,channel通过javaagent的方式启动,嵌入msha-agent代码,实现消息标记和消息过滤;
不同rocketmq broker集群之间通过msha-mq-sync同步数据,使得两个broker集群数据互为灾备,都拥有全量的数据。
4.4 效果演示
4.4.1 双机房同时对外提供服务。
机房unit-1的消息的生产和消费日志
2022-11-24 23:24:09.757 INFO 18664 --- [nio-8090-exec-1] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416393455239168,投递结果:SEND_OK
2022-11-24 23:24:10.509 INFO 18664 --- [nio-8090-exec-2] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416397083312128,投递结果:SEND_OK
2022-11-24 23:24:10.561 INFO 18664 --- [nio-8090-exec-3] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416397301415936,投递结果:SEND_OK
2022-11-24 23:24:10.628 INFO 18664 --- [nio-8090-exec-4] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416397498548224,投递结果:SEND_OK
2022-11-24 23:24:10.683 INFO 18664 --- [nio-8090-exec-5] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416397779566592,投递结果:SEND_OK
2022-11-24 23:24:10.733 INFO 18664 --- [nio-8090-exec-6] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416398018641920,投递结果:SEND_OK
2022-11-24 23:24:10.784 INFO 18664 --- [nio-8090-exec-7] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416398236745728,投递结果:SEND_OK
2022-11-24 23:24:10.834 INFO 18664 --- [nio-8090-exec-8] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416398446460928,投递结果:SEND_OK
2022-11-24 23:24:10.893 INFO 18664 --- [nio-8090-exec-9] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416398693924864,投递结果:SEND_OK
2022-11-24 23:24:10.939 INFO 18664 --- [io-8090-exec-10] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483416398891057152,投递结果:SEND_OK
2022-11-24 23:24:12.036 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416393455239168,消息来自机房:unit-1
2022-11-24 23:24:14.587 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416397083312128,消息来自机房:unit-1
2022-11-24 23:24:14.603 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416398446460928,消息来自机房:unit-1
2022-11-24 23:24:14.618 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416397779566592,消息来自机房:unit-1
2022-11-24 23:24:14.634 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416397498548224,消息来自机房:unit-1
2022-11-24 23:24:14.650 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416398693924864,消息来自机房:unit-1
2022-11-24 23:24:14.667 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416397301415936,消息来自机房:unit-1
2022-11-24 23:24:14.683 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416398236745728,消息来自机房:unit-1
2022-11-24 23:24:14.700 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416398018641920,消息来自机房:unit-1
2022-11-24 23:24:14.716 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483416398891057152,消息来自机房:unit-1
机房unit-2的消息的生产和消费日志
2022-11-24 23:24:10.124 INFO 21636 --- [nio-8092-exec-1] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416395149737984,投递结果:SEND_OK
2022-11-24 23:24:10.535 INFO 21636 --- [nio-8092-exec-3] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416397192364032,投递结果:SEND_OK
2022-11-24 23:24:10.584 INFO 21636 --- [nio-8092-exec-2] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416397402079232,投递结果:SEND_OK
2022-11-24 23:24:10.652 INFO 21636 --- [nio-8092-exec-4] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416397683097600,投递结果:SEND_OK
2022-11-24 23:24:10.707 INFO 21636 --- [nio-8092-exec-5] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416397913784320,投递结果:SEND_OK
2022-11-24 23:24:10.758 INFO 21636 --- [nio-8092-exec-6] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416398127693824,投递结果:SEND_OK
2022-11-24 23:24:10.808 INFO 21636 --- [nio-8092-exec-7] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416398337409024,投递结果:SEND_OK
2022-11-24 23:24:10.861 INFO 21636 --- [nio-8092-exec-8] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416398555512832,投递结果:SEND_OK
2022-11-24 23:24:10.915 INFO 21636 --- [nio-8092-exec-9] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416398786199552,投递结果:SEND_OK
2022-11-24 23:24:10.963 INFO 21636 --- [io-8092-exec-10] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483416398987526144,投递结果:SEND_OK
2022-11-24 23:24:24.469 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416395149737984,消息来自机房:unit-2
2022-11-24 23:24:24.486 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416397913784320,消息来自机房:unit-2
2022-11-24 23:24:27.000 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416397192364032,消息来自机房:unit-2
2022-11-24 23:24:27.015 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416398337409024,消息来自机房:unit-2
2022-11-24 23:24:27.030 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416397402079232,消息来自机房:unit-2
2022-11-24 23:24:27.046 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416397683097600,消息来自机房:unit-2
2022-11-24 23:24:27.062 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416398127693824,消息来自机房:unit-2
2022-11-24 23:24:27.076 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416398555512832,消息来自机房:unit-2
2022-11-24 23:24:27.091 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416398786199552,消息来自机房:unit-2
2022-11-24 23:24:27.106 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483416398987526144,消息来自机房:unit-2
从两个机房的生产和消费日志可见,短信id一一对应,每个机房只消费本机房生产的数据,数据机房内闭环。
4.4.2 发送过程中进行故障切换
当发现有一个机房出现故障,建设为机房unit-2故障,人工执行切流操作。
从日志中可以看到,路由切换以任务的形式(task=263d41ba-96d6-459e-a5c5-20458af54f07)分步骤执行(共7个步骤执行,phase=1到7),从2022-11-24 23:28:15.460开始,2022-11-24 23:28:23.577结束,耗时8.117秒。
机房unit-2切流日志
2022-11-24 23:28:15.460 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=1
2022-11-24 23:28:16.869 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=2
2022-11-24 23:28:17.883 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=3
2022-11-24 23:28:19.003 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=4
2022-11-24 23:28:20.017 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=5
2022-11-24 23:28:21.133 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=6
2022-11-24 23:28:22.252 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07 , phase=7
2022-11-24 23:28:23.264 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Complete process MSHA route switch: task=263d41ba-96d6-459e-a5c5-20458af54f07
2022-11-24 23:28:23.577 INFO 18664 --- [0-DemoNamespace] SERVER : application.unit-1 Changed MSHA metadata: key=metadata.routers , value=[Metadata.Router: unitName=unit-1 , values=[beijing, guangzhou], Metadata.Router: unitName=unit-2 , values=[]]
切换后机房unit-2的生产消费日志
2022-11-24 23:28:23.513 INFO 21636 --- [nio-8092-exec-8] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417458263195648,投递结果:SEND_OK
2022-11-24 23:28:23.557 INFO 21636 --- [nio-8092-exec-9] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417458447745024,投递结果:SEND_OK
2022-11-24 23:28:23.599 INFO 21636 --- [io-8092-exec-10] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417458623905792,投递结果:SEND_OK
2022-11-24 23:28:23.641 INFO 21636 --- [nio-8092-exec-1] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417458795872256,投递结果:SEND_OK
2022-11-24 23:28:23.680 INFO 21636 --- [nio-8092-exec-3] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417458959450112,投递结果:SEND_OK
2022-11-24 23:28:23.713 INFO 21636 --- [nio-8092-exec-2] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483417459102056448,投递结果:SEND_OK
从日志中可以看到,切流之后机房unit-2之投递消息到mq,并没有消费。
切流后机房1的生产消费日志
2022-11-24 23:28:37.461 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458263195648,消息来自机房:unit-2
2022-11-24 23:28:37.475 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417472561577984,消息来自机房:unit-1
2022-11-24 23:28:37.475 INFO 18664 --- [nio-8090-exec-6] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417516819873792,投递结果:SEND_OK
2022-11-24 23:28:37.490 INFO 18664 --- [nio-8090-exec-7] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417516886982656,投递结果:SEND_OK
2022-11-24 23:28:37.504 INFO 18664 --- [nio-8090-exec-8] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417516941508608,投递结果:SEND_OK
2022-11-24 23:28:37.517 INFO 18664 --- [nio-8090-exec-9] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517000228864,投递结果:SEND_OK
2022-11-24 23:28:37.525 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458623905792,消息来自机房:unit-2
2022-11-24 23:28:37.536 INFO 18664 --- [io-8090-exec-10] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517079920640,投递结果:SEND_OK
2022-11-24 23:28:37.540 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417473811480576,消息来自机房:unit-1
2022-11-24 23:28:37.549 INFO 18664 --- [nio-8090-exec-1] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517134446592,投递结果:SEND_OK
2022-11-24 23:28:37.554 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417456757440512,消息来自机房:unit-1
2022-11-24 23:28:37.563 INFO 18664 --- [nio-8090-exec-2] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517188972544,投递结果:SEND_OK
2022-11-24 23:28:37.578 INFO 18664 --- [nio-8090-exec-3] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517256081408,投递结果:SEND_OK
2022-11-24 23:28:37.592 INFO 18664 --- [nio-8090-exec-4] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517310607360,投递结果:SEND_OK
2022-11-24 23:28:37.592 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458795872256,消息来自机房:unit-2
2022-11-24 23:28:37.606 INFO 18664 --- [nio-8090-exec-5] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517369327616,投递结果:SEND_OK
2022-11-24 23:28:37.619 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417475547922432,消息来自机房:unit-1
2022-11-24 23:28:37.625 INFO 18664 --- [nio-8090-exec-6] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517453213696,投递结果:SEND_OK
2022-11-24 23:28:37.633 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458334498816,消息来自机房:unit-1
2022-11-24 23:28:37.639 INFO 18664 --- [nio-8090-exec-7] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517511933952,投递结果:SEND_OK
2022-11-24 23:28:37.647 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458716180480,消息来自机房:unit-1
2022-11-24 23:28:37.656 INFO 18664 --- [nio-8090-exec-8] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517579042816,投递结果:SEND_OK
2022-11-24 23:28:37.663 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417459513098240,消息来自机房:unit-1
2022-11-24 23:28:37.668 INFO 18664 --- [nio-8090-exec-9] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517633568768,投递结果:SEND_OK
2022-11-24 23:28:37.683 INFO 18664 --- [io-8090-exec-10] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517696483328,投递结果:SEND_OK
2022-11-24 23:28:37.690 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417459034947584,消息来自机房:unit-1
2022-11-24 23:28:37.697 INFO 18664 --- [nio-8090-exec-1] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517755203584,投递结果:SEND_OK
2022-11-24 23:28:37.711 INFO 18664 --- [nio-8090-exec-2] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517813923840,投递结果:SEND_OK
2022-11-24 23:28:37.719 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417475866689536,消息来自机房:unit-1
2022-11-24 23:28:37.728 INFO 18664 --- [nio-8090-exec-3] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517885227008,投递结果:SEND_OK
2022-11-24 23:28:37.733 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417460884635648,消息来自机房:unit-1
2022-11-24 23:28:37.742 INFO 18664 --- [nio-8090-exec-4] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517943947264,投递结果:SEND_OK
2022-11-24 23:28:37.746 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458447745024,消息来自机房:unit-2
2022-11-24 23:28:37.753 INFO 18664 --- [nio-8090-exec-5] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417517990084608,投递结果:SEND_OK
2022-11-24 23:28:37.760 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417459164971008,消息来自机房:unit-1
2022-11-24 23:28:37.775 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417459102056448,消息来自机房:unit-2
2022-11-24 23:28:37.776 INFO 18664 --- [nio-8090-exec-6] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518086553600,投递结果:SEND_OK
2022-11-24 23:28:37.788 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417476227399680,消息来自机房:unit-1
2022-11-24 23:28:37.788 INFO 18664 --- [nio-8090-exec-7] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518136885248,投递结果:SEND_OK
2022-11-24 23:28:37.802 INFO 18664 --- [nio-8090-exec-8] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518191411200,投递结果:SEND_OK
2022-11-24 23:28:37.802 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417461186625536,消息来自机房:unit-1
2022-11-24 23:28:37.815 INFO 18664 --- [nio-8090-exec-9] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518245937152,投递结果:SEND_OK
2022-11-24 23:28:37.817 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417459454377984,消息来自机房:unit-1
2022-11-24 23:28:37.829 INFO 18664 --- [io-8090-exec-10] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518308851712,投递结果:SEND_OK
2022-11-24 23:28:37.832 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417460146438144,消息来自机房:unit-1
2022-11-24 23:28:37.845 INFO 18664 --- [nio-8090-exec-1] com.example.controller.DemoController : 机房:unit-1 - 投递消息到mq,短信id:483417518371766272,投递结果:SEND_OK
2022-11-24 23:28:37.846 INFO 18664 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-1 从mq消费消息 - 短信id:483417458959450112,消息来自机房:unit-2
从日志中可见,机房2产生的数据,sms id从到483417458263195648到483417459102056448,共6条,都由机房1接收到并消费了,期间机房自身生产的消息继续正常消费,异地切流成功。
4.4.3 故障机房unit-2恢复,回切
切流日志,从2022-11-25 00:04:38.541开始,2022-11-25 00:04:51.168完成,耗时12.658秒。
2022-11-25 00:04:38.541 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=1
2022-11-25 00:04:39.349 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=2
2022-11-25 00:04:40.573 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=3
2022-11-25 00:04:41.692 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=4
2022-11-25 00:04:42.708 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=5
2022-11-25 00:04:43.844 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=6
2022-11-25 00:04:44.959 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398 , phase=7
2022-11-25 00:04:51.161 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Complete process MSHA route switch: task=530d585e-1b62-40fe-8356-f8ef8ed37398
2022-11-25 00:04:51.168 INFO 21636 --- [0-DemoNamespace] SERVER : application.unit-2 Changed MSHA metadata: key=metadata.routers , value=[Metadata.Router: unitName=unit-1 , values=[beijing], Metadata.Router: unitName=unit-2 , values=[guangzhou]]
机房unit-2正常投入使用,回切成功。
2022-11-25 00:04:51.392 INFO 21636 --- [nio-8092-exec-4] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426634892836864,投递结果:SEND_OK
2022-11-25 00:04:51.431 INFO 21636 --- [nio-8092-exec-5] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635056414720,投递结果:SEND_OK
2022-11-25 00:04:51.473 INFO 21636 --- [nio-8092-exec-6] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635228381184,投递结果:SEND_OK
2022-11-25 00:04:51.514 INFO 21636 --- [nio-8092-exec-7] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635404541952,投递结果:SEND_OK
2022-11-25 00:04:51.560 INFO 21636 --- [nio-8092-exec-8] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635593285632,投递结果:SEND_OK
2022-11-25 00:04:51.600 INFO 21636 --- [nio-8092-exec-9] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635756863488,投递结果:SEND_OK
2022-11-25 00:04:51.637 INFO 21636 --- [io-8092-exec-10] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426635920441344,投递结果:SEND_OK
2022-11-25 00:04:51.681 INFO 21636 --- [nio-8092-exec-1] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426636100796416,投递结果:SEND_OK
2022-11-25 00:04:51.725 INFO 21636 --- [nio-8092-exec-3] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426636289540096,投递结果:SEND_OK
2022-11-25 00:04:51.765 INFO 21636 --- [nio-8092-exec-2] com.example.controller.DemoController : 机房:unit-2 - 投递消息到mq,短信id:483426636457312256,投递结果:SEND_OK
2022-11-25 00:04:58.186 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426634892836864,消息来自机房:unit-2
2022-11-25 00:04:58.275 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635056414720,消息来自机房:unit-2
2022-11-25 00:04:58.200 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635228381184,消息来自机房:unit-2
2022-11-25 00:04:58.068 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635404541952,消息来自机房:unit-2
2022-11-25 00:04:58.082 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426636289540096,消息来自机房:unit-2
2022-11-25 00:04:58.215 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635756863488,消息来自机房:unit-2
2022-11-25 00:04:58.260 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635593285632,消息来自机房:unit-2
2022-11-25 00:04:58.290 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426635920441344,消息来自机房:unit-2
2022-11-25 00:04:58.304 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426636457312256,消息来自机房:unit-2
2022-11-25 00:04:58.318 INFO 21636 --- [MessageThread_1] com.example.mq.MshaConsumer : 机房:unit-2 从mq消费消息 - 短信id:483426636100796416,消息来自机房:unit-2