摘要:
记录使用freetsdb组建集群.
环境准备:
- 至少三台独立的虚拟机
- 编译安装freetsdb
freetsdb组建集群官方说明:
## 部署
FreeTSDB原生支持分布式集群能力，FreeTSDB集群的部署主要涉及到META节点和DATA节点，META节点存放的是系统运行所必须的元数据，DATA节点存放的是实际的时序数据。本文档将以3 META节点、2 DATA节点的集群为例演示如何搭建FreeTSDB集群。
### 部署META节点
基于可用性和资源成本的考虑，META节点推荐为3节点。
#### 配置META接地那
##### Step 1: 修改/etc/hosts文件，配置主机名字信息。
将各META节点的名字（比如cluster-meta-node-01）和对应的地址信息（比如<Meta_1_IP>）配置在各META节点和DATA节点的/etc/hosts文件中。
```
<Meta_1_IP> cluster-meta-node-01
<Meta_2_IP> cluster-meta-node-02
<Meta_3_IP> cluster-meta-node-03
```
配置完主机名字信息后，在各META节点和各DATA节点上分别执行如下ping命令，确保配置的正确性和网络的连通性。
>
> ping -qc 1 cluster-meta-node-01
>
> ping -qc 1 cluster-meta-node-02
>
> ping -qc 1 cluster-meta-node-03
##### Step 2: 配置和启动META服务
在各META节点上，分别执行如下命令。
###### I. 下载安装包
比如，在Linux系统上，使用以下命令下载freetsdb-v0.0.2-beta.1的安装包。
```
wget https:///freetsdb/freetsdb/releases/download/v0.0.2-beta.1/freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz
tar -zxvf freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz
```
###### II. 修改配置
修改配置文件./freetsdb-meta.conf，并将hostname设置为本机的主机名字（比如<cluster-meta-node-0x>）
```
# Hostname advertised by this host for remote addresses. This must be resolvable by all
other nodes in the cluster
hostname="<cluster-meta-node-0x>"
```
###### III. 启动META服务
在Linux shell命令行中，执行如下命令。
```
sudo ./freetsd-meta -config ./freetsd-meta.conf
```
###### Step 3: 将META节点加入到集群中
在其中的一个META节点上（比如cluster-meta-node-01），执行如下命令，将已配置好的META节点加入到集群中。
```
freetsd-ctl add-meta cluster-meta-node-01:8091
freetsd-ctl add-meta cluster-meta-node-02:8091
freetsd-ctl add-meta cluster-meta-node-03:8091
```
正确执行时，add-meta命令的执行输出如下：
```
Added meta node x at cluster-meta-node-0x:8091
```
将所有META节点加入到集群后，可以执行“freetsd-ctl show”来查看集群中META节点的信息。
>
> ```
> freetsd-ctl show
>
> ```
>
> The expected output is:
>
> ```
> Data Nodes:
>
> Meta Nodes:
> 1 cluster-meta-node-01:8091
> 2 cluster-meta-node-02:8091
> 3 cluster-meta-node-03:8091
>
> ```
### 部署DATA节点
本文档将以2个DATA节点的集群部署为例，演示如何添加DATA节点。
#### 配置DATA节点
##### Step 1: 修改/etc/hosts文件，配置DATA节点的主机名字信息。
将各DATA节点的名字（比如cluster-data-node-01）和对应的地址信息（比如<Data_1_IP>）配置在各META节点和DATA节点的/etc/hosts文件中。
```
<Data_1_IP> cluster-data-node-01
<Data_2_IP> cluster-data-node-02
```
配置完主机名字信息后，在各META节点和DATA节点上分别执行如下ping命令，确保配置的正确性和网络的连通性。
>
> ping -qc 1 cluster-meta-node-01
>
> ping -qc 1 cluster-meta-node-02
>
> ping -qc 1 cluster-meta-node-03
>
> ping -qc 1 cluster-data-node-01
>
> ping -qc 1 cluster-data-node-02
##### Step 2: 配置和启动DATA服务
在各DATA节点上，分别执行如下命令。
###### I. 下载安装包
比如，在Linux系统上，使用以下命令下载freetsdb-v0.0.2-beta.1的安装包。
```
wget https:///freetsdb/freetsdb/releases/download/v0.0.2-beta.1/freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz
tar -zxvf freetsdb-v0.0.2-beta.1_linux_amd64.tar.gz
```
###### II. Edit the data node configuration files
修改配置文件./freetsdb.conf，并将hostname设置为本机的主机名字（比如<cluster-data-node-0x>）
```
# Change this option to true to disable reporting.
reporting-disabled = false
bind-address = ":8088"
hostname="<cluster-data-node-0x>"
```
###### III. 启动DATA服务
在Linux shell命令行中，执行如下命令。
```
sudo ./freetsd -config ./freetsd.conf
```
#### 将DATA节点加入到集群中
在其中的一个META节点上（比如cluster-meta-node-01），执行如下命令，将已配置好的DATA节点加入到集群中。
```
freetsd-ctl add-data cluster-data-node-01:8088
freetsd-ctl add-data cluster-data-node-02:8088
```
正确执行时，add-data命令的执行输出如下：
```
Added data node y at cluster-data-node-0x:8088
```
将所有DATA节点加入到集群后，可以执行“freetsd-ctl show”来查看集群中DATA节点的信息。
>
> ```
> freetsd-ctl show
> ```
>
> The expected output is:
>
> ```
> Data Nodes:
> 4 cluster-data-node-01:8088
> 5 cluster-data-node-02:8088
>
> Meta Nodes:
> 1 cluster-meta-node-01:8091
> 2 cluster-meta-node-02:8091
> 3 cluster-meta-node-03:8091
> ```
恭喜你，FreeTSDB集群已搭建完成，欢迎进入FreeTSDB的精彩世界。
```
# 关闭防火墙
systemctl stop firewalld.service
systemctl disable firewalld.service
```
```
# 添加新的机器
./freetsd-ctl add-meta 192.168.58.132:8091
./freetsd-ctl add-data 192.168.58.131:8088
```
本地虚拟机部署freetsdb:
至少准备三台虚拟机, 为了增加对比, 使用四台虚拟机, ip地址分别为:
192.168.58.128
192.168.58.131
192.168.58.132
192.168.58.133
以192.168.58.132为例, 配置文件:
完整的58.132上的配置的文件夹: freetsdb-v0.0.2-58.132-其它文档类资源-
meta配置:
freetsd-meta.conf
bind-address = ":8089"
hostname = "192.168.58.132"
[meta]
enabled = true
dir = "/var/lib/freetsdb/meta"
bind-address = ":8089"
http-bind-address = ":8091"
https-enabled = false
https-certificate = ""
retention-autocreate = true
election-timeout = "1s"
heartbeat-timeout = "1s"
leader-lease-timeout = "500ms"
commit-timeout = "50ms"
cluster-tracing = false
raft-promotion-enabled = true
logging-enabled = true
pprof-enabled = false
lease-duration = "1m0s"
data配置:
freetsd.conf
reporting-disabled = false
bind-address = ":8088"
hostname = "192.168.58.132"
[data]
enabled = true
dir = "/var/lib/freetsdb/data"
engine = "tsm1"
wal-dir = "/var/lib/freetsdb/wal"
wal-logging-enabled = true
query-log-enabled = true
cache-max-memory-size = 1073741824
cache-snapshot-memory-size = 26214400
cache-snapshot-write-cold-duration = "10m0s"
compact-full-write-cold-duration = "24h0m0s"
max-points-per-block = 0
data-logging-enabled = true
[cluster]
force-remote-mapping = false
write-timeout = "5s"
shard-writer-timeout = "5s"
max-remote-write-connections = 3
shard-mapper-timeout = "5s"
[retention]
enabled = true
check-interval = "30m0s"
[shard-precreation]
enabled = true
check-interval = "10m0s"
advance-period = "30m0s"
[admin]
enabled = true
bind-address = ":8083"
https-enabled = false
https-certificate = "/etc/ssl/freetsdb.pem"
Version = ""
[monitor]
store-enabled = true
store-database = "_internal"
store-interval = "10s"
[subscriber]
enabled = true
[http]
enabled = true
bind-address = ":8086"
auth-enabled = false
log-enabled = true
write-tracing = false
pprof-enabled = false
https-enabled = false
https-certificate = "/etc/ssl/freetsdb.pem"
json-write-enabled = false
[[graphite]]
enabled = false
bind-address = ":2003"
database = "graphite"
protocol = "tcp"
batch-size = 5000
batch-pending = 10
batch-timeout = "1s"
consistency-level = "one"
separator = "."
udp-read-buffer = 0
[collectd]
enabled = false
bind-address = ":25826"
database = "collectd"
retention-policy = ""
batch-size = 5000
batch-pending = 10
batch-timeout = "10s"
read-buffer = 0
typesdb = "/usr/share/collectd/types.db"
[opentsdb]
enabled = false
bind-address = ":4242"
database = "opentsdb"
retention-policy = ""
consistency-level = "one"
tls-enabled = false
certificate = "/etc/ssl/freetsdb.pem"
batch-size = 1000
batch-pending = 5
batch-timeout = "1s"
log-point-errors = true
[[udp]]
enabled = false
bind-address = ":8089"
database = "udp"
retention-policy = ""
batch-size = 5000
batch-pending = 10
read-buffer = 0
batch-timeout = "1s"
precision = ""
udp-payload-size = 0
[continuous_queries]
log-enabled = true
enabled = true
run-interval = "1s"
[hinted-handoff]
enabled = true
dir = "/var/lib/freetsdb/hh"
max-size = 1073741824
max-age = "168h0m0s"
retry-rate-limit = 0
retry-interval = "1s"
retry-max-interval = "1m0s"
purge-interval = "1h0m0s"
其他虚拟机需要修改对应的hostname为本机的ip.
运行:
启动meta
nohup ./freetsd-meta -config ./freetsd-meta.conf > freetsd-meta.log &
启动data:
nohup ./freetsd -config ./freetsd.conf > freetsd.log &
组建集群:
添加meta:
在任意部署meta的机器上, 执行:
./freetsd-ctl add-meta 192.168.58.128:8091
./freetsd-ctl add-meta 192.168.58.131:8091
./freetsd-ctl add-meta 192.168.58.132:8091
./freetsd-ctl add-meta 192.168.58.133:8091
./freetsd-ctl add-data 192.168.58.128:8088
./freetsd-ctl add-data 192.168.58.131:8088
./freetsd-ctl add-data 192.168.58.132:8088
./freetsd-ctl add-data 192.168.58.133:8088
查看集群组建情况:
./freetsd-ctl show
Meta Nodes:
{1 192.168.58.128:8091 192.168.58.128:8089}
{5 192.168.58.132:8091 192.168.58.132:8089}
{6 192.168.58.133:8091 192.168.58.133:8089}
{8 192.168.58.131:8091 192.168.58.131:8089}
Data Nodes:
{2 192.168.58.128:8086 192.168.58.128:8088}
{7 192.168.58.133:8086 192.168.58.133:8088}
{10 192.168.58.132:8086 192.168.58.132:8088}
{11 192.168.58.131:8086 192.168.58.131:8088}
