概述

Redis 分片集群是为了应对业务增长带来的数据增长, 实现在线动态横向扩缩容，集群中的每个分片主节点处理 16384 个哈希槽的一个子集; 为了稳定也可以进行高可用部署, 即将各分片节点启用一个或多个副本节点; 当分片主节点挂掉,其对应的副本节点自动提升为新主节点,当原主节点重新上线后,自动成为新主节点的副本节点; 当一对分片主从节点都故障时,整个集群不能正常工作，需要人工介入恢复启动故障节点

分片集群

1、创建一个有 3 个分片节点的集群

# 创建具有 3 个分片节点的集群、每个分片有 1 个副本
# redis-cli --cluster create 192.168.31.17:6384 192.168.31.18:6384 192.168.31.19:6384 --cluster-replicas 1

# 创建具有 3 个分片节点的集群, 每个分片默认无副本
redis-cli --cluster create 192.168.31.17:6384 192.168.31.18:6384 192.168.31.19:6384
> >>> Performing hash slots allocation on 3 nodes...
> Master[0] -> Slots 0 - 5460
> Master[1] -> Slots 5461 - 10922
> Master[2] -> Slots 10923 - 16383
> M: 1db08d1048b20b22dfbf5c5d724d95f46a843d29 192.168.31.17:6384
>    slots:[0-5460] (5461 slots) master
> M: 3f9d82efdf04bb5fcf3741de4ea9314925bbeefd 192.168.31.18:6384
>    slots:[5461-10922] (5462 slots) master
> M: 16a0c6947fcb7d498b7c96aa5b5a70491298e966 192.168.31.19:6384
>    slots:[10923-16383] (5461 slots) master
> Can I set the above configuration? (type 'yes' to accept): yes
> >>> Nodes configuration updated
> >>> Assign a different config epoch to each node
> >>> Sending CLUSTER MEET messages to join the cluster
> Waiting for the cluster to join
> ..
> >>> Performing Cluster Check (using node 192.168.31.17:6384)
> M: 1db08d1048b20b22dfbf5c5d724d95f46a843d29 192.168.31.17:6384
>    slots:[0-5460] (5461 slots) master
> M: 16a0c6947fcb7d498b7c96aa5b5a70491298e966 192.168.31.19:6384
>    slots:[10923-16383] (5461 slots) master
> M: 3f9d82efdf04bb5fcf3741de4ea9314925bbeefd 192.168.31.18:6384
>    slots:[5461-10922] (5462 slots) master
> [OK] All nodes agree about slots configuration.
> >>> Check for open slots...
> >>> Check slots coverage...
> [OK] All 16384 slots covered.

验证

1、查看指定 redis 节点状态(包括键数、内存使用量、请求数、连接数等)

redis-cli --stat -p 6384 -h 192.168.31.17
> ------- data ------ --------------------- load -------------------- - child -
> keys       mem      clients blocked requests            connections
> 30022      9.25M    2       0       30518 (+2)          59
> 30022      9.25M    2       0       30520 (+2)          59
> 30022      9.25M    2       0       30522 (+2)          59
> 30022      9.25M    2       0       30524 (+2)          59
> 30022      9.25M    2       0       30526 (+2)          59
> 30022      9.25M    2       0       30528 (+2)          59
> 30022      9.25M    2       0       30530 (+2)          59

2、查看各主节点哈希槽、键的分布情况, 以及副本数、键总数

redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6384 (79c19006...) -> 49888 keys | 5462 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 50115 keys | 5461 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 49995 keys | 5461 slots | 1 slaves.
> [OK] 149998 keys in 3 masters.
> 9.16 keys per slot on average.

集群管理

1、redis-cli 交互式命令行

redis-cli -p 6384 -h 192.168.31.17
# 查看集群节点信息
cluster nodes

# meet 添加节点, 任意节点执行即可
cluster meet 192.168.31.17 6385

# forget 根据节点 ID 删除节点, 需要在所有节点执行该命令
cluster forget 82d14741e161d07012783ae7c9bf1ef20ddc28a0

# 将当前 cluster 节点配置为指定主节点(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的副本节点
cluster replicate 1db08d1048b20b22dfbf5c5d724d95f46a843d29

# 查看指定 master 节点(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的副本节点
cluster replicas 1db08d1048b20b22dfbf5c5d724d95f46a843d29

cluster reset   # 重置当前节点集群状态文件 nodes.conf

2、redis-cli --cluster 子命令

# 在集群中所有节点上执行命令 client list
redis-cli --cluster call 192.168.31.17:6384 client list

# 将节点 192.168.31.17:6385 添加为新的主节点, 需要从其它主节点迁移部分槽位到该新主节点才能正常工作
redis-cli --cluster add-node 192.168.31.17:6385 192.168.31.17:6384

# 将节点 192.168.31.17:6385 添加为主节点(1db08d1048b20b22dfbf5c5d724d95f46a843d29)的从节点
redis-cli --cluster add-node 192.168.31.17:6385 192.168.31.17:6384 --cluster-slave --cluster-master-id 1db08d1048b20b22dfbf5c5d724d95f46a843d29

# 删除节点 82d14741e161d07012783ae7c9bf1ef20ddc28a0 且关闭该节点
redis-cli --cluster del-node 192.168.31.17:6384 82d14741e161d07012783ae7c9bf1ef20ddc28a0

# 从指定节点迁移 1024 个槽位到目标节点 073ad3a6dfe64289be66c6d8e302b29afc4b0ec7
redis-cli --cluster reshard 192.168.31.17:6385 --cluster-from 1db08d1048b20b22dfbf5c5d724d95f46a843d29 --cluster-to 073ad3a6dfe64289be66c6d8e302b29afc4b0ec7 --cluster-slots 1024

# 使用如下命令, 进入交互式输入对应参数值进行节点间的槽位迁移
redis-cli --cluster reshard 192.168.31.17:6384

# 当各节点槽位不均衡时,执行该命令、自动均衡各节点槽位 slots
redis-cli --cluster rebalance 192.168.31.17:6384

扩容

1、集群主节点扩容, 由 3 个主节点扩展到 5 个主节点

// 添加新节点 192.168.31.{18,19}:6386
redis-cli --cluster add-node 192.168.31.18:6386 192.168.31.17:6384
redis-cli --cluster add-node 192.168.31.19:6386 192.168.31.17:6384
// 自动平衡集群各主节点哈希槽的分布
redis-cli --cluster rebalance 192.168.31.17:6384 --cluster-use-empty-masters
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6386 (c64357f2...) -> 29894 keys | 3278 slots | 0 slaves.
> 192.168.31.18:6384 (79c19006...) -> 29927 keys | 3276 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 29835 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 30083 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6386 (a0f71de7...) -> 30259 keys | 3278 slots | 0 slaves.
> [OK] 149998 keys in 5 masters.
> 9.16 keys per slot on average.

缩容

1、集群主节点扩容, 由 5 个主节点扩展到 3 个主节点

// 迁移节点 192.168.31.18:6386 的哈希槽
redis-cli --cluster reshard 192.168.31.17:6384 \
--cluster-from c64357f288902d1209fdf91f4804bf13b2a59fcf \
--cluster-to  79c19006ae3dcd12522090ddcc2b5b917b0feb2a \
--cluster-slots 3278 \
--cluster-yes

// 迁移节点 192.168.31.19:6386 的哈希槽
redis-cli --cluster reshard 192.168.31.17:6384 \
--cluster-from a0f71de78c2355f99de2b4c19796afb02e23c623 \
--cluster-to  df7d9f8e2d7dbad37bdd1094c2770d1feabbd228 \
--cluster-slots 3278 \
--cluster-yes

// 查看各主节点哈希槽、键的分布情况
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6386 (c64357f2...) -> 0 keys | 0 slots | 0 slaves.
> 192.168.31.18:6384 (79c19006...) -> 59821 keys | 6554 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 29835 keys | 3276 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 60342 keys | 6554 slots | 1 slaves.
> 192.168.31.19:6386 (a0f71de7...) -> 0 keys | 0 slots | 0 slaves.
> [OK] 149998 keys in 5 masters.
> 9.16 keys per slot on average.

// 自动平衡集群各主节点哈希槽的分布
redis-cli --cluster rebalance 192.168.31.17:6384

// 删除节点 192.168.31.{18,19}:6386
redis-cli --cluster del-node 192.168.31.17:6384 c64357f288902d1209fdf91f4804bf13b2a59fcf
redis-cli --cluster del-node 192.168.31.17:6384 a0f71de78c2355f99de2b4c19796afb02e23c623

// 查看各主节点哈希槽、键的分布情况
redis-cli --cluster info 192.168.31.17:6384
> 192.168.31.18:6384 (79c19006...) -> 49762 keys | 5461 slots | 1 slaves.
> 192.168.31.17:6385 (3faafe60...) -> 50060 keys | 5462 slots | 1 slaves.
> 192.168.31.19:6384 (df7d9f8e...) -> 50176 keys | 5461 slots | 1 slaves.
> [OK] 149998 keys in 3 masters.
> 9.16 keys per slot on average.

迁移

Redis-shake 是一个用于在两个 Redis 之间同步数据的工具,支持 Redis Cluster 的在线迁移与离线迁移(备份文件导入) 支持解析、恢复、备份、同步四个功能

1、在线迁移

首次全量同步,然后进行增量的拉取(通过 psync 命令)

# 创建迁移目标 redis 集群
redis-cli --cluster create 192.168.31.17:6386 192.168.31.18:6386 192.168.31.19:6386

# cat redis-shake.conf 源及目标端主要配置项, 其它配置项默认即可
source.type: cluster
# master 可改为 slave, 表示从副本同步数据到目标端, 192.168.31.19 为任意源端 redis 节点
source.address: master@192.168.31.19:6384
target.type: cluster
target.address: master@192.168.31.19:6386

./redis-shake -type sync -conf redis-shake.conf

2、离线迁移

适用于源实例与目标实例的网络无法连通的场景,或源端实例部署在其他云厂商 Redis 服务中,无法实现在线迁移

# cat redis-shake.conf 源及目标端主要配置项, 其它配置项默认即可
source.address: master@192.168.31.19:6384
# restore 导入数据时指定的备份文件
source.rdb.input: local_dump.0;local_dump.1;local_dump.2
target.type: cluster
target.address: master@192.168.31.19:6386

备份: ./redis-shake -type dump -conf redis-shake.conf

恢复: ./redis-shake -type restore -conf redis-shake.conf