集群概述

• 所谓集群，就是通过添加服务器的数量，提供相同的服务，从而让服务器达到一个稳定、高效的状态
• 单个redis存在不稳定性。当redis服务宕机了，就没有可用的服务了
• 单个redis的读写能力是有限的
• Redis集群是为了强化redis的读写能力
• redis集群中，每一个redis称之为一个节点
• 所有的redis节点彼此互联(PING-PONG机制)，内部使用二进制协议优化传输速度和带宽
• 节点的fail是通过集群中超过半数的节点检测失效时才生效
• 客户端与redis节点直连，不需要中间proxy层.客户端不需要连接集群所有节点,连接集群中任何一个可用节点即可
• redis集群中，有两种类型的节点：主节点(master)、从节点(slave)
• redis集群，是基于redis主从复制实现
• 主从复制模型中，有多个redis节点
• 其中，有且仅有一个为主节点Master。从节点Slave可以有多个
• 只要网络连接正常，Master会一直将自己的数据更新同步给Slaves，保持主从同步
• 主节点Master可读、可写
• 从节点Slave只读

部署Redis集群

基本原理

• 集群中至少应该有奇数个节点，所以至少有三个节点，每个节点至少有一个备份节点
• 架构图

管理节点：
m(Manager:192.168.1.20)
缓存节点：
redis1(redis1:192.168.1.11) 、redis2(redis2:192.168.1.12)、redis3(redis3:192.168.1.13)、 redis4(redis4:192.168.1.14)、redis5(redis5:192.168.1.15)、 redis6(redis6:192.168.1.16)

• redis-cluster把所有的物理节点映射到[0-16383]slot上（不一定是平均分配），cluster 负责维护node<->slot<->value
• Redis集群预分好16384个桶，当需要在 Redis 集群中放置一个 key-value 时，根据 CRC16(key) mod 16384的值，决定将一个key放到哪个桶中。

在redis1(192.168.1.11)上部署redis

• 安装编译器

[root@redis1 ~]# yum install -y gcc

• 编译安装redis

[root@redis1 ~]# tar xf redis-4.0.8.tar.gz
[root@redis1 ~]# cd redis-4.0.8

# 修改安装目录为/usr/local/redis
[root@redis1 redis-4.0.8]# vim +27 src/Makefile
PREFIX?=/usr/local/redis

# 编译安装
[root@redis1 redis-4.0.8]# make && make install

# 将redis命令目录添加至PATH环境变量
[root@redis1 redis-4.0.8]# vim /etc/bashrc   # 尾部追加
export PATH=$PATH:/usr/local/redis/bin
[root@redis1 redis-4.0.8]# source /etc/bashrc

# 初始化redis服务
[root@redis1 redis-4.0.8]# ./utils/install_server.sh  # 全部问题直接回车采用默认值
Welcome to the redis service installer
This script will help you easily set up a running redis server

Please select the redis port for this instance: [6379]
Selecting default: 6379
Please select the redis config file name [/etc/redis/6379.conf]
Selected default - /etc/redis/6379.conf
Please select the redis log file name [/var/log/redis_6379.log]
Selected default - /var/log/redis_6379.log
Please select the data directory for this instance [/var/lib/redis/6379]
Selected default - /var/lib/redis/6379
Please select the redis executable path [/usr/local/redis/bin/redis-server]
Selected config:
Port           : 6379
Config file    : /etc/redis/6379.conf
Log file       : /var/log/redis_6379.log
Data dir       : /var/lib/redis/6379
Executable     : /usr/local/redis/bin/redis-server
Cli Executable : /usr/local/redis/bin/redis-cli
Is this ok? Then press ENTER to go on or Ctrl-C to abort.
Copied /tmp/6379.conf => /etc/init.d/redis_6379
Installing service...
Successfully added to chkconfig!
Successfully added to runlevels 345!
Starting Redis server...
Installation successful!

配置管理主机

• 在管理主机manager1(192.168.1.20)上配置脚本运行环境(redis-3.2.1.gem需要提前准备好)

[root@manager1 ~]# yum install -y rubygems
[root@manager1 ~]# gem install redis-3.2.1.gem

• 部署集群管理脚本

[root@manager1 ~]# tar xf redis-4.0.8.tar.gz
[root@manager1 ~]# cp redis-4.0.8/src/redis-trib.rb /usr/local/bin/
[root@manager1 ~]# chmod +x /usr/local/bin/redis-trib.rb

# 查看帮助
[root@manager1 ~]# redis-trib.rb help

创建集群

• 启动redis1的集群功能

# 停止服务
[root@redis1 ~]# service redis_6379 stop
Stopping ...
Redis stopped

# 修改配置文件
[root@redis1 ~]# vim /etc/redis/6379.conf
protected-mode no        # 关闭保护模式，以允许不使用密码、不指定绑定地址提供服务
# requirepass guodong.com    # 不使用密码
cluster-enabled yes      # 启用集群功能
cluster-config-file nodes-6379.conf   # 集群配置文件位置
cluster-node-timeout 5000  # 心跳时间

# 清空原有数据
[root@redis1 ~]# rm -rf /var/lib/redis/6379/*

# 修改服务启动文件
[root@redis1 ~]# vim +43 /etc/init.d/redis_6379
... ...
            $CLIEXEC -p $REDISPORT shutdown
... ...

# 启动服务
[root@redis1 ~]# service redis_6379 start
Starting Redis server...

# 查看端口，集群服务运行在16379端口上
[root@redis1 ~]# ss -tlnp | grep redis
LISTEN     0      128          *:16379                    *:*                   users:(("redis-server",pid=18952,fd=10))
LISTEN     0      128          *:6379                     *:*                   users:(("redis-server",pid=18952,fd=7))
LISTEN     0      128       [::]:16379                 [::]:*                   users:(("redis-server",pid=18952,fd=9))
LISTEN     0      128       [::]:6379                  [::]:*                   users:(("redis-server",pid=18952,fd=6))

• 配置redis2、redis3、redis4、redis5、redis6的集群功能

# 为了操作方便，可以配置redis1到其他节点的免密登陆
[root@redis1 ~]# ssh-keygen 
[root@redis1 ~]# for i in {12..16}
> do
> ssh-copy-id 192.168.1.$I
> done

# 将redis1上编译好的redis拷贝到其他主机
[root@redis1 ~]# for i in 1{2..6}
> do
> scp -r /usr/local/redis 192.168.1.$i:/usr/local/
> done

# 在redis2、redis3、redis4、redis5、redis6上，将redis命令目录添加至PATH环境变量
[root@redis1 ~]# for i in 1{2..6}; do ssh 192.168.1.$i "echo 'export PATH=$PATH:/usr/local/redis/bin' >> /etc/bashrc"; done

# 将redis1上源码目录拷贝到其他主机
[root@redis1 ~]# for i in 1{2..6}; do scp -r redis-4.0.8/ 192.168.1.$i:/root/; done

# 分别在redis2、redis3、redis4、redis5、redis6上执行初始化服务器脚本
[root@nodeX ~]# cd redis-4.0.8/
[root@nodeX redis-4.0.8]# utils/install_server.sh

# 停止redis2、redis3、redis4、redis5、redis6上的redis服务
[root@redis1 ~]# for i in 1{2..6}; do ssh 192.168.1.$i service redis_6379 stop; done

# 拷贝redis1的配置文件到redis2、redis3、redis4、redis5、redis6
[root@redis1 ~]# for i in 1{2..6}; do scp /etc/redis/6379.conf 192.168.1.$i:/etc/redis/; done

# 清除各主机上的数据
[root@redis1 ~]# for i in 1{2..6}; do ssh 192.168.1.$i rm -rf /var/lib/redis/6379/*; done

# 启动redis2、redis3、redis4、redis5、redis6上的redis服务
[root@redis1 ~]# for i in 1{2..6}; do ssh 192.168.1.$i service redis_6379 start; done

• 在管理主机manager1上创建集群

[root@manager1 ~]# redis-trib.rb create --replicas 1 \
> 192.168.1.11:6379 192.168.1.12:6379 192.168.1.13:6379 \
> 192.168.1.14:6379 192.168.1.15:6379 192.168.1.16:6379

>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
192.168.1.11:6379
192.168.1.12:6379
192.168.1.13:6379
Adding replica 192.168.1.15:6379 to 192.168.1.11:6379
Adding replica 192.168.1.16:6379 to 192.168.1.12:6379
Adding replica 192.168.1.14:6379 to 192.168.1.13:6379
M: b1b8e906ea2c7b71e567485f02d0007ad902a5c5 192.168.1.11:6379
   slots:0-5460 (5461 slots) master
M: 7cd1b4dc246c4c11780be1186f118ce79794d182 192.168.1.12:6379
   slots:5461-10922 (5462 slots) master
M: 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc 192.168.1.13:6379
   slots:10923-16383 (5461 slots) master
S: c85a2a1474dea3fbf4ac4073c437b8eada04b806 192.168.1.14:6379
   replicates 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc
S: a49746ee742866f0db4cebbc2de17733c0ace5b8 192.168.1.15:6379
   replicates b1b8e906ea2c7b71e567485f02d0007ad902a5c5
S: 3c0b9fe153392132003c9707d5132647b59031d4 192.168.1.16:6379
   replicates 7cd1b4dc246c4c11780be1186f118ce79794d182
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 192.168.1.11:6379)
M: b1b8e906ea2c7b71e567485f02d0007ad902a5c5 192.168.1.11:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc 192.168.1.13:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 7cd1b4dc246c4c11780be1186f118ce79794d182 192.168.1.12:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: a49746ee742866f0db4cebbc2de17733c0ace5b8 192.168.1.15:6379
   slots: (0 slots) slave
   replicates b1b8e906ea2c7b71e567485f02d0007ad902a5c5
S: c85a2a1474dea3fbf4ac4073c437b8eada04b806 192.168.1.14:6379
   slots: (0 slots) slave
   replicates 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc
S: 3c0b9fe153392132003c9707d5132647b59031d4 192.168.1.16:6379
   slots: (0 slots) slave
   replicates 7cd1b4dc246c4c11780be1186f118ce79794d182
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

• 在管理主机上查看集群信息

[root@manager1 ~]# redis-trib.rb info 192.168.1.11:6379
192.168.1.11:6379 (b1b8e906...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.13:6379 (3dd1687f...) -> 0 keys | 5461 slots | 1 slaves.
192.168.1.12:6379 (7cd1b4dc...) -> 0 keys | 5462 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.

[root@manager1 ~]# redis-trib.rb check 192.168.1.11:6379
>>> Performing Cluster Check (using node 192.168.1.11:6379)
M: b1b8e906ea2c7b71e567485f02d0007ad902a5c5 192.168.1.11:6379
   slots:0-5460 (5461 slots) master
   1 additional replica(s)
M: 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc 192.168.1.13:6379
   slots:10923-16383 (5461 slots) master
   1 additional replica(s)
M: 7cd1b4dc246c4c11780be1186f118ce79794d182 192.168.1.12:6379
   slots:5461-10922 (5462 slots) master
   1 additional replica(s)
S: a49746ee742866f0db4cebbc2de17733c0ace5b8 192.168.1.15:6379
   slots: (0 slots) slave
   replicates b1b8e906ea2c7b71e567485f02d0007ad902a5c5
S: c85a2a1474dea3fbf4ac4073c437b8eada04b806 192.168.1.14:6379
   slots: (0 slots) slave
   replicates 3dd1687f4c9d37a4bc095705a2ab0a35b27b59cc
S: 3c0b9fe153392132003c9707d5132647b59031d4 192.168.1.16:6379
   slots: (0 slots) slave
   replicates 7cd1b4dc246c4c11780be1186f118ce79794d182
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

• 在集群节点上查看集群信息

# 直接将服务器上的redis-cli拷贝到客户端即可
[root@redis1 ~]# scp /usr/local/redis/bin/redis-cli 192.168.1.10:/usr/local/bin

# 在客户端上登陆服务器，并查看集群状态
[root@node10 ~]# redis-cli -h 192.168.1.11
192.168.1.11:6379> PING
PONG

192.168.1.11:6379> CLUSTER INFO
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:954
cluster_stats_messages_pong_sent:895
cluster_stats_messages_sent:1849
cluster_stats_messages_ping_received:890
cluster_stats_messages_pong_received:954
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1849

访问集群

• 客户端可以连接任何一台集群服务器
• 存储数据时，根据算法，客户端自动重定向到指定服务器存储

[root@node10 ~]# redis-cli -c -h 192.168.1.11
192.168.1.11:6379> SET name tom
-> Redirected to slot [5798] located at 192.168.1.12:6379
OK

192.168.1.12:6379> SET gender male
-> Redirected to slot [15355] located at 192.168.1.13:6379
OK

192.168.1.13:6379> SET email tom@guodong.com
-> Redirected to slot [10780] located at 192.168.1.12:6379
OK

192.168.1.12:6379> SET phone 15011227878
OK

192.168.1.12:6379> SET address beijing
-> Redirected to slot [3680] located at 192.168.1.11:6379
OK