redis灵魂拷问:怎样搭建一个哨兵主从集群

一直在使用redis，但是从来没有搭建过redis集群，今天来从0到1搭建一套redis哨兵主从集群。

实验环境准备

1.本地部署环境：

vmware虚机：2个(192.168.59.132和192.168.59.141)

操作虚机系统：centos7

192.168.59.132：部署一个redis实例作为主节点，端口6379，部署一个哨兵节点，端口26379

192.168.59.141：部署了2个实例，端口分别是6379和6389，作为192.168.59.132上实例的从节点，部署两个哨兵节点，端口26379和26389

redis版本：6.0.7

2.升级gcc

注意centos7默认gcc版本是4.8.5，但是安装6.0.7需要使用gcc版本是5.3以上，所以先升级gcc

yum -y install centos-release-sclyum -y install devtoolset-9-gcc devtoolset-9-gcc-c++ devtoolset-9-binutilsscl enable devtoolset-9 bash

这时我们再查看一下gcc版本，可以看到升级成功了：

[root@master redis-6.0.7]# gcc -vUsing built-in specs.COLLECT_GCC=gccCOLLECT_LTO_WRAPPER=/opt/rh/devtoolset-9/root/usr/libexec/gcc/x86_64-redhat-linux/9/lto-wrapperTarget: x86_64-redhat-linuxConfigured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/opt/rh/devtoolset-9/root/usr --mandir=/opt/rh/devtoolset-9/root/usr/share/man --infodir=/opt/rh/devtoolset-9/root/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --with-default-libstdcxx-abi=gcc4-compatible --enable-plugin --enable-initfini-array --with-isl=/builddir/build/BUILD/gcc-9.3.1-20200408/obj-x86_64-redhat-linux/isl-install --disable-libmpx --enable-gnu-indirect-function --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linuxThread model: posixgcc version 9.3.1 20200408 (Red Hat 9.3.1-2) (GCC) 

3.编译安装包

官网下载redis安装包后，我们进行安装

tar -xvf redis-6.0.7.tar.gzcd redis-6.0.7/makemake install PREFIX=/usr/local/redis

安装redis主从集群

安装前提示:

1.redis的日志路径和日志文件都需要创建好，redis不会主动创建

2.如果启动不指定redis.conf文件，会使用默认安装路径下的文件，在/usr/local/redis/etc这个目录

上一步编译成功后，我们修改主节点机器上(192.168.59.132)redis.conf文件，修改下面3行

#bind 127.0.0.1bind 19.168.59.132# requirepass foobaredrequirepass foobared#logfile ""logfile /root/redis/redis-6.0.7/logs/redis-6379.log

之后进入src目录，执行如下命令后启动成功：

[root@master redis-6.0.7]# ./src/redis-server redis.conf 37652:C 05 Sep 2020 06:22:39.156 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo37652:C 05 Sep 2020 06:22:39.156 # Redis version=6.0.7, bits=64, commit=00000000, modified=0, pid=37652, just started37652:C 05 Sep 2020 06:22:39.156 # Configuration loaded37652:M 05 Sep 2020 06:22:39.157 * Increased maximum number of open files to 10032 (it was originally set to 1024).                _._                                                             _.-``__ ''-._                                                   _.-``    `.  `_.  ''-._           Redis 6.0.7 (00000000/0) 64 bit  .-`` .-```.  ```\/    _.,_ ''-._                                    (    '      ,       .-`  | `,    )     Running in standalone mode |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379 |    `-._   `._    /     _.-'    |     PID: 37652  `-._    `-._  `-./  _.-'    _.-'                                    |`-._`-._    `-.__.-'    _.-'_.-'|                                   |    `-._`-._        _.-'_.-'    |           http://redis.io          `-._    `-._`-.__.-'_.-'    _.-'                                    |`-._`-._    `-.__.-'    _.-'_.-'|                                   |    `-._`-._        _.-'_.-'    |                                    `-._    `-._`-.__.-'_.-'    _.-'                                         `-._    `-.__.-'    _.-'                                                 `-._        _.-'                                                         `-.__.-'                                               
37652:M 05 Sep 2020 06:22:39.159 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.37652:M 05 Sep 2020 06:22:39.159 # Server initialized37652:M 05 Sep 2020 06:22:39.159 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.37652:M 05 Sep 2020 06:22:39.164 * Loading RDB produced by version 6.0.737652:M 05 Sep 2020 06:22:39.164 * RDB age 38 seconds37652:M 05 Sep 2020 06:22:39.164 * RDB memory usage when created 0.77 Mb37652:M 05 Sep 2020 06:22:39.164 * DB loaded from disk: 0.000 seconds37652:M 05 Sep 2020 06:22:39.164 * Ready to accept connections

使用客户端测试，可以看到已经启动成功：

[root@master redis-6.0.7]# ./src/redis-cli -h 192.168.59.132 -a foobaredWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.59.132:6379> keys *(empty array)

修改从节点机器(192.168.59.141)redis.conf文件，增加下面2行：

#下面内容新增，作为192.168.59.132的从节点，同时需要密码认证slaveof 192.168.59.132 6379masterauth foobared

安装好以后，我们在192.168.59.141这个虚机的redis解压目录下，copy一份redis.conf到redis6389.conf，修改下面3行然后增加2行从节点的配置：

#pidfile /var/run/redis_6379.pidpidfile /var/run/redis_6389.pid#port 6379port 6389#logfile ""logfile /root/redis/redis-6.0.7/logs/redis-6389.log#下面内容新增，作为192.168.59.132的从节点，同时需要密码认证slaveof 192.168.59.132 6379masterauth foobared

之后启动这2个redis实例，命令如下：

./src/redis-server redis.conf./src/redis-server redis6389.conf

启动成功后我们进入主节点客户端，查看从节点状态：

[root@master redis-6.0.7]# ./src/redis-cli -h 192.168.59.132 -a foobaredWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.59.132:6379> info replication# Replicationrole:masterconnected_slaves:2slave0:ip=192.168.59.141,port=6379,state=online,offset=1176,lag=0slave1:ip=192.168.59.141,port=6389,state=online,offset=1176,lag=1master_replid:bcd22e05007e87182251ab6b24601878c89f85f0master_replid2:0000000000000000000000000000000000000000master_repl_offset:1176second_repl_offset:-1repl_backlog_active:1repl_backlog_size:1048576repl_backlog_first_byte_offset:1repl_backlog_histlen:1176

到这里，集群就搭建成功了。我们写入一个数据：

192.168.59.132:6379> set foo 123456OK

之后我们到从节点查看，可以看到，数据同步成功：

[root@worker1 redis-6.0.7]# ./src/redis-cli -h 192.168.59.141 -p 6389 -a foobaredWarning: Using a password with '-a' or '-u' option on the command line interface may not be safe.192.168.59.141:6389> get foo"123456"

安装哨兵集群

哨兵的安装我选择在192.168.59.132安装一个实例，在192.168.59.141安装2个实例，192.168.59.132上sentinel.conf的配置修改内容如下：

# protected-mode noprotected-mode no# bind 127.0.0.1 192.168.1.1bind 192.168.59.132#daemonize nodaemonize yes#logfile ""logfile /root/redis/redis-6.0.7/logs/sentinel-26379.log#sentinel monitor <master-name> <ip> <redis-port> <quorum>sentinel monitor master 192.168.59.132 6379 2#sentinel failover-timeout mymaster 180000sentinel failover-timeout master 180000#sentinel down-after-milliseconds mymaster 30000sentinel down-after-milliseconds master 30000# sentinel auth-pass mymaster MySUPER--secret-0123passw0rdsentinel auth-pass master foobared192.168.59.141上sentinel26379.conf的修改跟上面一样，sentinel26389.conf的配置修改内容端口不一样，如下：#port 26379port 26389# protected-mode noprotected-mode no# bind 127.0.0.1 192.168.1.1bind 192.168.59.132#daemonize nodaemonize yes#logfile ""logfile /root/redis/redis-6.0.7/logs/sentinel-26389.log#sentinel monitor <master-name> <ip> <redis-port> <quorum>sentinel monitor master 192.168.59.132 6379 2#sentinel failover-timeout mymaster 180000sentinel failover-timeout master 180000#sentinel down-after-milliseconds mymaster 30000sentinel down-after-milliseconds master 30000# sentinel auth-pass mymaster MySUPER--secret-0123passw0rdsentinel auth-pass master foobared

配置好后，执行启动命令：

src/./redis-sentinel sentinel.confsrc/./redis-sentinel sentinel26379.confsrc/./redis-sentinel sentinel26389.conf

启动成功日志如下：

34887:X 05 Sep 2020 03:06:13.420 * Increased maximum number of open files to 10032 (it was originally set to 1024).34887:X 05 Sep 2020 03:06:13.421 * Running mode=sentinel, port=26379.34887:X 05 Sep 2020 03:06:13.421 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.34887:X 05 Sep 2020 03:06:13.422 # Sentinel ID is bd7bc3f82bef60606578a8aeb8bf1631e6d7f94134887:X 05 Sep 2020 03:06:13.422 # +monitor master master 192.168.59.132 6379 quorum 234887:X 05 Sep 2020 03:06:13.423 * +slave slave 192.168.59.141:6379 192.168.59.141 6379 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:06:13.424 * +slave slave 192.168.59.141:6389 192.168.59.141 6389 @ master 192.168.59.132 6379

可见，已经对2个redis从节点进行了监控。

测试

我们把redis主节点kill掉，查看哨兵日志：

34887:X 05 Sep 2020 03:12:09.696 # +sdown master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:09.799 # +odown master master 192.168.59.132 6379 #quorum 2/234887:X 05 Sep 2020 03:12:09.799 # +new-epoch 134887:X 05 Sep 2020 03:12:09.799 # +try-failover master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:09.802 # +vote-for-leader bd7bc3f82bef60606578a8aeb8bf1631e6d7f941 134887:X 05 Sep 2020 03:12:09.869 # bd7bc3f82bef60606578a8aeb8bf1631e6d7f941 voted for bd7bc3f82bef60606578a8aeb8bf1631e6d7f941 134887:X 05 Sep 2020 03:12:09.947 # +elected-leader master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:09.947 # +failover-state-select-slave master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.020 # +selected-slave slave 192.168.59.141:6389 192.168.59.141 6389 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.020 * +failover-state-send-slaveof-noone slave 192.168.59.141:6389 192.168.59.141 6389 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.097 * +failover-state-wait-promotion slave 192.168.59.141:6389 192.168.59.141 6389 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.400 # +promoted-slave slave 192.168.59.141:6389 192.168.59.141 6389 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.400 # +failover-state-reconf-slaves master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.469 * +slave-reconf-sent slave 192.168.59.141:6379 192.168.59.141 6379 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:10.945 # -odown master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:11.484 * +slave-reconf-inprog slave 192.168.59.141:6379 192.168.59.141 6379 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:11.484 * +slave-reconf-done slave 192.168.59.141:6379 192.168.59.141 6379 @ master 192.168.59.132 637934887:X 05 Sep 2020 03:12:11.540 # +failover-end master master 192.168.59.132 637934887:X 05 Sep 2020 03:12:11.540 # +switch-master master 192.168.59.132 6379 192.168.59.141 638934887:X 05 Sep 2020 03:12:11.541 * +slave slave 192.168.59.141:6379 192.168.59.141 6379 @ master 192.168.59.141 638934887:X 05 Sep 2020 03:12:11.541 * +slave slave 192.168.59.132:6379 192.168.59.132 6379 @ master 192.168.59.141 638934887:X 05 Sep 2020 03:12:41.597 # +sdown slave 192.168.59.132:6379 192.168.59.132 6379 @ master 192.168.59.141 6389

从日志中看出，master节点已经从192.168.59.132切换到了6379 192.168.59.141 6389，可见哨兵已经生效了。

我们在springboot中配置redis，配置如下：

#------redis配置------------#spring.redis.database=0#spring.redis.host=192.168.59.138#spring.redis.password=#spring.redis.port=6379#spring.redis.port=6379spring.redis.sentinel.master=master#配置哨兵节点spring.redis.sentinel.nodes=192.168.59.132:26379,192.168.59.141:26379,192.168.59.141:26389spring.redis.password=foobaredspring.redis.timeout=5000#最大连接数spring.redis.lettuce.pool.max-active=50#最大阻塞等待时间spring.redis.lettuce.pool.max-wait=5000#连接池中最大空闲连接spring.redis.lettuce.pool.max-idle=50#连接池中最小空闲连接spring.redis.lettuce.pool.min-idle=5spring.redis.lettuce.pool.time-between-eviction-runs=1spring.main.allow-bean-definition-overriding=truespring.jackson.serialization.FAIL_ON_EMPTY_BEANS=false

下面代码输出12345678，可以看到集群已经可以使用

@Testpublic void testGet(){    //Assert.assertEquals(redisTemplate.opsForValue().get("zhujinjun"), "123456");    redisTemplate.opsForValue().set("foo", "12345678");    System.out.println(redisTemplate.opsForValue().get("foo"));}

上面源代码地址：

https://github.com/jinjunzhu/spring-boot-mybatis.git

总结

本文主要讲述了redis哨兵主从集群的搭建，这个过程中要注意一些参数的设置。搭建好集群后就可以做各种试验了，欢迎感兴趣的朋友们一起。