redis-shake-在rockylinux8上数据同步测试
目录
1.概述
1.1redis-shake软件概述
1.1.1redisshake基本介绍
RedisShake是基于redis-port基础上进行改进的是一款开源的Redis迁移工具，支持Cluster集群的在线迁移与离线迁移（备份文件导入）。数据可平滑迁移，当部署在其他云厂商Redis服务上的Cluster集群数据，由于SYNC、PSYNC命令被云厂商禁用，无法在线迁移时，可以选择离线迁移。
1.1.2redisshake使用背景
RedisShake是基于redis-port基础上进行改进的是一款开源的Redis迁移工具，支持Cluster集群的在线迁移与离线迁移（备份文件导入）。数据可平滑迁移，当部署在其他云厂商Redis服务上的Cluster集群数据，由于SYNC、PSYNC命令被云厂商禁用，无法在线迁移时，可以选择离线迁移。
1.1.3redisshake功能说明
RedisShake是基于redis-port基础上进行改进的是一款开源的Redis迁移工具，支持Cluster集群的在线迁移与离线迁移（备份文件导入）。数据可平滑迁移，当部署在其他云厂商Redis服务上的Cluster集群数据，由于SYNC、PSYNC命令被云厂商禁用，无法在线迁移时，可以选择离线迁移。
1.1.4redisshake特性概览
高性能：全量同步阶段并发执行，增量同步阶段异步执行，能够达到毫秒级别延迟（取决于网络延迟）。同时，我们还对大key同步进行分批拉取，优化同步性能。
在 Redis 5.0、Redis 6.0 和 Redis 7.0 上测试
支持使用lua自定义过滤规则
支持大实例迁移
支持restore模式和sync模式
支持阿里云 Redis 和 ElastiCache
监控体系：用户可以通过我们提供的restful拉取metric来对redis-shake进行实时监控：curl 127.0.0.1:9320/metric。
数据校验：如何校验同步的正确性？可以采用我们开源的redis-full-check。
支持版本：支持2.8-5.0版本的同步，此外还支持codis，支持云下到云上，云上到云上，云上到云下（阿里云目前支持主从版），其他云到阿里云等链路，帮助用户灵活构建混合云场景。
断点续传。支持断开后按offset恢复，降低因主备切换、网络抖动造成链路断开重新同步拉取全量的性能影响。
1.2测试环境概述
在两台机器创建两个redis服务，一个redis6，另一个redis7，在其中一个添加数据，配置redis-shake进行数据同步测试；
|
序号 |
机器名 |
ip地址 |
操作系统版本 |
基础环境 |
|
1 |
redis6 |
10.9.254.31 |
RockyLinux8.7 |
redis6、redis-shake |
|
2 |
redis7 |
10.9.254.32 |
RockyLinux8.7 |
redis7 |
2.环境准备
2.1下载redis
https://download.redis.io/releases/
https://download.redis.io/releases/redis-6.0.0.tar.gz
https://download.redis.io/releases/redis-7.0.0.tar.gz
|
[root@redis6 ~]# wget https://download.redis.io/releases/redis-6.0.0.tar.gz [root@redis6 ~]# ll |
|
-rw-r--r-- 1 root root 2204177 Jun 27 2020 redis-6.0.0.tar.gz |
|
[root@redis7 ~]# wget https://download.redis.io/releases/redis-7.0.0.tar.gz [root@redis7 ~]# ll |
|
-rw-r--r-- 1 root root 2943054 Apr 27 2022 redis-7.0.0.tar.gz |
2.2安装redis
安装redis6，相同方法在另一台安装redis7；
|
[root@redis6 ~]# yum -y install wget gcc tcl make |
|
Upgraded: cpp-8.5.0-22.el8_10.x86_64 gcc-8.5.0-22.el8_10.x86_64 gcc-c++-8.5.0-22.el8_10.x86_64 libgcc-8.5.0-22.el8_10.x86_64 libgomp-8.5.0-22.el8_10.x86_64 libstdc++-8.5.0-22.el8_10.x86_64 libstdc++-devel-8.5.0-22.el8_10.x86_64 wget-1.19.5-12.el8_10.x86_64 Installed: make-1:4.2.1-11.el8.x86_64 tcl-1:8.6.8-2.el8.x86_64 Complete! |
|
[root@redis6 ~]# tar -zxvf redis-6.0.0.tar.gz [root@redis6 ~]# cd redis-6.0.0 [root@redis6 ~/redis-6.0.0]# make |
|
LINK redis-server INSTALL redis-sentinel CC redis-cli.o LINK redis-cli CC redis-benchmark.o LINK redis-benchmark INSTALL redis-check-rdb INSTALL redis-check-aof Hint: It's a good idea to run 'make test' ;) make[1]: Leaving directory '/root/redis-6.0.0/src' |
|
[root@redis6 ~/redis-6.0.0]# cd /root/redis-6.0.0/src [root@redis6 ~/redis-6.0.0/src]# make install |
|
CC Makefile.dep Hint: It's a good idea to run 'make test' ;) INSTALL install INSTALL install INSTALL install INSTALL install INSTALL install |
2.3配置redis
|
[root@redis6 ~]# mkdir -p /usr/local/redis/etc/ [root@redis6 ~]# mkdir -p /usr/local/redis/bin [root@redis6 ~]# cd /root/redis-6.0.0/ [root@redis6 ~/redis-6.0.0]# cp redis.conf /usr/local/redis/etc/ [root@redis6 ~/redis-6.0.0]# cd /root/redis-6.0.0/src [root@redis6 ~/redis-6.0.0/src]# cp mkreleasehdr.sh redis-benchmark redis-check-aof redis-check-rdb redis-cli redis-server /usr/local/redis/bin/ [root@redis6 ~/redis-6.0.0/src]# vim /usr/local/redis/etc/redis.conf |
|
bind 0.0.0.0 daemonize yes requirepass "P@88w0rd" |
|
[root@redis6 ~/redis-6.0.0/src]# vim /etc/profile |
|
export REDIS_HOME=/usr/local/redis export PATH=$PATH:$REDIS_HOME/bin/ |
|
[root@redis6 ~/redis-6.0.0/src]# source /etc/profile [root@redis6 ~/redis-6.0.0/src]# redis-server /usr/local/redis/etc/redis.conf |
|
15019:C 05 Dec 2024 17:23:23.593 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 15019:C 05 Dec 2024 17:23:23.593 # Redis version=6.0.0, bits=64, commit=00000000, modified=0, pid=15019, just started 15019:C 05 Dec 2024 17:23:23.593 # Configuration loaded |
|
[root@redis6 ~/redis-6.0.0/src]# netstat -ntlp |
|
Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 15020/redis-server tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 949/sshd tcp6 0 0 :::22 :::* LISTEN 949/sshd |
|
[root@redis6 ~/redis-6.0.0/src]# redis-server --version |
|
Redis server v=6.0.0 sha=00000000:0 malloc=jemalloc-5.1.0 bits=64 build=0 |
设置脚本控制启停
|
[root@redis6 ~]# vim /usr/local/redis/bin/redis-start |
|
#/bin/bash /usr/local/redis/bin/redis-server /usr/local/redis/etc/redis.conf |
|
[root@redis6 ~]# vim /usr/local/redis/bin/redis-stop |
|
#!/bin/bash echo "redis-server will stop" redis_pid=$(ps -ef |grep -v 'grep' | egrep /usr/local/redis/bin/redis-server | awk '{printf $2 " "}'); if [ "$redis_pid" != "" ]; then kill -9 "$redis_pid" echo "redis-server stop" else echo "can not find redis-server" fi exit 1 |
3.安装配置redisshake
3.1下载redisshake
https://github.com/tair-opensource/RedisShake/releases/tag/v4.2.2
|
[root@redis6 ~]# wget https://github.com/tair-opensource/RedisShake/releases/download/v4.2.2/redis-shake-linux-amd64.tar.gz [root@redis6 ~]# ll |
|
-rw-r--r-- 1 root root 6636344 Dec 5 13:22 redis-shake-linux-amd64.tar.gz |
3.2安装redisshake
|
[root@redis6 ~]# tar -zxvf redis-shake-linux-amd64.tar.gz |
|
./redis-shake ./shake.toml |
|
[root@redisshake ~]# ll |
|
-rwxr-xr-x 1 1001 docker 11739313 Nov 19 15:16 redis-shake -rw-r--r-- 1 root root 6636344 Dec 5 13:22 redis-shake-linux-amd64.tar.gz -rw-r--r-- 1 1001 docker 6002 Nov 19 15:15 shake.toml |
|
[root@redis6 ~]# mv redis-shake /usr/local/bin/ [root@redis6 ~]# ll /usr/local/bin/ |
|
-rwxr-xr-x 1 1001 127 11739313 Nov 19 15:16 redis-shake |
3.3配置redisshake
|
[root@redis6 ~]# cp shake.toml shake.toml.bak [root@redis6 ~]# vim shake.toml |
|
[sync_reader] address = "10.9.254.31:6379" password = "P@88w0rd" [redis_writer] address = "10.9.254.32:6379" password = "P@88w0rd" |
4.同步测试
4.1源端创建测试数据
|
[root@redis6 ~]# redis-cli -h 10.9.254.31 -p 6379 -a P@88w0rd |
|
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.9.254.31:6379> ping PONG 10.9.254.31:6379> SET key1 value1 OK |
4.2执行redis数据迁移
|
[root@redis6 ~]# redis-shake ./shake.toml |
|
2024-12-09 13:55:40 INF load config from file: ./shake.toml 2024-12-09 13:55:40 INF log_level: [info], log_file: [/root/data/shake.log] 2024-12-09 13:55:40 INF changed work dir. dir=[/root/data] 2024-12-09 13:55:40 INF GOMAXPROCS defaults to the value of runtime.NumCPU [2] 2024-12-09 13:55:40 INF not set pprof port 2024-12-09 13:55:40 INF create SyncStandaloneReader: 10.9.254.31:6379 2024-12-09 13:55:40 INF create RedisStandaloneWriter: 10.9.254.32:6379 2024-12-09 13:55:40 INF start syncing... 2024-12-09 13:55:40 INF [reader_10.9.254.31_6379] source db is not doing bgsave! continue. 2024-12-09 13:55:45 INF read_count=[1], read_ops=[0.00], write_count=[1], write_ops=[0.00], syncing aof, diff=[0] 2024-12-09 13:55:50 INF read_count=[1], read_ops=[0.20], write_count=[1], write_ops=[0.20], syncing aof, diff=[0] 2024-12-09 13:55:55 INF read_count=[1], read_ops=[0.00], write_count=[1], write_ops=[0.00], syncing aof, diff=[0] 2024-12-09 13:56:00 INF read_count=[1], read_ops=[0.00], write_count=[1], write_ops=[0.00], syncing aof, diff=[0] 2024-12-09 13:56:05 INF read_count=[1], read_ops=[0.00], write_count=[1], write_ops=[0.00], syncing aof, diff=[0] |
4.3验证迁移结果
|
[root@redis7 ~]# redis-cli -h 10.9.254.32 -p 6379 -a P@88w0rd |
|
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.9.254.32:6379> ping PONG 10.9.254.32:6379> GET key1 "value1" |
手动在源redis中添加数据再次验证
|
[root@redis6 ~]# redis-cli -h 10.9.254.31 -p 6379 -a P@88w0rd Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.9.254.31:6379> ping PONG 10.9.254.31:6379> SET key2 value2 OK |
|
[root@redis7 ~]# redis-cli -h 10.9.254.32 -p 6379 -a P@88w0rd Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe. 10.9.254.32:6379> ping PONG 10.9.254.32:6379> GET key2 "value2" |
汇聚全球AI编程工具，助力开发者即刻编程。
更多推荐
13
0- 0
已为社区贡献4条内容
所有评论(0)