If the master uses diskless synchronization and the replica uses diskless load, this means no RDB file persists on disk, the compression of RDB costs much CPU and time both on master and replica, and network is much fast, not a bottleneck. So maybe we can disable RDB compression to reduce full sync time.
I tested on AWS, there are two EC2, master enabled diskless sync, replica enabled diskless load.
I used memtier-benchmark to generate 21GB data, all values are repeated x, this means all data is very easy to compress, the RDB file is about 635MB (enabling compression by default).
When we disabled RDB compression, the full sync costed 36s, but it costed 45s when enabling RDB compression.
then I used redis-benchmark to generate 24GB data, the value is a random string, so the data is very hard to compress, the RDB is about 20GB even enabling compression.
When we disabled RDB compression, the full sync costed 44s, but it costed 75s when enabling RDB compression.
Therefore, regardless of whether the data is suitable for compression, disabling compression can always reduce the time of full synchronization.
since rdbcompression configuration is designed for RDB file, it seems not a good idea to disable it directly, maybe the replica can ask the master to deliver a RDB without compression by replconf rdb-no-compress 1 command, when the replica can use diskless load.
Comment From: tezc
If we place the master and replica on the same machine, no-compression method becomes 3-4 times faster. It suggests that bandwidth is the main bottleneck now. The above tests show %20-40 gains but I suspect we might see even better numbers on networks with higher bandwidth between nodes.
The downside is that disabling compression increases network traffic but as it gives 20–40% boost (and potentially more under better conditions), seems like this is good trade-off.
@oranagra wdyt?