Describe the bug
Upgrading the redis-server from 7.2.7 to 7.4.2, there are obvious performance degradation of SREM and ZREMRANGEBYLEX command. The main source of these performance degradation should be commit e2b7932.
To reproduce
Compile the redis-server before and after e2b7932, and using memtier_benchmark to test the SREM command:
memtier_benchmark "--data-size" "10" --pipeline 50 --command "SREM test_set value_set:159 value_set:1 value_set:91 value_set:72 value_set:134 value_set:13 value_set:93 value_set:3" --command-key-pattern="P" --key-minimum=1 --key-maximum 10000 --test-time 180 -c 50 -t 4 --hide-histogram
before e2b7932
ALL STATS
==================================================================================================
Type Ops/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
--------------------------------------------------------------------------------------------------
Srems 1316307.41 7.58762 7.77500 11.07100 12.41500 237809.44
Totals 1316307.41 7.58762 7.77500 11.07100 12.41500 475618.89
after e2b7932
ALL STATS
==================================================================================================
Type Ops/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
--------------------------------------------------------------------------------------------------
Srems 1294339.95 7.71696 7.93500 11.26300 12.54300 233840.71
Totals 1294339.95 7.71696 7.93500 11.26300 12.54300 467681.43
Also use memtier_benchmark to test the ZREMRANGEBYLEX command:
memtier_benchmark "--data-size" "10" --pipeline 50 --command "ZREMRANGEBYLEX test_zadd1 [sortset105 [sortset146" --command-key-pattern="P" --key-minimum=1 --key-maximum 10000 --test-time 180 -c 50 -t 4 --hide-histogram
before e2b7932
ALL STATS
==========================================================================================================
Type Ops/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
----------------------------------------------------------------------------------------------------------
Zremrangebylexs 1712281.15 5.83172 5.53500 9.72700 11.19900 137116.26
Totals 1712281.15 5.83172 5.53500 9.72700 11.19900 274232.53
after e2b7932
ALL STATS
==========================================================================================================
Type Ops/sec Avg. Latency p50 Latency p99 Latency p99.9 Latency KB/sec
----------------------------------------------------------------------------------------------------------
Zremrangebylexs 1679914.16 5.94317 6.04700 9.21500 10.62300 134524.38
Totals 1679914.16 5.94317 6.04700 9.21500 10.62300 269048.75
Expected behavior
Can this degradation be eliminated or alleviated?
Additional information
I performed instrumentation on the processCommand function in redis-server to measured its execution time. Before and after e2b7932, with SREM command, the average execution time of the processCommand is 2.096us and 2.233us, respectively. And with ZREMRANGEBYLEX command, the average execution time of the processCommand is 1.857us and 1.938us, respectively.
Comment From: sundb
@Gallopm did you populate data before the benchmark?
Comment From: Gallopm
Yes, you can use the following python script to populate similiar data:
import redis
import random
import string
def random_string(length=10):
letters = string.ascii_letters
return ''.join(random.choice(letters) for i in range(length))
def load_data_into_redis(host='localhost', port=6379, num_keys=10000):
r = redis.Redis(host=host, port=port, db=0)
# Prepare data for SREM test, totally num_keys + 2
for i in range(num_keys):
key = f"key_set:{i}"
random_number = random.randint(15, 199)
for j in range(random_number):
value = random_string(50)
r.sadd(key, value)
random_SPOP_Num = random.randint(160, 299)
for i in range(random_SPOP_Num):
r.sadd('test_SPOP', random_string(50))
value = f"value_set:{i}"
r.sadd('test_set', value)
# Prepare data for ZREMRANGEBYLEX test, totally num_keys + 1
for i in range(num_keys):
key = f"key_SortSet:{i}"
random_number = random.randint(100, 199)
for j in range(random_number):
numSorce = random.randint(1, 15)
value = random_string(50)
mapping = {value: numSorce}
r.zadd(key, mapping)
random_ZADD_Num = random.randint(160, 299)
for i in range(random_ZADD_Num):
value = f"sortset{i}"
numSorce = random.randint(1, 20)
mapping = {value: numSorce}
r.zadd('test_zadd1', mapping)
if __name__ == "__main__":
load_data_into_redis(num_keys=10000)