Notice! - If a Redis module was involved, please open an issue in the module's repo instead! - If you're using docker on Apple M1, please make sure the image you're using was compiled for ARM!

Crash report

Paste the complete crash log between the quotes below. Please include a few lines from the log preceding the crash report to provide some context. operat

Actions: step1: start redis server on the server; step2: on the other server , execute benchmark,commands as follow: command1: ./memtier_benchmark -s90.90.54.74 -p 6379 --data-size 5120 --ratio 0:1 --key-pattern P:P --key-minimum=1 --key-maximum 3000000 --distinct-client-seed --test-time 120 -c 800 -t 1 --pipeline=100 --hide-histogram

command 2: ./memtier_benchmark -s90.90.54.74 -p 6379--data-size 5120 --ratio 1:0 --key-pattern P:P --key-minimum=1 --key-maximum 3000000 --distinct-client-seed --test-time 120 -c 800 -t 1 --pipeline=100 --hide-histogram step3 : excute command1, then excute ctrl+c , excute command2 ,then excute ctrl+c

step4:sometime produces coredump

crash reason is the content of server.clients_index varable changed, here is rax.c log info as follow: such as some child raxnode adrress is illegal,as 0xffffffffff98

unlinkClient client id:4497 fd: 507 args id:0 rax.c:raxRemove:1041: ### Delete: Lookup current node: 0xffffb8c48690 [] key:0 size:6 children:0xffffb8c2f300 Lookup current node: 0xffffb8c2f300 [ ] key:0 size:9 children:0xffffb8c9ee90 0xffffafcc1900 0xffffb8c1bfe0 0xffffb4261350 0xffffb4262640 0xffffb8c1bfc0 0xffffb8c32150 0xffffaee7c870 0xffffaf9654d0 Lookup current node: 0xffffaee7c870 [ ;Œ‘œ°¼] key:0 size:7 children:0xffffb8c71ce0 0xffffb8c72260 0xffffb8c9ef20 0xffffb8cb3ff0 0xffffb4440ba0 0xffffffffff98 0xffffb4557b60 Lookup stop node is: 0xffffb8cb3ff0 [] key:1 size:0 children: rax.c:raxRemove:1064: Key deleted in node without children. Cleanup needed. rax.c:raxRemove:1068: Freeing child 0xffffb8cb3ff0 [] key:0 rax.c:raxRemove:1078: Unlinking child 0xffffb8cb3ff0 from parent 0xffffaee7c870 raxRemoveChild before: 0xffffaee7c870 [

=== REDIS BUG REPORT START: Cut & paste starting from here === 2770809:M 20 Aug 2025 15:21:05.289 # Redis 7.0.15 crashed by signal: 11, si_code: 1 2770809:M 20 Aug 2025 15:21:05.289 # Accessing address: 0xffffffffff98 2770809:M 20 Aug 2025 15:21:05.289 # Crashed running the instruction at: 0x52e4d4

------ STACK TRACE ------ EIP: ./src/redis-server :6392[0x52e4d4] Backtrace: linux-vdso.so.1(__kernel_rt_sigreturn+0x0)[0xffffb94e694c] ./src/redis-server :6392[0x52e4d4] ./src/redis-server :6392(raxRemove+0xac)[0x5303fc] ./src/redis-server :6392(unlinkClient+0xc8)[0x470d50] ./src/redis-server :6392(freeClient+0x2a4)[0x4713f0] ./src/redis-server :6392(beforeNextClient+0x50)[0x471918] /usr/lib64/libkbaio.so(readQueryFromClient+0x2e0)[0xffffb9394388] ./src/redis-server :6392[0x54c5dc] ./src/redis-server :6392[0x54ced4] ./src/redis-server :6392(aeProcessEvents+0x2f4)[0x445a60] ./src/redis-server :6392(aeMain+0x24)[0x445d34] ./src/redis-server :6392(main+0xe74)[0x45e520] /usr/lib64/libc.so.6(+0x2b000)[0xffffb91db000] /usr/lib64/libc.so.6(__libc_start_main+0x94)[0xffffb91db0d8] ./src/redis-server :6392(_start+0x30)[0x43fb30]

------ REGISTERS ------ 2770809:M 20 Aug 2025 15:21:05.290 # X18:0000000000000006 X19:00000000005e9f40 X20:0000000000000001 X21:000000000067fda0 X22:000000000043fb34 X23:0000ffffb934f000 X24:0000000000000000 X25:0000ffffc9f8cbf8 X26:000000000067fda0 X27:0000000000000000 X28:0000000000000000 X29:0000ffffc9f8c540 X30:000000000052e440 pc:000000000052e4d4 sp:0000ffffc9f8c540 pstate:0000000060000000 fault_address:0000ffffffffff98

2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54f) -> 0000ffffb411f880 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54e) -> 0000000000000008 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54d) -> 0000000000000000 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54c) -> 0000ffffb411f87c 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54b) -> 0000ffffb411f880 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c54a) -> 0000ffffffffff98 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c549) -> 0000ffffb8c56d00 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c548) -> 0000ffffc9f8c7d8 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c547) -> 0000000000000008 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c546) -> 0000ffffc9f8c730 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c545) -> 0000000000000000 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c544) -> 0000ffffc9f8c60c 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c543) -> 0000ffffc9f8c610 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c542) -> 0000ffffc9f8c570 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c541) -> 00000000005303fc 2770809:M 20 Aug 2025 15:21:05.290 # (0000ffffc9f8c540) -> 0000ffffc9f8c5c0

------ INFO OUTPUT ------

Server

redis_version:7.0.15 redis_git_sha1:ea749154 redis_git_dirty:1 redis_build_id:5d0b733ccdbc38ae redis_mode:standalone os:Linux 5.10.0-216.0.0.115.oe2203sp4.aarch64 aarch64 arch_bits:64 monotonic_clock:POSIX clock_gettime multiplexing_api:kbaio atomicvar_api:c11-builtin gcc_version:10.3.1 process_id:2770809 process_supervised:no run_id:d5ffa95e0c37947a324dec25cc264ef13f614f5a tcp_port:6392 server_time_usec:1755674465289397 uptime_in_seconds:32 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:10844001 executable:redis-7.0.15/./src/redis-server config_file:redis-7.0.15/./redis.conf io_threads_active:0

Clients

connected_clients:788 cluster_connections:0 maxclients:10000 client_recent_max_input_buffer:49194 client_recent_max_output_buffer:410080 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory

used_memory:393934176 used_memory_human:375.68M used_memory_rss:461533184 used_memory_rss_human:440.15M used_memory_peak:656184840 used_memory_peak_human:625.79M used_memory_peak_perc:60.03% used_memory_overhead:11374205 used_memory_startup:1021432 used_memory_dataset:382559971 used_memory_dataset_perc:97.37% allocator_allocated:394293624 allocator_active:404451328 allocator_resident:502824960 total_system_memory:539414835200 total_system_memory_human:502.37G used_memory_lua:31744 used_memory_vm_eval:31744 used_memory_lua_human:31.00K used_memory_scripts_eval:0 number_of_cached_scripts:0 number_of_functions:0 number_of_libraries:0 used_memory_vm_functions:32768 used_memory_vm_total:64512 used_memory_vm_total_human:63.00K used_memory_functions:184 used_memory_scripts:184 used_memory_scripts_human:184B maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.03 allocator_frag_bytes:10157704 allocator_rss_ratio:1.24 allocator_rss_bytes:98373632 rss_overhead_ratio:0.92 rss_overhead_bytes:-41291776 mem_fragmentation_ratio:1.17 mem_fragmentation_bytes:67292648 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_total_replication_buffers:0 mem_clients_slaves:0 mem_clients_normal:7697981 mem_cluster_links:0 mem_aof_buffer:0 mem_allocator:jemalloc-5.2.1 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence

loading:0 async_loading:0 current_cow_peak:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:126380 rdb_bgsave_in_progress:0 rdb_last_save_time:1755674433 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_saves:0 rdb_last_cow_size:0 rdb_last_load_keys_expired:0 rdb_last_load_keys_loaded:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_rewrites:0 aof_rewrites_consecutive_failures:0 aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0

Stats

total_connections_received:6369 total_commands_processed:2123755 instantaneous_ops_per_sec:6500 total_net_input_bytes:727775660 total_net_output_bytes:695690443 total_net_repl_input_bytes:0 total_net_repl_output_bytes:0 instantaneous_input_kbps:219.69 instantaneous_output_kbps:2609.48 instantaneous_input_repl_kbps:0.00 instantaneous_output_repl_kbps:0.00 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:0 evicted_keys:0 evicted_clients:0 total_eviction_exceeded_time:0 current_eviction_exceeded_time:0 keyspace_hits:137418 keyspace_misses:1859957 pubsub_channels:0 pubsub_patterns:0 pubsubshard_channels:0 latest_fork_usec:0 total_forks:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 total_active_defrag_time:0 current_active_defrag_time:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:0 dump_payload_sanitizations:0 total_reads_processed:291626 total_writes_processed:201573 io_threaded_reads_processed:0 io_threaded_writes_processed:0 reply_buffer_shrinks:2827 reply_buffer_expands:0

Replication

role:master connected_slaves:0 master_failover_state:no-failover master_replid:fc9fc3341ab88353525f4f334c84f0bc8ca9ec77 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

CPU

used_cpu_sys:23.944659 used_cpu_user:5.441947 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000722 used_cpu_sys_main_thread:1.222561 used_cpu_user_main_thread:5.462103

Modules

Commandstats

cmdstat_get:calls=1997375,usec=1520855,usec_per_call=0.76,rejected_calls=0,failed_calls=0 cmdstat_set:calls=126380,usec=313228,usec_per_call=2.48,rejected_calls=0,failed_calls=0

Errorstats

Latencystats

latency_percentiles_usec_get:p50=0.001,p99=8.031,p99.9=13.055 latency_percentiles_usec_set:p50=2.007,p99=6.015,p99.9=16.063

Cluster

cluster_enabled:0

Keyspace

db0:keys=53258,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------

Additional information

1. OS distribution and version
2. Steps to reproduce (if any)

Comment From: sundb

@wtwzxl thx, i saw multiplexing_api:kbaio in your report. is it a modified version?

Comment From: wtwzxl

@wtwzxl thx, i saw multiplexing_api:kbaio in your report. is it a modified version?

yes

I read redis 7.0.15 source codes and found only two modifications to server.client_index , createClient->linkClient 和 freeClient->unlinkClient , Redis is single-threaded, so linkClient and unlinkClient are executed in a serial manner. Therefore, I can't figure out why a child node in server.client_index has turned into an invalid address.

Comment From: sundb

it seems that the stored pointer 0xffffffffff98 is already incorrect, and was not corrupted during deletion. So you should check if there was any problem with linkClient.