背景
- 在这篇文章中做过使用del命令删除大key的实验,结果是del命令随着key的增大,主线程阻塞的时间就越长。
- 这与之前看redis5.0.8版本的代码中关于多线程删除操作的感官不符,于是决定先查看redis关于删除操作的代码,找出关键点,再做实验进行验证。
准备工作
- 需要复用这篇文章中,使用过的数据构造方法和测试脚本代码。
代码分析步骤
- 在server.c中找到redisCommandTable(命令表,redis的所有命令都对应这张表中的一个回调函数),找到del命令对应的回调函数delCommand。查看delCommand函数的代码内容如下:
-
Java
void delCommand(client *c) { delGenericCommand(c,0); }
- delCommand继续只是单纯的调用了一个通用删除方法delGenericCommand,继续追踪delGenericCommand,代码如下:
-
Java
/* This command implements DEL and LAZYDEL. */ void delGenericCommand(client *c, int lazy) { int numdel = 0, j; for (j = 1; j < c->argc; j++) { expireIfNeeded(c->db,c->argv[j]); int deleted = lazy ? dbAsyncDelete(c->db,c->argv[j]) : dbSyncDelete(c->db,c->argv[j]); if (deleted) { signalModifiedKey(c->db,c->argv[j]); notifyKeyspaceEvent(NOTIFY_GENERIC, "del",c->argv[j],c->db->id); server.dirty++; numdel++; } } addReplyLongLong(c,numdel); }
- 该方法就比较有意思了,首先注释就说这个命令实现了删除和懒删除(也就是我们想要的异步删除)。再看此行代码:
-
Java
int deleted = lazy ? dbAsyncDelete(c->db,c->argv[j]) : dbSyncDelete(c->db,c->argv[j]);
- 也就是说根据参数lazy的不同而调用异步删除或同步删除。那么那里调用了这个方法,传递了什么参数呢?根据ide的追踪,发现有2个函数调用了当前函数,且lazy参数一个传0,一个传1,刚好对应了一个同步,一个异步。
-
Java
void delCommand(client *c) { delGenericCommand(c,0); } void unlinkCommand(client *c) { delGenericCommand(c,1); }
- 其中,delCommand是del命令的回调,unlinkCommand是unlink命令的回调。这就说明unlink命令才会让redis采用异步删除的方式。
实验验证
- 构造数据和测试php脚本请参考这篇文章。
- 首先采用redis-cli --pipe的方式向redis添加一个key为sigkey的hash数据,其hlen为3000万,占用内存约1.8G。
- 向redis添加一个key为m的string类型数据,用于测试获取。
- 使用redis-cli连接redis。
- 开启测试php脚本。
- redis-cli中执行命令unlink ligkey。
- 关闭观测php脚本,观测结果。
实验结果
- 有出现标记信号的输出如下:
-
Java
0.57013500 1643333929--$4 mack -------------------xxx--------------------- 0.70107700 1643333929--$4 mack 0.80816900 1643333929--$4 mack 0.92466400 1643333929--$4 mack 0.03091900 1643333930--$4 mack 0.13540800 1643333930--$4 mack 0.23995700 1643333930--$4 mack 0.34635700 1643333930--$4 mack 0.45889300 1643333930--$4 mack 0.56787100 1643333930--$4 mack -------------------xxx--------------------- 0.69811500 1643333930--$4 mack
- 可以看到出现波动的两次间隔均与0.13秒相差不大,且出现这两个波动的时间节点与执行删除命令不同(执行命令的时候观测并未出现标记),可以确认为正常网络波动,排除unlink阻塞影响。
总结
- 根据源码分析及实际实验操作可得,del命令使用同步删除,unlink使用异步删除。
- 在删除数据体量很小的简单类型时建议使用del命令,在删除大key时应该使用unlink命令。
- 删除小key使用del的原因是:虽然del是同步删除,会阻塞主线程,但是unlink同样会在主线程执行一些判断和其它操作。而这些操作可能带来的开销比实际删除一个小key还略大。所以能直接删的key就没必要使用异步删除了。
补充
- 我们还可以根据需要,通过配置的方式使得某些命令进行全局的异步操作。
-
Java
############################# LAZY FREEING #################################### # Redis has two primitives to delete keys. One is called DEL and is a blocking # deletion of the object. It means that the server stops processing new commands # in order to reclaim all the memory associated with an object in a synchronous # way. If the key deleted is associated with a small object, the time needed # in order to execute the DEL command is very small and comparable to most other # O(1) or O(log_N) commands in Redis. However if the key is associated with an # aggregated value containing millions of elements, the server can block for # a long time (even seconds) in order to complete the operation. # # For the above reasons Redis also offers non blocking deletion primitives # such as UNLINK (non blocking DEL) and the ASYNC option of FLUSHALL and # FLUSHDB commands, in order to reclaim memory in background. Those commands # are executed in constant time. Another thread will incrementally free the # object in the background as fast as possible. # # DEL, UNLINK and ASYNC option of FLUSHALL and FLUSHDB are user-controlled. # It's up to the design of the application to understand when it is a good # idea to use one or the other. However the Redis server sometimes has to # delete keys or flush the whole database as a side effect of other operations. # Specifically Redis deletes objects independently of a user call in the # following scenarios: # # 1) On eviction, because of the maxmemory and maxmemory policy configurations, # in order to make room for new data, without going over the specified # memory limit. # 2) Because of expire: when a key with an associated time to live (see the # EXPIRE command) must be deleted from memory. # 3) Because of a side effect of a command that stores data on a key that may # already exist. For example the RENAME command may delete the old key # content when it is replaced with another one. Similarly SUNIONSTORE # or SORT with STORE option may delete existing keys. The SET command # itself removes any old content of the specified key in order to replace # it with the specified string. # 4) During replication, when a replica performs a full resynchronization with # its master, the content of the whole database is removed in order to # load the RDB file just transferred. # # In all the above cases the default is to delete objects in a blocking way, # like if DEL was called. However you can configure each case specifically # in order to instead release memory in a non-blocking way like if UNLINK # was called, using the following configuration directives: lazyfree-lazy-eviction no lazyfree-lazy-expire no lazyfree-lazy-server-del no replica-lazy-flush no