[SERVER-975] make ClientCusror::yield do an adaptive sleep (WAS: Improve yielding for remove queries) Created: 06/Apr/10  Updated: 12/Jul/16  Resolved: 06/Apr/10

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 1.4.1, 1.5.0

Type: Improvement Priority: Major - P3
Reporter: Marc Seeger Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

ubuntu 9.10 64 bit, mongodb 1.4


Participants:

 Description   

I'm currently runnning this query:

{
"opid" : 24472544,
"active" : true,
"lockType" : "write",
"waitingForLock" : false,
"secs_running" : 6600,
"op" : "remove",
"ns" : "mydb.mycoll",
"query" : {
"entry" :

{ "$ne" : "string" }

},
"client" : "127.0.0.1:48464",
"desc" : "conn"
},

It's now taking > 100 minutes (and still a long way to go) for a 17 million documents collection.

The main problem is that it seems to completely lock the application and that it does not yield.
When running .stats() or .count() from the mongo shell, it actually does let 3-4 inserts through, but only to retake the lock and keep it after a few seconds.

This is what I usually see in mongostats:
insert/s query/s update/s delete/s getmore/s command/s mapped vsize res % locked % idx miss conn time
0 0 0 0 0 2 49161 49967 5892 0.979 0.0131 14 17:01:28
0 0 0 0 0 2 49161 49967 5891 0.935 0.0102 14 17:01:29
0 0 0 0 0 2 49161 49967 5890 1.08 0.00866 14 17:01:30
0 0 0 0 0 2 49161 49967 5888 0.999 0.0106 14 17:01:31
0 0 0 0 0 2 49161 49967 5886 0.977 0.0112 14 17:01:32
0 0 0 0 0 2 49161 49967 5885 1.01 0.00845 14 17:01:33
0 0 0 0 0 2 49161 49967 5884 0.964 0.0147 14 17:01:34
0 0 0 0 0 2 49161 49967 5885 1.04 0.0121 14 17:01:35
0 0 0 0 0 2 49161 49967 5884 0.986 0.0147 14 17:01:36
0 0 0 0 0 2 49161 49967 5883 1.03 0.0121 14 17:01:37
0 0 0 0 0 2 49161 49967 5881 0.986 0.0117 14 17:01:38
0 0 0 0 0 2 49161 49967 5880 0.958 0.0123 14 17:01:39
0 0 0 0 0 2 49161 49967 5878 1.03 0.0133 14 17:01:40
0 0 0 0 0 2 49161 49967 5878 0.975 0.0124 14 17:01:41
0 0 0 0 0 2 49161 49967 5876 1.05 0.0122 14 17:01:42
0 0 0 0 0 2 49161 49967 5878 0.931 0.0151 14 17:01:43



 Comments   
Comment by Eliot Horowitz (Inactive) [ 26/Apr/10 ]

in a release

Comment by auto [ 13/Apr/10 ]

Author:

{'login': 'erh', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

Message: adaptive sleep for ClientCusror::yield SERVER-975
http://github.com/mongodb/mongo/commit/f8339b000d80c35248a5c7a7e6a8fe2182702f6c

Comment by auto [ 06/Apr/10 ]

Author:

{'login': '', 'name': 'Eliot Horowitz', 'email': 'eliot@10gen.com'}

Message: adaptive sleep for ClientCusror::yield SERVER-975
http://github.com/mongodb/mongo/commit/009e2e4e0a1c1bbcfcc1acb4b18f035dee7b31fb

Comment by Marc Seeger [ 06/Apr/10 ]

3-4 inserts ONLY once I manually call .count() from the shell. Otherwise I can watch it for 5 minutes without anything else happening

here is the currentOP: http://pastebin.com/6uC2ek17

Comment by Eliot Horowitz (Inactive) [ 06/Apr/10 ]

right - you said it let 3-4 inserts in.
so i'm unclear what exactly the issue is.

do you think other ops are having to wait too long?
can you send db.currentOp sample output?

Comment by Marc Seeger [ 06/Apr/10 ]

It was my impression that the "remove data" step shouldn't be blocking but should, in between, let other queries/inserts/updates do their thing.
I have TONS of operations waiting (from other processes etc)

Comment by Eliot Horowitz (Inactive) [ 06/Apr/10 ]

I'm not sure what you mean by improved yielding.
Do you have operations waiting?

It sounds like its doing exactly what its supposed to.

remove some data, then let other ops in, then continue...

Generated at Thu Feb 08 02:55:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.