[SERVER-2998] mongos does not guarantee ordering between getmores and unchecked write operations run on the same connection Created: 26/Apr/11 Updated: 12/Dec/23 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | features we're not sure of |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | Aaron Staple | Assignee: | Backlog - Cluster Scalability |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | RachitaD | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Cluster Scalability
|
| Participants: |
| Description |
|
For example: c = t.find( {$or:[ {a:2}, {b:3}, {c:4}]} ).batchSize( 2 ); ); The first getmore, caused by itcount(), may happen either before or after the remove if mediated by mongos. The following workaround is currently needed: c = t.find( {$or:[ {a:2}, {b:3}, {c:4}]} ).batchSize( 2 ); ); Here is an example logging expected behavior from the above script: m30001| Mon Apr 25 23:31:42 [conn4] query test.jstests_or5 ntoreturn:2 reslen:102 nreturned:2 0ms 0ms , { b: 3.0 }, { c: 4.0 } ] } bytes:86 nreturned:2 0ms , { b: 3.0 }, { c: 4.0 }] } bytes:20 nreturned:0 0ms Here is an example of unexpected behavior that sometimes occurs: m30001| Mon Apr 25 23:11:51 [conn4] query test.jstests_or5 ntoreturn:2 reslen:102 nreturned:2 0ms , { b: 3.0 }, { c: 4.0 } ] } bytes:86 nreturned:2 0ms , { b: 3.0 }, { c: 4.0 } ] } bytes:86 nreturned:2 0ms , { b: 3.0 }, { c: 4.0 } ] } bytes:20 nreturned:0 0ms 0ms As far as I know there is a single connection from mongo to mongos in these examples, but there seem to be two connections from mongos to mongod and this causes the unexpected operation ordering. |
| Comments |
| Comment by Eliot Horowitz (Inactive) [ 27/Apr/11 ] |
|
@andrew, no, it doesn't violate read your own writes. writes after an initial query before other getMore calls might not be seen, but you can't rely on that since you don't know the batch sizes, etc... |
| Comment by Andrew Armstrong [ 27/Apr/11 ] |
|
I'm not sure if this is entirely related to the problem, but I know from experience with MySQL in the past that without explicitly providing an 'order by' clause, if you attempt to page this result set (such as LIMIT 0, 20..., LIMIT 20, 20, ....) you may re-read previous records because the underlying data has changed (or just happened to be fetched differently on your second query), so paging an unordered result set (like your above example) has undefined behavior I suppose. Our fix for that particular problem was to simply just order by an id field (something consistent that was indexed) which then did provide consistent behavior. PS: Isn't your example violating the 'read your own writes' property that is currently expected? Cheers |
| Comment by Aaron Staple [ 26/Apr/11 ] |
|
Just wanted to add a note that we've seen similar behavior with ordering between an unchecked write and an admin command. Apparently the admin commands use a separate connection from mongos to mongod. |
| Comment by Eliot Horowitz (Inactive) [ 26/Apr/11 ] |
|
Ithink its as designed. |
| Comment by Dwight Merriman [ 26/Apr/11 ] |
|
it's normal for getmores to be lazy, so i wonder, is this actually a problem or works as designed? |
| Comment by auto [ 26/Apr/11 ] |
|
Author: {u'login': u'astaple', u'name': u'Aaron', u'email': u'aaron@10gen.com'}Message: SERVER-2998 workarounds |