[SERVER-10435] Compound index on hashmap field results in inconsistent count Created: 05/Aug/13 Updated: 29/Apr/15 Resolved: 29/Apr/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Index Maintenance |
| Affects Version/s: | 2.2.1, 2.4.5 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | arun j | Assignee: | David Hows |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Windows 7, CentOS, Mac OS |
||
| Attachments: |
|
| Operating System: | ALL |
| Steps To Reproduce: | The repro steps are as per the above description |
| Participants: |
| Description |
|
I have a document with a field "ia", it's a HashMap (with key value pair) in my java entity. The mongo document PlayerAttribute looks like this:
This is the index definition for the collection,
This is the query I used from mongo shell,
First time it gives me the result greater than zero (but that is incorrect. For eg., if there are 7 docs match the query, it would say 6) and the next subsequent times it gives 0 and this happens for 5 to 10 minutes. After about 5 to 10 minutes, it gives the non zero number again (one time) and next few times it is zero. When I deleted this index ("name" : "pa_gid_iattr_n_idx"), it gives correct count all the time. Any idea why this index on hashmap field ("name" : "pa_gid_iattr_n_idx") makes mongo to produce incorrect and inconsistent result? Thanks in advance for the response. |
| Comments |
| Comment by David Hows [ 15/Aug/13 ] | |
|
Hi Arun, That was my hypothesis and why I wanted to have a look at a few example documents (to gauge size) and see the explain output. If you are only interested in specific sub elements that are within the "ia" subdocument why not index those directly? There are some examples in our indexing documentation For the query you have shown me you could use something like this:
Regards, | |
| Comment by arun j [ 13/Aug/13 ] | |
|
Hi David, Since the query uses the ia field, do you have any recommendations on how to handle this query efficiently? Thanks | |
| Comment by David Hows [ 13/Aug/13 ] | |
|
Hi Arun, That looks like a big difference, were you able to run those with the .explain() instead of the count so that wen can see if there are any obvious differences in the results from those two queries? Would it be possible for you to find those 6 matching documents and post them to this ticket? You should be able to find them by using a "hint" on another index, or temporarily dropping the bad index as you mentioned earlier. Regards, | |
| Comment by arun j [ 12/Aug/13 ] | |
|
Hi David, I've attached the mongodb log as per your request. Please look at line # 969 (see below) that returns the count > 0, nreturned is 6, this is the first time when I run the query, Mon Aug 12 11:15:27.879 [conn1] query player.PlayerAttribute query: { gameId: "com.glu.stuntracing", ia.Registered: "true" }ntoreturn:0 ntoskip:0 nscanned:94 keyUpdates:0 locks(micros) W:13289 r:15879 nreturned:6 reslen:10125 16ms And executing the same query again returns count 0 always, please see below from the log, nreturned is 0, Mon Aug 12 11:15:41.802 [conn1] query player.PlayerAttribute query: { gameId: "com.glu.stuntracing", ia.Registered: "true" }ntoreturn:0 ntoskip:0 nscanned:23 keyUpdates:0 locks(micros) r:4192 nreturned:0 reslen:20 4ms | |
| Comment by arun j [ 12/Aug/13 ] | |
|
Please look at line # 969 (see below) that returns the count > 0, nreturned is 6, Mon Aug 12 11:15:27.879 [conn1] query player.PlayerAttribute query: { gameId: "com.glu.stuntracing", ia.Registered: "true" }ntoreturn:0 ntoskip:0 nscanned:94 keyUpdates:0 locks(micros) W:13289 r:15879 nreturned:6 reslen:10125 16ms And executing the same query again returns count 0 always, please see below from the log, nreturned is 0, Mon Aug 12 11:15:41.802 [conn1] query player.PlayerAttribute query: { gameId: "com.glu.stuntracing", ia.Registered: "true" }ntoreturn:0 ntoskip:0 nscanned:23 keyUpdates:0 locks(micros) r:4192 nreturned:0 reslen:20 4ms | |
| Comment by arun j [ 12/Aug/13 ] | |
|
Hi Stephen, Thanks for pointing out the query inconsistency, below is the actual query that has the issue, db.PlayerAttribute.find( {"gameId":"XYZ", "ia.CrashesCount" : "0"}).count(); And I am not doing any numeric range comparision in my query, all I am doing is a string match. Let me know if you need any additional information. Thanks | |
| Comment by Stennie Steneker (Inactive) [ 12/Aug/13 ] | |
|
Hi Arun, You've listed two versions of MongoDB (2.2.1, 2.4.5) and three O/S (Windows 7, CentOS, Mac OS) in the details for this issue. Can you please be specific on which combination of MongoDB and O/S this issue can be reproduced with? I would also note that:
If you can confirm the environment versions and provide the additional information requested by David we can investigate further. Thanks, | |
| Comment by David Hows [ 12/Aug/13 ] | |
|
Hi Arun, How big do those hashes get? Can you attach the logs from the node in question? I suspect that there may be issues with the index in question, such as problems creating or updating entries. Can you please run the command with a .explain() instead of a .count()? That will tell us what is going on with the index selection and return results of the queries. Regards, | |
| Comment by arun j [ 09/Aug/13 ] | |
|
Any thoughts on this issue? |