-
Type:
Bug
-
Resolution: Done
-
Priority:
Critical - P2
-
None
-
Affects Version/s: 1.8.1
-
Component/s: Querying
-
Labels:None
-
Environment:Driver: Java 2.6.3
CentOS release 5.6
(Final), running 2.6.22.19-vs2.3.0.34.1 x86_64. The server is a VM
running on the same physical hardware (though a different VM) as the
web application querying the DB. File system where the data is stored
is ufs with plenty of free space. Total system memory is 32GB, mostly
used by mongo with 1.5GB free.Driver: Java 2.6.3 CentOS release 5.6 (Final), running 2.6.22.19-vs2.3.0.34.1 x86_64. The server is a VM running on the same physical hardware (though a different VM) as the web application querying the DB. File system where the data is stored is ufs with plenty of free space. Total system memory is 32GB, mostly used by mongo with 1.5GB free.
-
ALL
The behavior we are seeing
is that when we run a query, it will most of the time return a set of
results that make sense, and then once out of 1000 queries, it will be
missing a single record. I've tried sorting the results to see if
it's always the same time, using a snapshot query, setting the Java
client batch size to Integer.MAX_VALUE, and using the toArray() method
on the Java client DBCursor object to get it to quickly pull the
data. I've also removed any indexes that the queries might be using,
so it doesn't appear to be an issue with indexes. We are only querying
the master, though we saw this when querying slaves as well, though
the data missing was very old and it wouldn't have been a replication
issue.
I am only able to reproduce this issue on our production servers
(running Linux), and even with copying all of the files locally, I
have been unable to reproduce this issue on my local machine (using
replicates on Mac and Windows). All versions of the DB are 1.8.1.
I can reproduce the issue in the shell as well, though not as easily.
I turned on profiling logs and here
is the output for the 3 queries (full with 523 results, missing with
522 results, full again with 523 results):
{ "ts" : ISODate("2011-06-24T02:31:23.438Z"), "info" : "query
pacman.packages ntoreturn:2147483647 reslen:619920 nscanned:7110
\nquery: { query: { cluster: \"c1\", installableToEnvironment:
, deployments: { $elemMatch:
{ environmentType: \"STAGING\", $or: [
,
{ uninstallDate:
} ] } } }, orderby:
{ _id: 1 }} nreturned:523 bytes:619904", "millis" : 17 }
{ "ts" : ISODate("2011-06-24T02:31:23.924Z"), "info" : "query
pacman.packages ntoreturn:2147483647 reslen:618733 nscanned:7110
\nquery: { query: { cluster: \"c1\", installableToEnvironment:
, deployments: { $elemMatch:
{ environmentType: \"STAGING\", $or: [
,
{ uninstallDate:
} ] } } }, orderby:
{ _id: 1 }} nreturned:522 bytes:618717", "millis" : 19 }
{ "ts" : ISODate("2011-06-24T02:31:24.354Z"), "info" : "query
pacman.packages ntoreturn:2147483647 reslen:619920 nscanned:7110
\nquery: { query: { cluster: \"c1\", installableToEnvironment:
, deployments: { $elemMatch:
{ environmentType: \"STAGING\", $or: [
,
{ uninstallDate:
} ] } } }, orderby:
{ _id: 1 }} nreturned:523 bytes:619904", "millis" : 19 }