-
Type: New Feature
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: 1.5.3
-
Component/s: Performance
-
None
mongoexport is potentially very slow when passing in a query argument.
Below is an example of a shell find() that takes 30ms for 571 result documents, but many minutes when trying to export these documents using mongoexport.
If it is known for sure that a collection won't change during an export, can mongoexport be told to be more efficient and less strict when creating the output documents? Possibly it's a simple matter of allowing {$snapshot: false} to be specificed, or maybe there is another reason for the large change in efficiency.
Either way, providing a simple query which uses an index such as the example given, is currently an unfeasible way to export collection subsets, and one must rely on other manual means to achieve this.
-
- Simple query in shell
> db.articles.find( {"authors.name": "cannon cp"}).explain()
{ "authors.name" : "cannon cp" }
{
"cursor" : "BtreeCursor authors.name_1",
"nscanned" : 571,
"nscannedObjects" : 571,
"n" : 571,
"millis" : 30,
"indexBounds" : [
[
,
{ "authors.name" : "cannon cp" }]
]
}
- Simple query in shell
-
- Same query in mongoexport takes multiple minutes with high nscanned
> ./mongoexport -vv -h localhost -c articles -d pubmed -q ' {"authors.name": "cannon cp"}' -o articles.json
connected to: localhost
exported 571 records
- Same query in mongoexport takes multiple minutes with high nscanned
-
- Server output
Thu Jul 8 14:58:51 query pubmed.articles reslen:174620 nscanned:1387320Unknown macro: { query}nreturned:101 39276ms
Thu Jul 8 15:03:23 getmore pubmed.articles cid:7259446948513230975 getMore:Unknown macro: { query}bytes:716901 nreturned:470 271669ms
Thu Jul 8 15:03:24 end connection 127.0.0.1:54779
- Server output