[JAVA-193] perf slowdown when slaveOk() is set on the DB or Query object and then performing cursor.itcount() Created: 18/Oct/10  Updated: 29/Oct/10  Resolved: 20/Oct/10

Status: Closed
Project: Java Driver
Component/s: Performance
Affects Version/s: 2.2
Fix Version/s: 2.3

Type: Bug Priority: Minor - P4
Reporter: Alvin Richards (Inactive) Assignee: Eliot Horowitz (Inactive)
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File SlaveTest.java    

 Description   

Problem:
Seeing a large perf difference when the SlaveOk() is set (DB or cursor), e.g.

cur = coll.find(query).limit(20).addOption(Bytes.QUERYOPTION_SLAVEOK);

and then performing

count = cur.itcount()

Without SlaveOk()
Inserting data :
Querying data :
Queries : 121002
Rows found : 2420040
Finished :

With SlaveOk()
Inserting data :
Querying data :
Queries : 181
Rows found : 3620
Finished :

Reproduce:

  • create a two node replset
  • compile attached java code
    javac -classpath mongo-2.2.jar SlaveTest.java
  • run without SalveOk()
    java -cp mongo-2.2.jar SlaveTest "localhost:27000,localhost:27001" test blogs 0 1
  • run with slaveOk()
    "localhost:27000,localhost:27001,localhost:27002" test blogs 2 1

Business Case:

  • performance


 Comments   
Comment by Joseph Wang [ 22/Oct/10 ]

we see this problem running java 2.2 driver against mongodb-linux-x86_64-1.6.3.

Comment by Eliot Horowitz (Inactive) [ 20/Oct/10 ]

Not sure which other case fixed this - but the difference was 2.2 and head

Comment by Alvin Richards (Inactive) [ 19/Oct/10 ]

./software/mongodb-osx-x86_64-1.7.1/bin/mongod --fork --dbpath /data/db/db1 --port 27000 --replSet fred/
./software/mongodb-osx-x86_64-1.7.1/bin/mongod --fork --dbpath /data/db/db2 --port 27001 --replSet fred/localhost:27000
./software/mongodb-osx-x86_64-1.7.1/bin/mongod --fork --dbpath /data/db/db3 --port 27002 --replSet fred/localhost:27000

using 1.7.1

Comment by Eliot Horowitz (Inactive) [ 19/Oct/10 ]

Also what version of mongod

Comment by Eliot Horowitz (Inactive) [ 19/Oct/10 ]

That's irrelavent - will find all nods when it starts.

How are you starting the mongod?

Comment by Alvin Richards (Inactive) [ 19/Oct/10 ]

Looks like you just have a single host as a parameter. Just re-ran

"localhost:27000,localhost:27002" scaleout blogs 0 1

Inserting data :
Querying data :
Queries : 116517
Rows found : 2330340
Finished :

"localhost:27000,localhost:27002" scaleout blogs 2 1

Inserting data :
Querying data :
Queries : 181
Rows found : 3620
Finished :

Comment by Eliot Horowitz (Inactive) [ 19/Oct/10 ]

Can't seem to reproduce

$ javac SlaveTest2.java && java SlaveTest2 localhost:27017 test foo 0 1
Inserting data :
Querying data :
Queries : 120465
Rows found : 2409300
Finished :
$ javac SlaveTest2.java && java SlaveTest2 localhost:27017 test foo 2 1
Inserting data :
Querying data :
Queries : 123037
Rows found : 2460740
Finished :

Comment by Alvin Richards (Inactive) [ 18/Oct/10 ]

Workaround:

cursor.itcount() is really meant for testing purposes only, it iterates through the cursor. If you need a count which takes into account the limit/skip use size().

For the same data, size() gives the following perf numbers

Inserting data :
Querying data :
Queries : 325430
Rows found : 6508600
Finished :

Generated at Thu Feb 08 08:51:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.