[JAVA-906] Java driver version 2.11.2 not honoring connectionsPerHost setting Created: 30/Jul/13  Updated: 26/Nov/13  Resolved: 26/Nov/13

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: 2.11.2
Fix Version/s: None

Type: Bug Priority: Critical - P2
Reporter: Carl Hostrander Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File TestProgram.java    

 Description   

when utilizing the following code I am only see 3 connections opened on the server when I specify I want 100. Our mongo instance is a 3 server replicaset.

mongoClient = new MongoClient( 
	Arrays.asList(
	new ServerAddress("host1", 27017), 
	new ServerAddress("host2", 27017),
        new ServerAddress("host3", 27017)), 
new MongoClientOptions.Builder().connectionsPerHost(500).build());
			
db = mongoClient.getDB("db");
db.authenticate("username", "password".toCharArray());
 

we tried to set other settings as well, but they don't seem to be honored as well.

 
Builder builder = MongoClientOptions.builder();
builder.connectionsPerHost(100);
builder.readPreference(ReadPreference.nearest());
builder.socketFactory(SocketFactory.getDefault());
builder.socketKeepAlive(true);
builder.threadsAllowedToBlockForConnectionMultiplier(5);
builder.writeConcern(WriteConcern.UNACKNOWLEDGED);
 



 Comments   
Comment by Jeffrey Yemin [ 26/Nov/13 ]

Hi Carl,

We haven't heard back from you in a while, so I'm going to close this issue, but please let me know if there is anything more we can do to help.

Comment by Jeffrey Yemin [ 31/Jul/13 ]

In your code I see you're using unacknowledged writes. In that case adding threads won't help. Try it with WriteConcern.ACKNOWLEDGED (on the save call), and you should see the effect of adding threads. But you're not likely to get better throughput than you would with a single thread using WriteConcern.UNACKNOWLEDGED.

Comment by Carl Hostrander [ 31/Jul/13 ]

Hi Jeff,

I attached my sample program I used. I made 10 threads and saw 10 connections on the database. Upon running the program, I really didn't see any big improvements in performance. (Yes - that can be multiple issues)

Can you please take a look at the code, and verify if I am / am not using the driver correctly?

Comment by Jeffrey Yemin [ 30/Jul/13 ]

Hi Carl,

Yes, if you are using the default WriteConcern of ACKNOWLEDGED along with upsert, if you want to maximize throughput you will need to use multiple threads.

Comment by Carl Hostrander [ 30/Jul/13 ]

so my problem we are trying to solve:

1. mongoDB -> 525 million records in a collection
2. we have a file of json records daily ~40 million records that we must read and either insert or update.
3. looking at the connection on 3 connections open.

do I need to make my java program multi-threaded?

Comment by Carl Hostrander [ 30/Jul/13 ]

Hi Jeff,

Thanks for your response.

So my next question is how do I get the driver to open up 100 connections and use them?

Comment by Jeffrey Yemin [ 30/Jul/13 ]

Hi Carl,

connectionsPerHost represents the maximum number of connections that the driver will open and pool for each server, not the minimum. You can only get to that maximum if the number of threads concurrently using the MongoClient instance is >= connectionsPerHost.

Comment by Carl Hostrander [ 30/Jul/13 ]

My email for contact is : chostrander@dataium.com

Generated at Thu Feb 08 08:53:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.