[GODRIVER-392] Best way to connect to sharded cluster Created: 30/Apr/18  Updated: 27/Oct/23  Resolved: 02/May/18

Status: Closed
Project: Go Driver
Component/s: Connections, Documentation
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Trivial - P5
Reporter: Sebastian Bauer Assignee: Jeffrey Yemin
Resolution: Works as Designed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified


 Description   

Going to move from mgo to this Go driver. Having a sharded cluster with replica sets for each shard.

What I did with mgo (and in C++ implementations as well): Connecting directly to every single replica set member without mongos to have better control of my connections and load of each node.

I'm wondering if this is a good approach or there is a better best practise. Thought about connecting to shards and let the driver handle the replica sets. But I will lose some of the control with this.



 Comments   
Comment by Sebastian Bauer [ 02/May/18 ]

You are right. Regarding mongos this is a general question and I asked on Stack Overflow (for reference): https://stackoverflow.com/questions/50136885/speedup-broadcast-finds-with-mongos

Comment by Jeffrey Yemin [ 02/May/18 ]

Hi Sebastian. It seems your question is no longer really about the Go driver but about how to get the performance you need out of a sharded cluster regardless of which driver you happen to be using. If that's the case, the best place to ask questions about general MongoDB usage is either the mongodb-user Google Group or Stack Overflow as public discussion there can help others learn and resolve similar issues.

That said, I'm going to close this issue, but feel free to comment back if you feel I've mischaracterized your question.

Comment by Sebastian Bauer [ 02/May/18 ]

Since we are reading millions of records the mongos approach is pretty slow compared to direct connections. It's not 100% clear to me if mongos is caching before sending the result. Connection is with secondaryPreferred on a local mongos.

Using mongos would be pretty cool regarding client complexity. But performance seems to be a problem.

Comment by Sebastian Bauer [ 30/Apr/18 ]

Thank you, Jeff, for your feedback. I'm a bit concerned using mongos for all this traffic. We are talking about a setup with more than 40 shards.

To use all available replica set members, there is no read preference like "round robin". Even better would be to balance to the nodes that has most free resources. That's why we are using this manual approach right now.

Most queries will be broadcast operations. I will check if there are performance impacts with big results.

Comment by Jeffrey Yemin [ 30/Apr/18 ]

Our advice is to connect only to mongos routers in a sharded cluster for CRUD operations.  Any other usage pattern is not supported.  See https://docs.mongodb.com/manual/core/sharded-cluster-query-router/ for details.  

Which driver your application happens to use is not really relevant to this decision.

Generated at Thu Feb 08 08:34:06 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.