[JAVA-2561] Allow for limited threadpool size in the Async driver Created: 19/Jul/17  Updated: 29/Oct/23  Resolved: 08/Sep/17

Status: Closed
Project: Java Driver
Component/s: Async
Affects Version/s: None
Fix Version/s: 3.6.0

Type: New Feature Priority: Major - P3
Reporter: Mark Paluch Assignee: Ross Lawley
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Using MongoDB Reactive Streams driver 1.5



 Description   

Allow users to provide to limit the thread pool size or to supply an own ExecutorService.

Was:

Unlimited thread pool growth using AsynchronousSocketChannel

Mongo Driver async uses AsynchronousSocketChannel without channel grouping that leads to the creation of an unbounded default thread pool. The pool grows with concurrent connection use, and it's not possible to limit the thread pool size or to supply an own ExecutorService.

This arrangement can allocate a lot of resources which is not desirable in the particular scenario of nonblocking usage. It's not possible to limit resources or a strategy for back-pressure caused by higher workload than available resources (reject, queue, allocate resources). Supplying an ExecutorService via AsynchronousSocketChannelStreamFactoryFactory could help to address the issue.

The issue originated in MongoDB RS driver usage, hence I'm filing the ticket here and not in the JAVA (async) driver project.



 Comments   
Comment by Oleg Pudeyev (Inactive) [ 22/Jun/20 ]

Fix: https://github.com/mongodb/mongo-java-driver/commit/19336af05937e8a986cc583af6f86688bedf32a6

Comment by Frank Lyaruu [ 08/Sep/17 ]

Seems to work great with the Reactive Streams driver. Thanks!

Comment by Ross Lawley [ 08/Sep/17 ]

This has now been merged to master and will be released in 3.6

Comment by Jeffrey Yemin [ 07/Sep/17 ]

You can test this branch: https://github.com/rozza/mongo-java-driver/tree/j2561.

Much obliged,
Jeff

Comment by Frank Lyaruu [ 07/Sep/17 ]

I see there is code in review, is there a branch I can test with or do I need to wait for the code review?

Comment by Ross Lawley [ 04/Sep/17 ]

Marking this for the 3.6 release.

Note: once the work is committed into Master and passed the test suite it will automatically become available in the 3.6.0-SNAPSHOT build.

Comment by Frank Lyaruu [ 30/Aug/17 ]

Hi, are there any updates here? I ported some of our code to use the reactive driver as the regular one didn't scale well with large datasets. However as it stands now my application just eats up all the connections it can get.

Is there a nightly build I can use? Or some workaround to limit the number of connections?

Comment by Ross Lawley [ 19/Jul/17 ]

Hi mp911de,

Thanks for the ticket, as the fix is required in the Async driver I've moved into the Java project. Ticket looks good, we will review and schedule in due course.

Ross

Generated at Thu Feb 08 08:57:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.