[JAVA-1884] Allow limiting the amount of concurrent queries executed in the async driver Created: 10/Jul/15 Updated: 06/Dec/17 Resolved: 06/Dec/17 |
|
| Status: | Closed |
| Project: | Java Driver |
| Component/s: | Async |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Peter Hendriks | Assignee: | Ross Lawley |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
When using the async driver, we have found we cannot limit the amount of concurrent queries executed in the async execution model. Although we'd assumed limiting the amount of connections in the connection pool would limit the amount of queries possible in flight, it turns out, this is not how it works out all the time. When the query returns a larger amount of results than the batch size (20 by default), the retrieval of results in the async model will yield the connection back to the pool after each batch has executed. In this case, an additional query request may start, and the number of concurrent queries may become larger than the connection pool. We need a model to limit the amount of concurrent queries possible to prevent memory issues when too many queries would be processed at once. This a common use case, it would be very useful to support directly in the mongo async driver. |
| Comments |
| Comment by Ross Lawley [ 06/Dec/17 ] |
|
I'm closing this as "Won't Fix" as limiting the threadpool size by providing your own Executor service as per There are no current plans to limit the number of queries in flight. If Ross |
| Comment by Ross Lawley [ 04/Sep/17 ] |
|
Hi peterhendriks, Apologies for the lack of response on this ticket, it appears to have fallen through the cracks. I think providing a limit to the thread pool size or being able to provide your own ExecutorService should allow for finer grained control over memory resources. Do you agree? See: Ross |