[SERVER-45577] Submit range deletion tasks using fixed executor Created: 14/Jan/20 Updated: 29/Oct/23 Resolved: 12/Feb/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Sharding |
| Affects Version/s: | None |
| Fix Version/s: | 4.3.4 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Esha Maharishi (Inactive) | Assignee: | Jack Mulrow |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Sprint: | Sharding 2020-02-10, Sharding 2020-02-24 | ||||
| Participants: | |||||
| Linked BF Score: | 0 | ||||
| Description |
| Comments |
| Comment by Githook User [ 11/Feb/20 ] |
|
Author: {'username': 'jsmulrow', 'name': 'Jack Mulrow', 'email': 'jack.mulrow@mongodb.com'}Message: |
| Comment by Esha Maharishi (Inactive) [ 10/Feb/20 ] |
|
jack.mulrow I didn't totally follow what you're saying about the CatalogCacheLoaderMock, but if you have an idea, go for it.
Edit: Oh I see, using the CatalogCacheLoaderMock instead of the real CatalogCache would allow us to avoid the deadlock. |
| Comment by Jack Mulrow [ 07/Feb/20 ] |
|
I tried using a real ThreadPool and wasn't able to make it work, but I noticed we already have a CatalogCacheLoaderMock class, which I think I can use to get around this issue instead, since it's the calls through the CatalogCache to ShardRemote::_runExhaustiveCursorCommand() that use the fixed executor and are causing the unit tests to hang when we also use the fixed executor to schedule the deletion. I'll give this approach a shot next week. |
| Comment by Esha Maharishi (Inactive) [ 03/Feb/20 ] |
|
Hmm, that's true, I forgot about the arbitrary TaskExecutor TODO. I don't think we should leave it as the arbitrary executor, since unlike the fixed executor, which uses a ThreadPool with unlimited size, the arbitrary executors use a NetworkInterfaceThreadPool, which I believe uses a single thread. I checked if it's possible to make unit tests use a fixed executor with a larger thread pool size, but it doesn't seem easy to do this, since unit tests use an executor with a ThreadPoolMock, which seems to have an explicit single worker thread. Maybe we can change the unit tests to use a fixed executor with a real ThreadPool, like initial_syncer_test.cpp does? I'm not sure if this would work, but could you give this a try and see if it's possible? |