[SERVER-57267] Dereference of invalid iterator in connection pool Created: 27/May/21 Updated: 29/Oct/23 Resolved: 04/Jun/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 5.0.0-rc3, 5.1.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | George Wangensteen | Assignee: | George Wangensteen |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | servicearch-wfbf-day | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Operating System: | ALL | ||||||||||||
| Backport Requested: |
v5.0
|
||||||||||||
| Steps To Reproduce: | I found this by inspection + looking at the core dump generated by the failure of jstests/sharding/query/merge_write_concern.js in the ASAN build variant of this patch build: https://spruce.mongodb.com/version/6080a2f7a4cf47408799d65c/tasks which was done in the linked server ticket as a test of our ability to upgrade abseil. See the linked SERVER ticket for more details. |
||||||||||||
| Sprint: | Service Arch 2021-06-14 | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
In ConnectionPool::dropConnections here, we used a range-based for loop to iterate through the ConnectionPool's _pools member, a. stdx::unordered_map. Within the loop, we call ConnectionPool::SpecificPool::triggerShutdown on each specific pool we find in the _pools map. This member function will erase this specific pool from the parent ConnectionPool's _pools member here. According to the std::unordered_map documentation for the "erase" overload set, "References and iterators to the erased elements are invalidated. Other iterators and references are not invalidated." While the abseil documentation is a bit unclear, I think the same applies for the absl::node_hash_map we're actually using (and, in any case, we should probably live by the standard library documentation for stdx:: types). This means that once the specific pool element is erased in triggerShutdown, any iterators to it are invalidated – which implies when that when the ranged-based for loop attempts to advance to the next element in the map by calling next() on the iterator, it will be using/dereferencing an invalid iterator. Our current version of abseil seems to leave this iterator in a valid state long enough for the for-loop to advance, but this is not guaranteed: when we attempted an upgrade to the newest version of abseil (see |
| Comments |
| Comment by Vivian Ge (Inactive) [ 06/Oct/21 ] |
|
Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you! |
| Comment by Githook User [ 17/Jun/21 ] |
|
Author: {'name': 'George Wangensteen', 'email': 'george.wangensteen@mongodb.com', 'username': 'gewa24'}Message: (cherry picked from commit 6598a5843ed7318527c8b60bdaaacfe312bd5c51) |
| Comment by Githook User [ 04/Jun/21 ] |
|
Author: {'name': 'George Wangensteen', 'email': 'george.wangensteen@mongodb.com', 'username': 'gewa24'}Message: |