[SERVER-72281] Provide Antithesis with tsan & asan enabled binaries Created: 20/Dec/22 Updated: 12/Dec/23 Resolved: 30/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Adrian Ryan | Assignee: | Trevor Guidry |
| Resolution: | Duplicate | Votes: | 0 |
| Labels: | correctness-releaseability-working-group | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Correctness
|
||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
In order to try to improve our testing of MongoDB's server, we want to look into using Mongo binaries with tsan and/or asan enabled. We'd first like to get versions of these binaries so that we can run some tests on our end, to understand whether these help us find issues and what performance implications they have inside our platform. With these binaries we can run a controlled experiment & present a report on what we think the best use of testing is moving forward. |
| Comments |
| Comment by Ryan Berryhill [ 30/Nov/23 ] |
|
This ticket is now subsumed by |
| Comment by Judah Schvimer [ 09/Nov/23 ] |
|
I split asan out to |
| Comment by Max Hirschhorn [ 03/Nov/23 ] |
|
Yes and I would recommend starting out with a --sanitize=address binary first. And after we've gotten it running well in Antithesis infrastructure we can switch for a --sanitize=address,undefined binary next. I expect the --sanitize=address binary to be sufficient to ensure things like ASAN_OPTIONS environment variables and ulimit (also systemd?) virtual memory limits are configured appropriately. UBSan doesn't allocate shadow memory like ASan and TSan do and so it should be a much easier extension once we've done the first step. |
| Comment by Trevor Guidry [ 03/Nov/23 ] |
|
My understanding is that ASAN and UBSAN can be combined but TSAN is standalone. |
| Comment by javi Arguello [ 02/Nov/23 ] |
|
Awesome. Thanks trevor.guidry@mongodb.com . Let me know if there's anything you need from me in terms of pushing the images to our registry |
| Comment by javi Arguello [ 02/Nov/23 ] |
|
Let's try as many as we can. Per Judah's message: "A/UB/TSAN (all the SANs we can)". Maybe we could start with whichever is easiest to send over to us?
In terms of the tests, I believe we should try running the new binary with all of them multiple times. The more we can explore the search space, the better chance we have of finding a valuable bug |
| Comment by Trevor Guidry [ 02/Nov/23 ] |
|
I'm heads down focused on getting a project done by EOW but can take a look next week. |
| Comment by Steve Gross [ 02/Nov/23 ] |
|
Excellent, good to know. Which specific SAN variant(s)? do we want to try out, and for which test(s)? |
| Comment by javi Arguello [ 02/Nov/23 ] |
|
#2: Antithesis will be adding tests temporarily to evaluate their utility and will credit all the test hours used for these new tests while the evaluation is ongoing |
| Comment by Steve Gross [ 02/Nov/23 ] |
|
Let's clarify:
|
| Comment by javi Arguello [ 02/Nov/23 ] |
|
Bringing this card back to the forefront. After discussing with judah.schvimer@mongodb.com , we think it could be beneficial to run SAN enabled binaries inside Antithesis for exposing memory leaks important for the server architecture team.
trevor.guidry@mongodb.com would this be something you could help us with? |
| Comment by javi Arguello [ 14/Aug/23 ] |
|
Jumping in here from the Antithesis side. In discussion with tausif.rahman@mongodb.com, this project was deemed lower priority and hence moved to the backlog |