[CDRIVER-1055] Unit Tests Under -fsanitize=address Timeout Created: 19/Dec/15 Updated: 18/Oct/16 Resolved: 18/Oct/16 |
|
| Status: | Closed |
| Project: | C Driver |
| Component/s: | tests |
| Affects Version/s: | 1.3.0 |
| Fix Version/s: | 1.5.0 |
| Type: | Bug | Priority: | Minor - P4 |
| Reporter: | Alex Bishop | Assignee: | Hannes Magnusson |
| Resolution: | Done | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Environment: |
Linux ubuntu 3.16.0-55-generic #74~14.04.1-Ubuntu SMP |
||
| Issue Links: |
|
||||||||||||||||
| Description |
|
Added to /libbson/CMakeLists.txt:
The unit tests will more often than not stall and abort due to a timeout. Given a lucky ordering of the tests from threading, they all pass.
Running the unit tests without parallelization are good.
|
| Comments |
| Comment by Hannes Magnusson [ 18/Oct/16 ] | ||||||||||||||||
|
We've removed the threading argument from the test suite as it just didn't work. | ||||||||||||||||
| Comment by Alex Bishop [ 27/Dec/15 ] | ||||||||||||||||
|
No problem and understood. This was more for my own curiosity and completeness, had free time from the holidays. We run your unit tests along side all others during our build process and using '-p' is acceptable. I didn't run w/o ASAN ever and would be surprised if it were still prevalent. ASAN might be a helpful catalyst for debugging. | ||||||||||||||||
| Comment by A. Jesse Jiryu Davis [ 27/Dec/15 ] | ||||||||||||||||
|
Thanks for the report and helping us diagnose. As Hannes said, there's problems with the test suite's concurrency in the default mode, even without ASAN. We'll follow up on all these issues in the new year. | ||||||||||||||||
| Comment by Alex Bishop [ 25/Dec/15 ] | ||||||||||||||||
|
I think the root cause is that non-trivial work is performed in a forking thread. My investigations lead me to believe a memory lock is deadlocked. Each fork strips out all but the calling thread which will leave one in a bad way with respect to critical regions.
One will find this deadlock occurs most often during /bson/writer/shared_buffer since it's the slowest and perhaps the most malloc happy. Slow enough to allow two threads to interact. The regular state of affairs for the backtrace below:
Same with gcc & clang. Was using libbson tag 1.1.9 to avoid cmake incompatibility, 1.3.0 will be same same. A solution might be a test-runner able to be called with exec in the child fork. However, the context mechanics, seed and other shared tooling will likely be an issue. It might be wise to just fork or thread, IMHO. Also, when running with tsan, there is a read data race upon test->count upon printing out the json test results. | ||||||||||||||||
| Comment by Hannes Magnusson [ 21/Dec/15 ] | ||||||||||||||||
|
We are aware of running the test suite without -p doesn't really work, but we believe that is a problem with the test framework itself, not the driver. |