[CDRIVER-2053] test-libbson starts too many threads Created: 14/Feb/17  Updated: 06/Mar/17  Resolved: 24/Feb/17

Status: Closed
Project: C Driver
Component/s: libbson, tests
Affects Version/s: 1.5.0
Fix Version/s: 1.6.1

Type: Improvement Priority: Major - P3
Reporter: A. Jesse Jiryu Davis Assignee: A. Jesse Jiryu Davis
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File CDRIVER-2053-limit-threads-in-test-suite.patch    
Issue Links:
Duplicate
is duplicated by CDRIVER-2033 libbson /bson/as_json/string test fai... Closed

 Description   

The libbson test suite fails on 32-bit MIPS and 32-bit MIPS Little Endian:

https://buildd.debian.org/status/package.php?p=libbson&suite=sid

The bug was reported to me originally against 1.5.4. In fact the tests had failed since 1.5.0 but the since the Debian maintainers didn't catch up on their backlog and file a bug related to the failure immediately. I should ensure I get emails about future test failures.

Roberto Sanchez and I logged into a MIPS machine (I believe it was a 64-bit MIPS big endian?) in the Debian autobuilder network and were able to reproduce the crash: late in the test suite, test-libbson aborts with return code 134. We could fix it by commenting-out a dozen tests; but it didn't seem to matter which ones we commented out. We could also fix the tests by running them single-threaded.

The threaded test runner starts a thread per test and runs them all at once. I hypothesize that we added enough tests between 1.4.2 and 1.5.0 that we hit a limit on the number of threads a MIPS process can start. There are almost 250 tests total.

To fix:

  • Update the parallel runner to limit the number of concurrent threads
  • Don't run in threaded mode by default, anyway - it could make it hard to diagnose future build failures

Debian Stretch has just entered a freeze and will be released soon. We have libbson and libmongoc 1.4.2 in Stretch, and we won't be able to get anything new included, since this build failure was introduced circa 1.5.0. I think that's ok, the new features in libbson and libmongoc were MongoDB 3.4 support and native TLS on non-Linux, not having these in Debian is fine. Regardless, there's no urgency about fixing this now, we missed the deadline for 1.5.x in Stretch.

The next Debian release, Buster, will be frozen 18 to 24 months from now. Its timeline will be announced soon.



 Comments   
Comment by A. Jesse Jiryu Davis [ 03/Mar/17 ]

r1.6: https://github.com/mongodb/libbson/commit/2ea6bf92472972179b25feeb64ce135a68720128

Comment by Githook User [ 24/Feb/17 ]

Author:

{u'username': u'ajdavis', u'name': u'A. Jesse Jiryu Davis', u'email': u'jesse@mongodb.com'}

Message: CDRIVER-2053 run tests in only ten threads

Previously we had a thread per test; once the number exceeded circa 250,
test-libbson --threads failed on MIPS.
Branch: master
https://github.com/mongodb/libbson/commit/25da908e223295627b881a3a0947b5d546eb66da

Generated at Wed Feb 07 21:14:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.