[SERVER-7849] manyclients.js failing since switch to V8 Created: 05/Dec/12 Updated: 11/Jul/16 Resolved: 09/Jan/13 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 2.3.2 |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | Ian Whalen (Inactive) | Assignee: | Eliot Horowitz (Inactive) |
| Resolution: | Done | Votes: | 0 |
| Labels: | buildbot | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||
| Issue Links: |
|
||||||||
| Operating System: | ALL | ||||||||
| Participants: | |||||||||
| Description |
|
http://buildbot.mongodb.org/builders/Nightly%20OS%20X%2010.5%2064-bit/builds/1107 |
| Comments |
| Comment by auto [ 08/Jan/13 ] |
|
Author: {u'date': u'2013-01-08T21:31:10Z', u'email': u'ben.becker@10gen.com', u'name': u'Ben Becker'}Message: |
| Comment by auto [ 08/Jan/13 ] |
|
Author: {u'date': u'2013-01-08T20:56:49Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}Message: |
| Comment by Tad Marshall [ 08/Jan/13 ] |
|
There are some print() statements in the test, and I got output from them in only one out of four trials. I agree that the test is aggressive and I also concluded that testing whether a large number of connections could be completed with none taking over five seconds was not useful. |
| Comment by auto [ 08/Jan/13 ] |
|
Author: {u'date': u'2013-01-08T05:27:56Z', u'email': u'eliot@10gen.com', u'name': u'Eliot Horowitz'}Message: |
| Comment by Eliot Horowitz (Inactive) [ 08/Jan/13 ] |
|
tad - where do you see the logging issue? |
| Comment by Tad Marshall [ 07/Jan/13 ] |
|
I tested on Windows 64-bit debug. The test fails with 200 threads on my machine, but passes with 50, 100 or 150. There are other problems: we are losing log messages (at least any from "print()") and so the logged output is not reliable. Changing a timeout value seemed to allow the creation of more connections than the test is supposed to create (I got 401 simultaneous connections when set to 200). In one trial, the test just "stopped running", with 53 threads and an equal number of open TCP connections, but all were waiting for a response to a "find". We should get the logging solid and see what information we are missing. |
| Comment by Ben Becker [ 01/Jan/13 ] |
|
OS X nightly needs more file descriptors (though segv is odd): Linux 32-bit failure is definitely a bug: http://buildlogs.mongodb.org/Nightly%20Linux%20RHEL%2032-bit/builds/332/test/parallel/manyclients.js |
| Comment by Eliot Horowitz (Inactive) [ 01/Jan/13 ] |
|
Still happening. |
| Comment by auto [ 26/Dec/12 ] |
|
Author: {u'date': u'2012-12-26T20:57:05Z', u'name': u'Ben Becker', u'email': u'ben.becker@10gen.com'}Message: |
| Comment by Ben Becker [ 24/Dec/12 ] |
|
This appears to have exposed an issue with our use of the v8 API in JSThread::operator()(). TLS could be invalid at this point, so I'm not exactly sure how this was expected to work given that we use the same isolate. This may also be why some of the older builders are more likely to encounter this bug. |
| Comment by Ben Becker [ 21/Dec/12 ] |
|
The smartos failure looks like a ulimit -n issue. |
| Comment by Ian Whalen (Inactive) [ 21/Dec/12 ] |
|
Same test is failing with different error messages on the Nightly Solaris-SmartOS 64-bit builder: |
| Comment by Randall Hunt [ 14/Dec/12 ] |
|
changed ulimit on all machines |
| Comment by Eric Milkie [ 12/Dec/12 ] |
|
There are many limits that ulimit can adjust. I'm not sure that number is the one that we're hitting now, but I haven't really looked into this further yet. |
| Comment by Ian Whalen (Inactive) [ 12/Dec/12 ] |
|
randall has said the ulimit has been raised to 2048 across all OS X machines. does it need to go higher? |
| Comment by Eric Milkie [ 12/Dec/12 ] |
|
Might just be ulimit problems at this point. |
| Comment by Ian Whalen (Inactive) [ 12/Dec/12 ] |
|
reopening because OS X DEBUG is still failing at: http://buildbot.mongodb.org/builders/OS%20X%2010.5%2064-bit%20DEBUG/builds/1721 |
| Comment by auto [ 10/Dec/12 ] |
|
Author: {u'date': u'2012-12-10T20:27:22Z', u'name': u'Ben Becker', u'email': u'ben.becker@10gen.com'}Message: |