[SERVER-10090] repl4.js failing on Nightly Linux 32-bit, V2.4 Windows 32-bit Created: 03/Jul/13  Updated: 10/Dec/14  Resolved: 30/Jul/13

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Matt Kangas Assignee: Eric Milkie
Resolution: Duplicate Votes: 0
Labels: buildbot
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

buildbot: V2.4 Windows 32-bit
buildbot: Nightly Linux 32-bit


Issue Links:
Duplicate
duplicates SERVER-10344 Race condition when starting up new m... Closed
Operating System: Windows
Participants:

 Description   

Build 147 on Wed Jul 3

http://buildlogs.mongodb.org/V2.4%20Windows%2032-bit/builds/147/test/replication/repl4.js

assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined
Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.soon (src/mongo/shell/assert.js:110:60)
    at soonCount (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:4:12)
    at doTest (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:35:5)
    at D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:38:1
Wed Jul 03 10:20:39.423 JavaScript execution failed: assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:L7
failed to load: D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js

Note that this failure occurred on revision a2ddc68ba7c9. It previously succeeded with this revision on July 02.

This failure has occurred several other times recently:

Jul 03 01:07	a2ddc68ba7c9...	failure	#147	Failed test_9 (repl4.js)
 02 16:35	a2ddc68ba7c9...	success
Jul 01 00:00	a6462d21ea94...	failure	#144	Failed test_9 (repl4.js)
Jun 30 04:01	a6462d21ea94... success
Jun 24 20:41	13b33da1e499...	failure	#135	Failed test_9 (repl4.js)
Jun 24 20:37	13b33da1e499...	success
Jun 22 06:49	13b33da1e499...	failure	#130	Failed test_9 (repl4.js)
...
Jun 09 06:34	9c4382ecb90f...	failure	#115	Failed test_9 (repl4.js)
...
Jun 01 00:00	e43cbcd9797f...	failure #106	Failed test_9 (repl4.js)

All of the failures are identical:

But critically, it doesn't fail consistently. Success is interleaved with these failures.

Any ideas? Eric, assigning to you for triage.



 Comments   
Comment by Matt Kangas [ 30/Jul/13 ]

I'm now fully convinced that these failures are all due to the issue Spencer identifies in SERVER-10344

Comment by Matt Kangas [ 26/Jul/13 ]

Again in V2.4 Linux 32-bit Build #146 July 26 rev 7ea27bae

http://buildbot.mongodb.org/builders/V2.4%20Linux%2032-bit/builds/146/steps/test_9/logs/stdio
http://buildlogs.mongodb.org/V2.4%20Linux%2032-bit/builds/146/test/replication/repl4.js

assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined
Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.soon (src/mongo/shell/assert.js:110:60)
    at soonCount (/mnt/slaves/Linux_32bit_V2.4/mongo/jstests/repl/repl4.js:4:12)
    at doTest (/mnt/slaves/Linux_32bit_V2.4/mongo/jstests/repl/repl4.js:35:5)
    at /mnt/slaves/Linux_32bit_V2.4/mongo/jstests/repl/repl4.js:38:1
Fri Jul 26 11:51:39.522 JavaScript execution failed: assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:L7
failed to load: /mnt/slaves/Linux_32bit_V2.4/mongo/jstests/repl/repl4.js

Comment by Matt Kangas [ 25/Jul/13 ]

This is still impacting V2.4 Windows 32-bit regularly as of July 25.

V2.4 Windows 32-bit Build #168 July 26 rev 7ea27bae26b
http://buildbot.mongodb.org/builders/V2.4%20Windows%2032-bit/builds/168/steps/test_9/logs/stdio
http://buildlogs.mongodb.org/V2.4%20Windows%2032-bit/builds/168/test/replication/repl4.js

Comment by Matt Kangas [ 17/Jul/13 ]

Again:

V2.4 Windows 32-bit Build #162 July 17

http://buildbot.mongodb.org/builders/V2.4%20Windows%2032-bit/builds/162/steps/test_9/logs/stdio
http://buildbot.mongodb.org/builders/V2.4%20Windows%2032-bit/builds/162/steps/test_9/logs/stdio

assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined
Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.soon (src/mongo/shell/assert.js:110:60)
    at soonCount (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:4:12)
    at doTest (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:35:5)
    at D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:38:1
Wed Jul 17 12:20:36.104 JavaScript execution failed: assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:L7
failed to load: D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js

Comment by Matt Kangas [ 13/Jul/13 ]

Eric, I see that you disabled this test on Windows (in master) in rev d3406f67.

This is also happening on Nightly Linux 32-bit. Intermittent, but not isolated to Windows.

Nightly Linux 32-bit Build #1366 Jun 15 (rev a73206f7a1)

http://buildbot.mongodb.org/builders/Nightly%20Linux%2032-bit/builds/1366/steps/test_12/logs/stdio
http://buildlogs.mongodb.org/Nightly%20Linux%2032-bit/builds/1366/test/replication/repl4.js

Nightly Linux 32-bit Build #1400 July 13 (rev 458cfbf633)

http://buildbot.mongodb.org/builders/Nightly%20Linux%2032-bit/builds/1400/steps/test_12/logs/stdio
http://buildlogs.mongodb.org/Nightly%20Linux%2032-bit/builds/1400/test/replication/repl4.js

Some kind of race is happening here, but what is it?

Sat Jul 13 07:41:23.348 shell: started program /mnt/slaves/Linux_32bit_Nightly/mongo/mongod --nohttpinterface --noprealloc --bind_ip 127.0.0.1 --smallfiles --port 31001 --dbpath /data/db/repl4tests-slave --slave --source 127.0.0.1:31000 --only c --oplogSize 40 --setParameter enableTestCommands=1
 m31001| Sat Jul 13 07:41:23.424 [initandlisten] MongoDB starting : pid=31961 port=31001 dbpath=/data/db/repl4tests-slave slave=1 32-bit host=bs-linux32.10gen.cc
 m31001| Sat Jul 13 07:41:23.430 [initandlisten] waiting for connections on port 31001
 m31001| Sat Jul 13 07:41:23.553 [initandlisten] connection accepted from 127.0.0.1:37846 #1 (1 connection now open)
 m31001| Sat Jul 13 07:41:24.437 [replslave] repl: syncing from host:127.0.0.1:31000
...
Sat Jul 13 07:41:53.832 assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:7
failed to load: /mnt/slaves/Linux_32bit_Nightly/mongo/jstests/repl/repl4.js

Comment by Matt Kangas [ 11/Jul/13 ]

Again: V2.4 Windows 32-bit Build #156

http://buildbot.mongodb.org/builders/V2.4%20Windows%2032-bit/builds/156/steps/test_9/logs/stdio
http://buildlogs.mongodb.org/V2.4%20Windows%2032-bit/builds/156/test/replication/repl4.js

Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.soon (src/mongo/shell/assert.js:110:60)
    at soonCount (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:4:12)
    at doTest (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:35:5)
    at D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:38:1
 m31001| Thu Jul 11 15:41:09.574 [replslave] repl: syncing from host:127.0.0.1:31000
Thu Jul 11 15:41:09.574 JavaScript execution failed: assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:L7
failed to load: D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js

Comment by Matt Kangas [ 06/Jul/13 ]

Happened again:

Builder V2.4 Windows 32-bit #151 revision 73dac6d52ea5 on July 6.

http://buildbot.mongodb.org/builders/V2.4%20Windows%2032-bit/builds/151/steps/test_9/logs/stdio
http://buildlogs.mongodb.org/V2.4%20Windows%2032-bit/builds/151/test/replication/repl4.js

assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined
Error: Printing Stack Trace
    at printStackTrace (src/mongo/shell/utils.js:37:15)
    at doassert (src/mongo/shell/assert.js:6:5)
    at Function.assert.soon (src/mongo/shell/assert.js:110:60)
    at soonCount (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:4:12)
    at doTest (D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:35:5)
    at D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js:38:1
 m31001| Sat Jul 06 12:26:39.973 [replslave] repl: syncing from host:127.0.0.1:31000
Sat Jul 06 12:26:39.973 JavaScript execution failed: assert.soon failed: function () { 
                return s.getDB( db )[ coll ].find().count() == count; 
                }, msg:undefined at src/mongo/shell/assert.js:L7
failed to load: D:\slave\Windows_32bit_V2.4\mongo\jstests\repl\repl4.js

Comment by Eric Milkie [ 03/Jul/13 ]

2.4.5 is already released; bumping to 2.4.6

Generated at Thu Feb 08 03:22:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.