[SERVER-20147] fsm_all_sharded_replication.js - agg_base.js fails due to missing documents in aggregation response Created: 20/Jul/15  Updated: 19/Sep/15  Resolved: 27/Aug/15

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 3.1.8

Type: Task Priority: Major - P3
Reporter: Kamran K. Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Text File 999-failure.log     Text File non-999-failure.log    
Issue Links:
Related
is related to SERVER-20084 the shell should not set slaveOk if a... Closed
Backwards Compatibility: Fully Compatible
Sprint: QuInt 8 08/28/15
Participants:

 Description   

The agg_base.js FSM workload occasionally fails in
fsm_all_sharded_replication.js because the aggregation query doesn't return all 1000 documents:

Example failure:

[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.340+0000         Error: [999] != [1000] are not equal : undefined
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at quietlyDoAssert (jstests/concurrency/fsm_libs/assert.js:53:15)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at Function.assert.eq (src/mongo/shell/assert.js:38:5)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at wrapAssertFn (jstests/concurrency/fsm_libs/assert.js:60:16)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at Function.assertWithLevel.(anonymous function) [as eq] (jstests/concurrency/fsm_libs/assert.js:99:13)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at Object.query (jstests/concurrency/fsm_workloads/agg_base.js:44:31)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at Object.runFSM [as run] (jstests/concurrency/fsm_libs/fsm.js:19:16)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at <anonymous>:8:13
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at Object.main (jstests/concurrency/fsm_libs/worker_thread.js:81:17)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.341+0000             at ____MongoToV8_newFunction_temp (<anonymous>:5:25)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000             at ____MongoToV8_newFunction_temp (<anonymous>:3:24)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000 
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at throwError (jstests/concurrency/fsm_libs/runner.js:268:23)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at jstests/concurrency/fsm_libs/runner.js:396:17
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at Array.forEach (native)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at runWorkloads (jstests/concurrency/fsm_libs/runner.js:351:22)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at serial (jstests/concurrency/fsm_libs/runner.js:415:13)
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000     at jstests/concurrency/fsm_all_sharded_replication.js:68:1 at jstests/concurrency/fsm_libs/runner.js:280
[js_test:fsm_all_sharded_replication] 2015-07-19T23:40:11.342+0000 failed to load: jstests/concurrency/fsm_all_sharded_replication.js

https://evergreen.mongodb.com/task/mongodb_mongo_master_amazon_concurrency_sharded_WT_92c6ee5cedf09928720b79592981dc1c6aeb0482_15_07_19_21_54_37

Note: The assertion sometimes fails with fewer than 999 documents, but it's usually 999.

[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400         3 threads threw
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400         Error: [632] != [1000] are not equal : undefined
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400             at quietlyDoAssert (jstests/concurrency/fsm_libs/assert.js:53:15)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400             at Function.assert.eq (src/mongo/shell/assert.js:38:5)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400             at wrapAssertFn (jstests/concurrency/fsm_libs/assert.js:60:16)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.517-0400             at Function.assertWithLevel.(anonymous function) [as eq] (jstests/concurrency/fsm_libs/assert.js:99:13)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at Object.query (jstests/concurrency/fsm_workloads/agg_base.js:44:31)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at Object.runFSM [as run] (jstests/concurrency/fsm_libs/fsm.js:19:16)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at <anonymous>:8:13
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at Object.main (jstests/concurrency/fsm_libs/worker_thread.js:81:17)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at ____MongoToV8_newFunction_temp (<anonymous>:5:25)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.518-0400             at ____MongoToV8_newFunction_temp (<anonymous>:3:24)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400 
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400         1 thread threw
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400         Error: [182] != [1000] are not equal : undefined
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400             at quietlyDoAssert (jstests/concurrency/fsm_libs/assert.js:53:15)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400             at Function.assert.eq (src/mongo/shell/assert.js:38:5)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400             at wrapAssertFn (jstests/concurrency/fsm_libs/assert.js:60:16)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400             at Function.assertWithLevel.(anonymous function) [as eq] (jstests/concurrency/fsm_libs/assert.js:99:13)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.519-0400             at Object.query (jstests/concurrency/fsm_workloads/agg_base.js:44:31)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400             at Object.runFSM [as run] (jstests/concurrency/fsm_libs/fsm.js:19:16)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400             at <anonymous>:8:13
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400             at Object.main (jstests/concurrency/fsm_libs/worker_thread.js:81:17)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400             at ____MongoToV8_newFunction_temp (<anonymous>:5:25)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400             at ____MongoToV8_newFunction_temp (<anonymous>:3:24)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400 
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400     at throwError (jstests/concurrency/fsm_libs/runner.js:268:23)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.520-0400     at jstests/concurrency/fsm_libs/runner.js:396:17
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.521-0400     at Array.forEach (native)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.521-0400     at runWorkloads (jstests/concurrency/fsm_libs/runner.js:351:22)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.521-0400     at serial (jstests/concurrency/fsm_libs/runner.js:415:13)
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.521-0400     at jstests/concurrency/fsm_all_sharded_replication.js:68:1 at jstests/concurrency/fsm_libs/runner.js:280
[js_test:fsm_all_sharded_replication] 2015-07-09T14:19:15.521-0400 failed to load: jstests/concurrency/fsm_all_sharded_replication.js

https://evergreen.mongodb.com/task/mongodb_mongo_master_osx_108_concurrency_sharded_WT_d1cb71465274bcb5f3bc962ef2740cf985f32113_15_07_09_16_47_51



 Comments   
Comment by Githook User [ 27/Aug/15 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'charlie.swanson@mongodb.com'}

Message: SERVER-20147 Remove agg_base.js from the blacklist in fsm_all_sharded_replication.js
Branch: master
https://github.com/mongodb/mongo/commit/68237a9e90c82b891b55a80b13f6af3f972d46bb

Comment by Kamran K. [ 25/Aug/15 ]

Note: agg_base.js was disabled because of SERVER-19418. It might still be worthwhile to run the test repeatedly to verify that the readPreference issue is fixed, though.

Comment by Charlie Swanson [ 25/Aug/15 ]

Sure. I'll run a patch build repeating this test, and see if it's safe to commit now. Assigning to me. Thanks!

Comment by Adam Midvidy [ 25/Aug/15 ]

charlie.swanson, SERVER-20084 has been resolved. Can you see if agg_base.js can be removed from the blacklist?

Comment by Charlie Swanson [ 21/Aug/15 ]

Assigning to Adam, feel free to re-assign if SERVER-20084 does not resolve this. Note, I haven't been able to reproduce this, so I guess the only work is taking agg_base.js off the blacklist, and we'll re-open if we see it again.

Comment by Charlie Swanson [ 21/Aug/15 ]

This appears to be caused by the linked ticket, which causes the pipeline to be run on the secondary, before all the data has been replicated, thus the wrong number of results are returned.

Comment by Ian Whalen (Inactive) [ 14/Aug/15 ]

assigning to charlie for initial thoughts just because agg.

Generated at Thu Feb 08 03:53:18 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.