[SERVER-20815] Remove yield_sort.js workload from FSM blacklists Created: 07/Oct/15  Updated: 15/Oct/15  Resolved: 09/Oct/15

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: 3.1.9
Fix Version/s: 3.2.0-rc0

Type: Task Priority: Major - P3
Reporter: Kamran K. Assignee: Jonathan Abrahams
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-17011 Cursor can return objects out of orde... Closed
Backwards Compatibility: Fully Compatible
Sprint: TIG A (10/09/15)
Participants:

 Description   

The concurrency suites now use read commands by default (SERVER-20067), so we should be able to remove yield_sort.js from the FSM blacklists (SERVER-17011).

Run a patch build on the "~ Linux Repeated Execution" variant to weed out any potential flakiness with the test or server code.



 Comments   
Comment by Githook User [ 09/Oct/15 ]

Author:

{u'username': u'hptabster', u'name': u'Jonathan Abrahams', u'email': u'jonathan@mongodb.com'}

Message: SERVER-20815 FSM enable yield_sort.js workload
Branch: master
https://github.com/mongodb/mongo/commit/136a70b87371c7a00c0904360487b6dacf2e5c0c

Comment by Charlie Swanson [ 08/Oct/15 ]

Looks like there's just a typo in the test:

var verifier = function sortVerifier(doc, prevDoc) {
    var correctOrder = true;
    if (prevDoc !== null) {
        correctOrder = (doc._id <= prevDoc._id);
    }
    return doc.a < nMatches && correctOrder;
};

should instead say

        correctOrder = (doc.c <= prevDoc.c);

since it was sorting by { c: -1 }

jonathan.abrahams, it should be a simple fix. Leaving assigned to you, as that will probably be simpler. Feel free to assign to me if you don't feel comfortable changing it.

Comment by Charlie Swanson [ 08/Oct/15 ]

Sure, I'll look into it a bit

Comment by David Storch [ 08/Oct/15 ]

charlie.swanson, do you have time to investigate?

Comment by Jonathan Abrahams [ 08/Oct/15 ]

This is the failure for yield_sort.js, with 3.1.9 mongo shell & mongod:

----
jstests/concurrency/fsm_workloads/yield_sort.js
----
 
 
Using 5 threads (requested 5)
connecting to: test
connecting to: test
connecting to: test
connecting to: test
connecting to: test
setting random seed: 796389244496
setting random seed: 6128580207005
setting random seed: 4073667433112
setting random seed: 3047587098553
setting random seed: 4914038488641
 
 
----
jstests/concurrency/fsm_workloads/yield_sort.js: Workload completed in 102 ms
----
 
 
2015-10-08T14:17:21.276-0400 E QUERY    [thread1] Error: 5 threads threw 2 different exceptions:
 
        3 threads threw
        Error: assert failed : Verifier failed!
        Query: { "query" : { "a" : { "$lt" : 100 } }, "orderby" : { "c" : -1 } }
        Query plan: {
        	"waitedMS" : NumberLong(0),
        	"queryPlanner" : {
        		"plannerVersion" : 1,
        		"namespace" : "db1.coll1",
        		"indexFilterSet" : false,
        		"parsedQuery" : {
        			"a" : {
        				"$lt" : 100
        			}
        		},
        		"winningPlan" : {
        			"stage" : "SORT",
        			"sortPattern" : {
        				"c" : -1
        			},
        			"inputStage" : {
        				"stage" : "SORT_KEY_GENERATOR",
        				"inputStage" : {
        					"stage" : "FETCH",
        					"inputStage" : {
        						"stage" : "IXSCAN",
        						"keyPattern" : {
        							"a" : 1,
        							"b" : 1
        						},
        						"indexName" : "a_1_b_1",
        						"isMultiKey" : false,
        						"isUnique" : false,
        						"isSparse" : false,
        						"isPartial" : false,
        						"indexVersion" : 1,
        						"direction" : "forward",
        						"indexBounds" : {
        							"a" : [
        								"[-inf.0, 100.0)"
        							],
        							"b" : [
        								"[MinKey, MaxKey]"
        							]
        						}
        					}
        				}
        			}
        		},
        		"rejectedPlans" : [ ]
        	},
        	"serverInfo" : {
        		"host" : "Jonathans-MacBook-Pro.local",
        		"port" : 27017,
        		"version" : "3.1.9",
        		"gitVersion" : "9c317d92145a77ab582acb215b3ed41af2f1b33c"
        	},
        	"ok" : 1
        }
        Previous doc: {
        	"_id" : 1,
        	"a" : 28,
        	"b" : 199,
        	"c" : 138,
        	"d" : 199,
        	"yield_text" : "are"
        }
        This doc: {
        	"_id" : 167,
        	"a" : 2,
        	"b" : 33,
        	"c" : 112,
        	"d" : 33,
        	"yield_text" : "words"
        }
 
        quietlyDoAssert@jstests/concurrency/fsm_libs/assert.js:53:15
        assert@src/mongo/shell/assert.js:25:5
        wrapAssertFn@jstests/concurrency/fsm_libs/assert.js:60:13
        assertWithLevel/assertWithLevel@jstests/concurrency/fsm_libs/assert.js:84:9
        advanceCursor@jstests/concurrency/fsm_workloads/yield.js:32:1
        sort@jstests/concurrency/fsm_workloads/yield_sort.js:35:9
        runFSM@jstests/concurrency/fsm_libs/fsm.js:19:13
        ____MongoToSM_newFunction_temp/<@:10:9
        main@jstests/concurrency/fsm_libs/worker_thread.js:81:17
        ____MongoToSM_newFunction_temp@:7:1
        ____MongoToSM_newFunction_temp@:5:24
 
 
        2 threads threw
        Error: assert failed : Verifier failed!
        Query: { "query" : { "a" : { "$lt" : 100 } }, "orderby" : { "c" : -1 } }
        Query plan: {
        	"waitedMS" : NumberLong(0),
        	"queryPlanner" : {
        		"plannerVersion" : 1,
        		"namespace" : "db1.coll1",
        		"indexFilterSet" : false,
        		"parsedQuery" : {
        			"a" : {
        				"$lt" : 100
        			}
        		},
        		"winningPlan" : {
        			"stage" : "SORT",
        			"sortPattern" : {
        				"c" : -1
        			},
        			"inputStage" : {
        				"stage" : "SORT_KEY_GENERATOR",
        				"inputStage" : {
        					"stage" : "FETCH",
        					"inputStage" : {
        						"stage" : "IXSCAN",
        						"keyPattern" : {
        							"a" : 1,
        							"b" : 1
        						},
        						"indexName" : "a_1_b_1",
        						"isMultiKey" : false,
        						"isUnique" : false,
        						"isSparse" : false,
        						"isPartial" : false,
        						"indexVersion" : 1,
        						"direction" : "forward",
        						"indexBounds" : {
        							"a" : [
        								"[-inf.0, 100.0)"
        							],
        							"b" : [
        								"[MinKey, MaxKey]"
        							]
        						}
        					}
        				}
        			}
        		},
        		"rejectedPlans" : [ ]
        	},
        	"serverInfo" : {
        		"host" : "Jonathans-MacBook-Pro.local",
        		"port" : 27017,
        		"version" : "3.1.9",
        		"gitVersion" : "9c317d92145a77ab582acb215b3ed41af2f1b33c"
        	},
        	"ok" : 1
        }
        Previous doc: {
        	"_id" : 59,
        	"a" : 68,
        	"b" : 141,
        	"c" : 97,
        	"d" : 141,
        	"yield_text" : "words"
        }
        This doc: {
        	"_id" : 97,
        	"a" : 97,
        	"b" : 103,
        	"c" : 97,
        	"d" : 103,
        	"yield_text" : "are"
        }
 
        quietlyDoAssert@jstests/concurrency/fsm_libs/assert.js:53:15
        assert@src/mongo/shell/assert.js:25:5
        wrapAssertFn@jstests/concurrency/fsm_libs/assert.js:60:13
        assertWithLevel/assertWithLevel@jstests/concurrency/fsm_libs/assert.js:84:9
        advanceCursor@jstests/concurrency/fsm_workloads/yield.js:32:1
        sort@jstests/concurrency/fsm_workloads/yield_sort.js:35:9
        runFSM@jstests/concurrency/fsm_libs/fsm.js:19:13
        ____MongoToSM_newFunction_temp/<@:10:9
        main@jstests/concurrency/fsm_libs/worker_thread.js:81:17
        ____MongoToSM_newFunction_temp@:7:1
        ____MongoToSM_newFunction_temp@:5:24
 
 :
throwError@jstests/concurrency/fsm_libs/runner.js:300:23
runWorkloads/<@jstests/concurrency/fsm_libs/runner.js:463:17
runWorkloads@jstests/concurrency/fsm_libs/runner.js:409:13
serial@jstests/concurrency/fsm_libs/runner.js:483:13
@/tmp/fsm_all.js:11:1

Comment by David Storch [ 08/Oct/15 ]

SERVER-17011 should be resolved if you have a 3.1.9 server and you're using a 3.1.9 shell. (Essentially the fix is that the find command includes separate batchSize and limit parameters rather than using a single "ntoreturn" field). It's been awhile since I looked at yield_sort.js, but if it still fails after unblacklisting there could very well be a bug in either the test or the server that is yet to be discovered. I'd say it's worth debugging to determine the root cause of the failure.

Comment by Jonathan Abrahams [ 08/Oct/15 ]

david.storch Should the changes from SERVER-20067 be sufficient to no longer blacklist this workload?

Generated at Thu Feb 08 03:55:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.