[SERVER-26309] Disable auto splitting in ShardingTest by default Created: 23/Sep/16  Updated: 19/Dec/17  Resolved: 29/Sep/16

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 3.2.11, 3.4.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Dianna Hohensee (Inactive) Assignee: Esha Maharishi (Inactive)
Resolution: Done Votes: 0
Labels: bkp, test-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-26306 call disableAutoSplit() in large_chun... Closed
Related
is related to SERVER-26368 make movechunk_with_*paranoia.js spli... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.2
Sprint: Sharding 2016-10-10
Participants:
Linked BF Score: 0

 Description   

Any test that does a moveChunk or split command is liable to run into issues because chunks are split unexpectedly.

For example:
1) bulk data insert
2) auto split is activated
3) manual moveChunk
4) assert on the count of data that moved fails because a smaller chunk was moved than the test expected.

We've seen ~5 BFs related to incorrect number of expected chunks, etc.
We could potentially have problems with > 50 sharding tests that didn't take auto splitting into account when written. We would also have to be very conscientious of auto splitting in future test writing if auto splitting were not disabled by default.



 Comments   
Comment by Githook User [ 04/Oct/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-26309 for renamed tests, change test name in blacklists to match
Branch: v3.2
https://github.com/mongodb/mongo/commit/dbd52147346f303ebba72a20f8fd8baa43094d10

Comment by Githook User [ 30/Sep/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-26309 for renamed tests, change test name in blacklists to match
Branch: master
https://github.com/mongodb/mongo/commit/99c905a87a620d50d2bad4659978325032372258

Comment by Githook User [ 29/Sep/16 ]

Author:

{u'username': u'monkey101', u'name': u'Dan Pasette', u'email': u'dan@mongodb.com'}

Message: Revert "SERVER-26309 Disable auto splitting in ShardingTest by default"

This reverts commit c47eb7408d712ac75c09b54079abfd0c28346a07.
Branch: master
https://github.com/mongodb/mongo/commit/9a4474ea06409f729121283dadbdf3a17b131cdb

Comment by Githook User [ 29/Sep/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-26309 also enable autosplit in authCommands.js
Branch: v3.2
https://github.com/mongodb/mongo/commit/8f40a061e80730df24a3d7dacac6962e0846db8f

Comment by Githook User [ 29/Sep/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-26309 Disable auto splitting in ShardingTest by default
Branch: v3.2
https://github.com/mongodb/mongo/commit/c6d86af073687d119a4d707bfc1a2d0f0df7419a

Comment by Githook User [ 29/Sep/16 ]

Author:

{u'name': u'Esha Maharishi', u'email': u'esha.maharishi@mongodb.com'}

Message: SERVER-26309 Disable auto splitting in ShardingTest by default
Branch: master
https://github.com/mongodb/mongo/commit/c47eb7408d712ac75c09b54079abfd0c28346a07

Comment by Esha Maharishi (Inactive) [ 26/Sep/16 ]

Oh, true, that's better than an sh.enableAutoSplit(). Good call.

Comment by Spencer Brody (Inactive) [ 26/Sep/16 ]

If we make an enableAutosplit option to shardingTest (analogous to the existing enableBalancer option), then we won't need to blacklist tests that depend on autosplitting from the continuous stepdown suite.

Comment by Dianna Hohensee (Inactive) [ 26/Sep/16 ]

Maybe we could have the stepdown suite's JS check for enabling auto split parameter, and avoid black listing those JS tests. It could do the write before starting stepdown, and the regular ShardingTest checks whether it's already set before trying a write? Might not be the cleanest idea.

Comment by Esha Maharishi (Inactive) [ 26/Sep/16 ]

Tests that saw real errors that caused build failures:

  • cursor1.js
    assert on config.chunks.count() can fail
    manually splits chunks by "middle" (can clash with point that was unexpectedly autosplit on)
  • large_chunk.js
    assert on config.chunks.count() can fail
    moves chunk by "find", then accesses what it assumes is the only chunk through config.chunks.find().toArray()[0] to compare what shard the chunk is on
  • geo_near_random1.js
    manually splits chunks by "middle" (can clash with point that was unexpectedly autosplit on)
    moves specific chunk by "find" after split by "middle"
  • stale_mongos_updates_and_removes.js
    manually splits chunks by "middle" (can clash with point that was unexpectedly autosplit on)
    moves specific chunk by "find" after split by "middle"
  • regex_targeting.js
    manually splits chunks by "middle" (can clash with point that was unexpectedly autosplit on)
    moves specific chunk by "find" after split by "middle"

The BF tickets for each of these failures are linked to this ticket.

Test code changes required to disable autoSplit by default in master:

  • call disableAutoSplit() in ShardingTest
  • in multiVersion, add an override that will start any 3.2 mongoses with --noAutoSplit in addition to the disableAutoSplit() call in ShardingTest
  • remove all calls to sh.disableAutoSplit() in jstests and unblacklist those tests from the stepdown suite
  • add sh.enableAutoSplit() helper that writes to config.settings
  • call sh.enableAutoSplit() in any tests that explicitly test autosplitting and blacklist those tests from the stepdown suite

Test code changes required to disable autoSplit by default in 3.2:

  • start mongoses with --noAutoSplit in ShardingTest
  • remove explicit passing of --noAutoSplit in jstests
  • create a ShardingTest option that turns on autoSplit
  • for any tests that explicitly test autosplitting, pass the option that turns on autoSplit

To keep in mind:

  • because disabling autosplit is done by writing to config.settings, need to make sure disableAutoSplit is called before stepdowns can happen in the sharding_continuous_config_stepdown suite
Comment by Daniel Pasette (Inactive) [ 23/Sep/16 ]

Totally agree with disabling. Do we have any coverage of the autosplit code paths once disabled? How many sharding tests will enable it when this task is complete if any?

Generated at Thu Feb 08 04:11:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.