[SERVER-36062] Mobile SE: Stop running parallel and concurrent suites on mobile variants Created: 11/Jul/18  Updated: 29/Oct/23  Resolved: 25/Jul/18

Status: Closed
Project: Core Server
Component/s: Storage
Affects Version/s: None
Fix Version/s: 4.0.2, 4.1.2

Type: Improvement Priority: Major - P3
Reporter: Sulabh Mahajan Assignee: Sulabh Mahajan
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Problem/Incident
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.0
Sprint: Storage Non-NYC 2018-07-16, Storage Engines 2018-07-30
Participants:
Linked BF Score: 0

 Description   

parallel, parallel_compatibility, concurrency and concurrency_simultaneous test suites are not adding much to the coverage for mobile while adding to the time it takes to run the variants. Mobile SE is not intended to run parallel and concurrent operations and hence running these suites over it isn't building confidence in it's stability any more than the other core tests.

For instance, on RHEL 6.2 (mobile), following are the runtimes during my trial runs when excluding map_reduce tests (SERVER-35473):
concurrency – 1.5 hours
concurrency_simultaneous – 3.5 hours



 Comments   
Comment by Githook User [ 07/Aug/18 ]

Author:

{'username': 'sulabhM', 'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com'}

Message: SERVER-36062 Stop running parallel; enable concurrent test suites on mobile

(cherry picked from commit f165a6aac867472f38bd0d6f52fcf39e2be17ce6)
Branch: v4.0
https://github.com/mongodb/mongo/commit/c9a2e9b4427f8ffe37667f6ed7d8ec50d8dfba68

Comment by Sulabh Mahajan [ 25/Jul/18 ]

We had a meeting to discuss this ticket and it was decided to do the following:

  • Stop running parallel, parallel_compatibility and concurrency_simultaneous
  • Keep running concurrency with two index related tests disabled (these tests take a long time to run because index operations are slow, issue tracked by SERVER-32709)
Comment by Githook User [ 25/Jul/18 ]

Author:

{'name': 'Sulabh Mahajan', 'email': 'sulabh.mahajan@mongodb.com', 'username': 'sulabhM'}

Message: SERVER-36062 Stop running parallel; enable concurrent test suites on mobile
Branch: master
https://github.com/mongodb/mongo/commit/f165a6aac867472f38bd0d6f52fcf39e2be17ce6

Comment by Max Hirschhorn [ 13/Jul/18 ]

Just because today's implementation of concurrency of the mobile storage engine matches what SQLite expects doesn't mean that we won't accidentally break it in the future. We're currently depending on a handful of tests that use startParallelShell() and the jstestfuzz_concurrent suite to find any futures issues when using multiple clients. Are we confident that those tests / test suites are sufficient enough to have caught the problematic behavior we'd seen before with SQLite?

Comment by Andrew Morrow (Inactive) [ 12/Jul/18 ]

Thanks sulabh.mahajan, that sounds reasonable.

Comment by Sulabh Mahajan [ 12/Jul/18 ]

acm,

Or are you saying that it doesn't matter given the locking model of the underlying storage engine in use

Yes, the locking model is different, so the testing requirements are lower. The concurrent/parallel operations (unless just reads) end up being serialised by the lock manager. Hence, I believe that beyond lock manager, we are only testing mobile under the same scenarios that other test suites should have already done, while greatly increasing the testing time and resources consumed.

Here are my reasons why we shouldn't run these suites on mobile:

  • concurrency – As I said, we wouldn't really be running these tests concurrently from the storage engine perspectives. If we think there are certain operations unique to this suite that the storage engine gets to be tested with, we should enhance the core testing to include them instead, eg: I do not see map_reduce tests anywhere else other than in concurrency suite. map_reduce at storage engine would itself run single operation at a time, so it already doesn't make much sense. Because mobile throttles them to single operation at a time, it takes really long to execute them wasting time and compute resources.
  • concurrency_simultaneous – It makes even lesser sense to run concurrent tests together.
  • parallel – As I understand, individually almost all of these tests are available in core and this suite ends up running them in parallel. Since mobile is anyways going to serialise them past the lock manager, it doesn't make sense to run them at all.
  • parallel_compatibility – compatibility tests are not relevant to the mobile storage engine because it’s related to server level compatibility mechanism that isn’t used by mobile.

Note: I don't know time taken for parallel and parallel_compatibility, because basic and basicPlus are ignoring the tags on the tests and I haven't been able to run with these tests turned on. I end up running and failing on capped collection/profiling tests.

Comment by Andrew Morrow (Inactive) [ 11/Jul/18 ]

Can you clarify a bit what you mean about "not intended to run parallel and concurrent operations"? The invoking application can certainly have multiple threads "connected" to embedded that are executing concurrent database operations. Or are you saying that it doesn't matter given the locking model of the underlying storage engine in use?

Generated at Thu Feb 08 04:41:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.