[SERVER-470] concurrency testing Created: 09/Dec/09  Updated: 12/Jul/16  Resolved: 05/Jan/10

Status: Closed
Project: Core Server
Component/s: Concurrency
Affects Version/s: None
Fix Version/s: 1.3.1

Type: Improvement Priority: Major - P3
Reporter: Eliot Horowitz (Inactive) Assignee: Aaron Staple
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-446 finish v8 engine Closed
Participants:

 Description   

now that we're going to multi-threaded, we need really good test suites to make sure we don't have disastorous bugs working.
we need some sort of a framework and a way to do this.
maybe using the exec shell stuff, but not 100% sure



 Comments   
Comment by auto [ 12/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: Revert "Revert "SERVER-470 don's use namespace 'bar' in jstests""

This reverts commit f124e60fd1f9b4f90cdbd752d138b968f59b4096.
http://github.com/mongodb/mongo/commit/c87c0cbed864c46e0ba0a30d3ef20a85cca88099

Comment by auto [ 11/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: Revert "SERVER-470 try removing 'bar' collection from tests again"

This reverts commit a583a72db9587e25a6fe83d4f29a152635893c38.
http://github.com/mongodb/mongo/commit/3c40bf8771435d7a4606e7e6c78e4e54badd72f2

Comment by auto [ 11/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 try removing 'bar' collection from tests again
http://github.com/mongodb/mongo/commit/a583a72db9587e25a6fe83d4f29a152635893c38

Comment by auto [ 11/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 only install fork() in shell, not db internal js
http://github.com/mongodb/mongo/commit/63df738650175dadb9ac5f590516b3585f229a6d

Comment by auto [ 06/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: Revert "SERVER-470 don's use namespace 'bar' in jstests"

This reverts commit 796528283a1ba9d078dc9d22fa971e8f682bbc69.
http://github.com/mongodb/mongo/commit/f124e60fd1f9b4f90cdbd752d138b968f59b4096

Comment by Aaron Staple [ 05/Jan/10 ]

Ok, I think we now have a fair initial set of tests in jstests/parallel. These tests don't measure performance, but they do push the db in an attempt to elicit race conditions. I filed a few database bugs as I was writing these tests, but the important ones have been fixed and at the moment all the parallel tests seem to pass reliably. We should add them to our v8 buildbot once that is up and running.

I'm going to resolve this bug, but please let me know if you want more tests of a certain type.

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 don's use namespace 'bar' in jstests
http://github.com/mongodb/mongo/commit/796528283a1ba9d078dc9d22fa971e8f682bbc69

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 finish basicPlus test
http://github.com/mongodb/mongo/commit/2b6fa5493780817d14927a3ccab8156c87b3deb3

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 test to run basic js test in parallel with other tests
http://github.com/mongodb/mongo/commit/5163695eca1f22cfde7d40968574dba21d97153f

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add test with 200 clients
http://github.com/mongodb/mongo/commit/b4230ff11ee0c4080b7e1846ad53f0f3acdf9221

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 improvements for allops test
http://github.com/mongodb/mongo/commit/9d671360dc7ddce193dd59d9875c27023d514fa3

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add test that does insert, delete, update and count
http://github.com/mongodb/mongo/commit/70e9ed6418b592c666c893dfd4fee7886e6b8575

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 improve event generation helpers
http://github.com/mongodb/mongo/commit/3a1907331af23a634d9a52e9457e967d5f691577

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 move more code to parallel testing fwk
http://github.com/mongodb/mongo/commit/88eb16dc0f8b06311b5e04670c6377be491652e4

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 start extracting framework for parallel test
http://github.com/mongodb/mongo/commit/5869ed26b864f335345c7a496f592bf23c3f1ed6

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 Reinstate fsync tests in basic runners
http://github.com/mongodb/mongo/commit/d8e6e36089c0836fc9719c020fc3aea6188a3cae

Comment by auto [ 05/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 avoid conflicting use of the popular 'foo' namespace
http://github.com/mongodb/mongo/commit/e93fad288b48b23b27bba19eb10f299945fb0c4b

Comment by auto [ 04/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add randomization and other improvements to basic parallel test
http://github.com/mongodb/mongo/commit/73dd7cb3ebe47a6a6e7b07236cc7dcd41c855382

Comment by auto [ 04/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 improve JS rand()
http://github.com/mongodb/mongo/commit/e2bac039d1707ccd5ebc23e2d40b6f3713adf625

Comment by auto [ 04/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 make randomization more determiniztic (wrt seed) in insert test
http://github.com/mongodb/mongo/commit/35d800321b1b87bcd940d07aed2eeaae0c072468

Comment by auto [ 04/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add ability to set seed when generating random numbers
http://github.com/mongodb/mongo/commit/5864c4a8dbfd7cb705805b72702084b4922e8d8f

Comment by auto [ 04/Jan/10 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 clean up multi thread testing function
http://github.com/mongodb/mongo/commit/3cec831af71e75e24234a68c036aba1753f746dd

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 MINOR don't need persistent handles for copied function arguments
http://github.com/mongodb/mongo/commit/3b84e7162e2d328ea7ea2e8c53c70cb6d8cb4dc4

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 complete setup for parallel clients in separate js scopes
http://github.com/mongodb/mongo/commit/33cd7bc1e148af949c86790a678b8732aae77ee1

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 MINOR remove test code
http://github.com/mongodb/mongo/commit/e88f0fdf4562ebf6b6d48c0823cecfadd79f6335

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 MINOR abstract shell init functionality so v8_utils need not link with shell
http://github.com/mongodb/mongo/commit/8573d53da75b62e86c7b0d6b56766aa3a86f9b1b

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 MINOR use return value to check in assert.parallelTests
http://github.com/mongodb/mongo/commit/d1e3f2afe5fc8aee3e40c3c2278cf560d981526c

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 MINOR Refactor shell scope init code
http://github.com/mongodb/mongo/commit/978582f803cddd0df6551e4ed4fe704e3fc012d4

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 allow shell threads to be run in fresh scopes
http://github.com/mongodb/mongo/commit/aece98652ee7284357a7e729ae2ee21aee259fe5

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 test to run basic js tests in parallel
http://github.com/mongodb/mongo/commit/24169676c46d8fd8d996d81f628f2d76c60425d9

Comment by auto [ 28/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add runParallelTests helper, improve insert test
http://github.com/mongodb/mongo/commit/ad6c3d4132ab83dfe5a2df5f13725e469f6499e9

Comment by auto [ 28/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 improve basic test
http://github.com/mongodb/mongo/commit/3699192679c7aef16068d32f309c1df387a79a9e

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 fix some v8 locking issues
http://github.com/mongodb/mongo/commit/81cf93cf8af6f637ce41239c7157df4a4d622123

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 enhance basic test
http://github.com/mongodb/mongo/commit/980006020db78f3a1f68777160da1dce72810fff

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 add fork test
http://github.com/mongodb/mongo/commit/8435fd5526effa9433bf3796be93e46fb38734ae

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 unlock v8 appropriately so can do work in parallel
http://github.com/mongodb/mongo/commit/d70db47d96390357b91565df295b6d061b4136d7

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 added fork()/Thread() to v8 engine-
http://github.com/mongodb/mongo/commit/7dcfe01698f5a64baa6088fd22223490b746483c

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-470 begin basic parallel test
http://github.com/mongodb/mongo/commit/f97a6f32d69d42bba3b886d6eccf58a6256b9508

Comment by Eliot Horowitz (Inactive) [ 11/Dec/09 ]

We have some of our own benchmarks, but the more the better.
master in github in our tests performs much better with many clients
would be curious about your test

Comment by Jon Stephens [ 11/Dec/09 ]

If you need help testing concurrency (actually doing benchmarks) let me know. I've been doing my own distributed testing on 1.14 and my performance starts to drop pretty fast over 30 concurrent connections which is a big issue in my environment.

Comment by Eliot Horowitz (Inactive) [ 09/Dec/09 ]

ok - so i think it makes sense to just finish v8, then do testing.
finishing v8 shouldn't be that hard, just some more types to add.

Comment by Aaron Staple [ 09/Dec/09 ]

Since we had threads working in v8 before, I'd vode for the v8 route.

Comment by Eliot Horowitz (Inactive) [ 09/Dec/09 ]

i think multithreaded js is ok.
one thing we want to do is finish the v8 engine (its almost done)
so if you wanted to use v8, it wouldn't be hard to get it to the point where it was viable.
it passes about 60% of the tests now.

Comment by Aaron Staple [ 09/Dec/09 ]

Ok, so in terms of a framework, we could delegate sequences of operations to be executed in parallel to 'runners'. These operation sequences could include our current js tests, and also some different predefined usage patterns that would put more load on the db. For the most extensive testing, it would be nice to have these usage patterns generated probabilistically, and the ordering and synchronization of the different operation sequences could be probabilistic too.

The shell exec stuff would be one way to implement these runners, but it might be cleaner to do it with multithreaded spidermonkey. However, as we know, most prepackaged sm distributions aren't compiled with multithreading enabled – which I believe is why we chose to go the shell exec route for testing collection cloning.

Generated at Thu Feb 08 02:54:13 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.