Issue Details (XML | Word | Printable)

Key: SERVER-446
Type: New Feature New Feature
Status: Closed Closed
Resolution: Fixed
Priority: Major Major
Assignee: aaron
Reporter: Eliot Horowitz
Votes: 7
Watchers: 5
Operations

If you were logged in you would be able to see more operations.
Core Server

finish v8 engine

Created: 29/Nov/09 10:14 PM   Updated: 12/Feb/10 02:46 PM
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: 1.3.2

Issue Links:
Depends

Backport: No
Roadmap: 1.4


 Description  « Hide
scons --usev8 builds all binaries with v8 instead of spider monkey.
scons --usev8 test && ./test js would be a good place to start


 All   Comments   Change History   git Commits      Sort Order: Ascending order - Click to sort in descending order
Alexander Kellett added a comment - 09/Dec/09 10:21 AM - edited
AFAIU v8 is still quite a bit slower than the C++ based engine in Safari. Have you considered using that instead?

Eliot Horowitz added a comment - 09/Dec/09 10:29 AM
No - that APi is not really usable, and I think v8 and safari are neck and neck for speed.
safari also doesn't support a lot of the platforms we need.

auto added a comment - 15/Dec/09 06:53 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement readOnly mode in v8 engine
http://github.com/mongodb/mongo/commit/8add9a17b9fe9d4a0a647c982326a2279aac2d23

auto added a comment - 15/Dec/09 07:55 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 Implemented special db types round trip support for v8
http://github.com/mongodb/mongo/commit/ebdb8282a2150c22a280705e99240d97b68e46d0

auto added a comment - 15/Dec/09 08:41 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 simple int/double differentiation
http://github.com/mongodb/mongo/commit/bec620a87a8ea0974431952d0aba291372002558

aaron added a comment - 15/Dec/09 08:48 PM
Right now in sm, if a double that could be represented as an int is moved to sm and back, it remains a double. Right now in v8, in the same situation, the double is returned as an int without loss of information. Is this ok?

I'm just asking because the jstests don't check this behavior in TypeConservation though they test several other cases (I added a commented out --D-- section to demonstrate this particular issue) and the implementation for the current v8 behavior is nicer than would be required if we needed to keep track of the original type.

Eliot Horowitz added a comment - 15/Dec/09 08:57 PM
i think this is ok.
i can theoretically see a problem this would could cause, but don't think its relevant.

auto added a comment - 16/Dec/09 02:44 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 round trip array support
http://github.com/mongodb/mongo/commit/b9ddc3e9ef3b1e234bc84dd6a80326f0196a249d

auto added a comment - 16/Dec/09 02:44 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 support conversion of Code, CodeWScope to v8 function
http://github.com/mongodb/mongo/commit/25b56fbba919cba1c370fc3dd9ee72b7deeef7ef

auto added a comment - 16/Dec/09 02:44 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement old style dbref support in v8
http://github.com/mongodb/mongo/commit/df01c097d655133080dabb4b40f7c1bc8754b522

auto added a comment - 16/Dec/09 02:44 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 support bindata round trip
http://github.com/mongodb/mongo/commit/385bb326413e01ba27e62ab582e9b57f9251c269

aaron added a comment - 16/Dec/09 05:32 PM
It looks like it's impossible in v8 to implement access interceptors on a value which v8 represents internally as an array. Right now I'm using access interceptors to implement write protection, but to do so I am converting BSON arrays to v8 objects instead of v8 arrays (and then back into BSON arrays round-trip). However, with this implementation we can't treat the BSON arrays as arrays in js code (for example forEach() won't work, since v8 thinks it's dealing with a non-array object). I could conceivably use v8 arrays instead of v8 objects in cases where write protection isn't required, but then we'd have two different array formats and that could be confusing to a js programmer.

Inheriting from Array doesn't work correctly in v8, so that isn't an option for getting around this problem. I could post a question on the v8 user group, but I may not get any helpful feedback.

What should I do? Just disable write protection for arrays?

Eliot Horowitz added a comment - 16/Dec/09 05:35 PM
Yeah - just don't worry about write protection for arrays
not that important.

auto added a comment - 16/Dec/09 05:50 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 don't attempt to write protect arrays in v8
http://github.com/mongodb/mongo/commit/78a9e77c33a0276893ec2412bc4305195a5b5665

aaron added a comment - 16/Dec/09 09:00 PM
It looks like the keySet() function is hardcoded in the spidermonkey engine currently. Would it make sense to write this in javascript instead?

auto added a comment - 16/Dec/09 09:04 PM

Eliot Horowitz added a comment - 16/Dec/09 09:04 PM
I believe its impossible.
i'm pretty sure keySet() is supposed to only return fields on the current object, not including prototype fields.
and i don't think there is a way to do that in pure js.
if there is - then sure.

auto added a comment - 16/Dec/09 09:04 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 fix v8 type() bug involving type hierarchy
http://github.com/mongodb/mongo/commit/4cf6964975f5e7a87c2ac4a0f1e59a98c07f1666

auto added a comment - 16/Dec/09 09:04 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 don't attempt to convert c++ backed 'external' objects to BSON
http://github.com/mongodb/mongo/commit/c3ea73736f3971adde40f41dfb7fdc9cff9b0a9a

aaron added a comment - 16/Dec/09 09:14 PM
Would something like this work for grabbing the keys that would be in keyset?

for( i in z ) { if ( !( i in z.__proto__ ) || z[i] != z.__proto__[i] ) { print( i ); } }

Eliot Horowitz added a comment - 16/Dec/09 09:18 PM
Yeah - probably can do something like that actually.
At least can't think of any problems right now.

auto added a comment - 16/Dec/09 09:38 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 test that BinData js object is being used
http://github.com/mongodb/mongo/commit/15971b9ad92f8c82897046c92b7bf55683f8c95b

auto added a comment - 21/Dec/09 12:59 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 use js BinData type from v8 engine
http://github.com/mongodb/mongo/commit/455bf0c0492421f8467f88e1f2371cb3e29dc5d9

auto added a comment - 21/Dec/09 12:59 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 MINOR enhance BinData round trip test to check data strings containing null bytes
http://github.com/mongodb/mongo/commit/499cb647b65ca9d1b0cbdc27f72359b30bf2bf39

aaron added a comment - 21/Dec/09 02:01 PM
I'm implementing a js version of keySet and adding it to Object rather than Object.prototype to avoid having a new and confusing 'keySet' key in all objects when doing manual iteration. This requires that keySet be called explicitly via Object.keySet() rather than simply doing obj.keySet().

Hopefully this new interface is ok.

auto added a comment - 21/Dec/09 02:21 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 js implementation of keySet()
http://github.com/mongodb/mongo/commit/81b7e051b2c42e47566b9005e3db13e6a59ea68b

auto added a comment - 21/Dec/09 02:34 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement keySet w/o array comprehension, since v8 doesn't support that
http://github.com/mongodb/mongo/commit/e7e790ee960c7e455b6655064f15a7547d38b039

aaron added a comment - 21/Dec/09 04:56 PM
It appears that when spidermonkey is invoked to return a value from a native c++ callback, values which the callback returned as 'undefined' type are converted to 'null' type. As a result, undefined values are converted to null when the db is queried.

This conversion is not performed by v8, and as a result one (or more) of the js tests don't pass, producing obscure output.

Should I make v8 perform the same conversion undefined->null? Alternatively I could just fix the test(s) individually.

aaron added a comment - 21/Dec/09 05:05 PM
An alternative solution would be to have both engines just drop undefined values.

Eliot Horowitz added a comment - 21/Dec/09 05:10 PM
lets make v8 work like spider monkey for now

auto added a comment - 21/Dec/09 05:38 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 allow js client access to MaxKey and MinKey values
http://github.com/mongodb/mongo/commit/ad93048085caf8d79b7850a61b2d0669f2b972e1

aaron added a comment - 21/Dec/09 07:08 PM
v8 reorders fields with numeric names

In v8:
> z = { "x":1,"4":2 };
{ "4" : 2, "x" : 1 }

while in sm:
> z = { "x":1,"4":2 }
{ "x" : 1, "4" : 2 }


I'm assuming this isn't a big deal for us. I'm just going to make the tests more flexible where they are failing currently.

auto added a comment - 21/Dec/09 07:10 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 don't drop undefined elements
http://github.com/mongodb/mongo/commit/ad8698567078da9b0d085dd33e02ca98147afe60

auto added a comment - 21/Dec/09 07:10 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 populate 'args' object on v8 invoke
http://github.com/mongodb/mongo/commit/5d9f70180b38379b45450fdf27ef97319e97745d

auto added a comment - 21/Dec/09 07:10 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 adjust test since v8 reorders numeric named fields
http://github.com/mongodb/mongo/commit/7c005a3fee2104ab00fb1c7c42c7cb5c0bd755b9

auto added a comment - 21/Dec/09 10:52 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 handles NumberLong type
http://github.com/mongodb/mongo/commit/de9e185d3d0ce38da0da0ff82509a90c383c4870

auto added a comment - 21/Dec/09 10:52 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 extract object id string validation to general engine layer
http://github.com/mongodb/mongo/commit/bbcc6c1d0c80305396f051efdc34935505b614be

auto added a comment - 21/Dec/09 10:52 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 use generic object id validation in v8 engine ObjectId constructor
http://github.com/mongodb/mongo/commit/0e045e2fe1f9dede478a2980d6f4f75b6bc7b0ad

aaron added a comment - 21/Dec/09 11:11 PM
Also getting rid of the bsonsize() function -- it's only used in one test, and can use the datasize command there instead.

If it's a problem, just let me know.

Eliot Horowitz added a comment - 21/Dec/09 11:14 PM
we definitely need that and cannot get rid of it
used for client debugging in eval, etc...

aaron added a comment - 21/Dec/09 11:22 PM
Gotcha. Is it ok if we change the syntax to Object.bsonsize( x ) ? That would be much easier to implement in v8 unless we don't care about polluting Object.prototype with an extra key.

Eliot Horowitz added a comment - 21/Dec/09 11:25 PM
That's fine - just make sure to update the website docs with a note about 1.3

auto added a comment - 21/Dec/09 11:27 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 save hostname in mongo object
http://github.com/mongodb/mongo/commit/793dfc134b24030ed19e29f4654077dfe56a75ce

aaron added a comment - 21/Dec/09 11:28 PM
Thanks for letting me know - will do

auto added a comment - 21/Dec/09 11:59 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement bsonsize() in v8
http://github.com/mongodb/mongo/commit/643ec0261b21c8053dfabe9a9446d7bdc3ef788d

auto added a comment - 21/Dec/09 11:59 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement new Object.bsonsize( x ) syntax in sm engine
http://github.com/mongodb/mongo/commit/6c7cc48dcd1788a97878f7c7a9ee6e4ea3092f63

auto added a comment - 22/Dec/09 01:54 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 v8 support storefunc
http://github.com/mongodb/mongo/commit/39967b06090dab2e55d71ab520708e5900b4c3d2

auto added a comment - 22/Dec/09 01:54 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 make type1.js work with v8 double/int scheme as well as sm
http://github.com/mongodb/mongo/commit/76275b5a6bae403289ab0dba760a3f5022761362

auto added a comment - 22/Dec/09 01:54 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 support multi update mode v8
http://github.com/mongodb/mongo/commit/80bd8d169d617a90cd515fd720ba38720066617d

auto added a comment - 22/Dec/09 01:54 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 clean up whitespace when creating js function in v8 engine
http://github.com/mongodb/mongo/commit/556a55420536d63159c762e377d4d1f640e54340

aaron added a comment - 22/Dec/09 02:04 PM
Last failure in the basic tests is that in mr1.js the db process runs out of memory with v8 engine. I can try to investigate myself, but please let me know first if you have any thoughts about the cause of this.

Eliot Horowitz added a comment - 22/Dec/09 02:09 PM
Is it running out of v8 memory or total memory? Is the v8 memory configurable?
If its system memory, then its probably a leak in the v8 side...

aaron added a comment - 22/Dec/09 03:00 PM
Updating my v8 code fixed the memory problem - must have gotten a bad rev. earlier.

auto added a comment - 22/Dec/09 03:05 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 implement gc() in v8 engine
http://github.com/mongodb/mongo/commit/770394a02d11fdf3b62be0c671083fb14826e1a0

Eliot Horowitz added a comment - 23/Dec/09 03:30 PM
Need to get running in a buildbot environment.

aaron added a comment - 28/Dec/09 02:45 PM
Is it ok if I disable the scope cache for v8? The current scope cache implementation doesn't work well with v8's management of entered/exited contexts.

Eliot Horowitz added a comment - 28/Dec/09 03:14 PM
The scope cache is a very big deal for a lot of things.

What's the problem?

For using v8 for testing it ok, but to use v8 as a spidermonkey replacement we need the caches to work.

aaron added a comment - 28/Dec/09 03:44 PM
I haven't investigated the details, but v8 expects v8::Context's to be entered in a hierarchical fashion and internally maintains a stack of the active contexts. When a v8::Context is exited, v8 automatically enters the next context in its stack. When we reuse V8Scope's, we can't completely reset the v8::Context's stored within them, and v8 asserts when the ordering relationships are not as expected.

I'm sure there's some way of making this work, I just wanted to make sure it's worth the time.

I assume scope caches were implemented because creating a scope is expensive in spidermonkey. Do we know the same is expensive in v8?

Eliot Horowitz added a comment - 28/Dec/09 03:56 PM
Currently its definitely expensive because it has to re-run all the js-helpers.
If we could handle that differently, then we can probable remove the cache

aaron added a comment - 28/Dec/09 04:38 PM
Ok, I'm trying to make it work and I'd like to know what the semantics of a Scope reset() are. It looks like in the sm engine reset() doesn't actually do anything:

        void reset(){
            smlock;
            assert( _convertor );
            return;
            <more lines after the return;>

Is that intentional? Is it ok to leave user data sitting in a scope in my v8 implementation?

Eliot Horowitz added a comment - 28/Dec/09 04:40 PM
Yes - that's fine for now.
Its really a place holder in case there are things we need to do.

auto added a comment - 29/Dec/09 02:38 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 use v8's raii objects correctly per documentation
http://github.com/mongodb/mongo/commit/91362b9e354970f54a2acab9cf7089bf603713e1

auto added a comment - 29/Dec/09 02:38 PM
Author:{'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 MINOR refactor usage of v8 raii objects
http://github.com/mongodb/mongo/commit/453aa1977ebb8a897ae803bc093fa8d09a4afe26

aaron added a comment - 29/Dec/09 02:40 PM
Okay, I believe I fixed the scope reuse issues with the above commits.

aaron added a comment - 21/Jan/10 07:29 PM
Would be cool to have a v8 buildbot. I tried running the js tests yesterday and saw a failure, so there's already been a regression in the last few weeks.

Eliot Horowitz added a comment - 29/Jan/10 03:12 PM
now a v8 64-bit linux buildbot slave
it does not seem to pass the tests yet

auto added a comment - 02/Feb/10 01:10 PM
Author:{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 make v8 undefined/null conversion same as sm
http://github.com/mongodb/mongo/commit/ad8292a9f7102a9a4f048710136f297419c5ab5a

auto added a comment - 02/Feb/10 01:10 PM
Author:{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 fix v8 regex conversion
http://github.com/mongodb/mongo/commit/56c790cb76ba471f152317ba962c5894b0330e7e

auto added a comment - 02/Feb/10 01:10 PM
Author:{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}
Message: SERVER-446 add smokeParallel target
http://github.com/mongodb/mongo/commit/40184615d5f10e8885b9927fce4a94964ca5f58a

Eliot Horowitz added a comment - 07/Feb/10 10:46 AM
passes all tests and is in bb