[SERVER-446] finish v8 engine Created: 29/Nov/09  Updated: 12/Jul/16  Resolved: 02/Feb/10

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: 1.3.2

Type: New Feature Priority: Major - P3
Reporter: Eliot Horowitz (Inactive) Assignee: Aaron Staple
Resolution: Done Votes: 7
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-470 concurrency testing Closed
is depended on by SERVER-326 Optimize MapReduce processing Closed
Participants:

 Description   

scons --usev8 builds all binaries with v8 instead of spider monkey.
scons --usev8 test && ./test js would be a good place to start



 Comments   
Comment by ZedroS [ 06/Apr/11 ]

thanks eliot

Comment by Eliot Horowitz (Inactive) [ 04/Apr/11 ]

@Zedross - it is finished. if you want track when we are going to switch, watch SERVER-2407

Comment by ZedroS [ 04/Apr/11 ]

for when is it due ?

Comment by Eliot Horowitz (Inactive) [ 07/Feb/10 ]

passes all tests and is in bb

Comment by auto [ 02/Feb/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 add smokeParallel target
http://github.com/mongodb/mongo/commit/40184615d5f10e8885b9927fce4a94964ca5f58a

Comment by auto [ 02/Feb/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 fix v8 regex conversion
http://github.com/mongodb/mongo/commit/56c790cb76ba471f152317ba962c5894b0330e7e

Comment by auto [ 02/Feb/10 ]

Author:

{'login': 'astaple', 'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 make v8 undefined/null conversion same as sm
http://github.com/mongodb/mongo/commit/ad8292a9f7102a9a4f048710136f297419c5ab5a

Comment by Eliot Horowitz (Inactive) [ 29/Jan/10 ]

now a v8 64-bit linux buildbot slave
it does not seem to pass the tests yet

Comment by Aaron Staple [ 21/Jan/10 ]

Would be cool to have a v8 buildbot. I tried running the js tests yesterday and saw a failure, so there's already been a regression in the last few weeks.

Comment by Aaron Staple [ 29/Dec/09 ]

Okay, I believe I fixed the scope reuse issues with the above commits.

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 MINOR refactor usage of v8 raii objects
http://github.com/mongodb/mongo/commit/453aa1977ebb8a897ae803bc093fa8d09a4afe26

Comment by auto [ 29/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 use v8's raii objects correctly per documentation
http://github.com/mongodb/mongo/commit/91362b9e354970f54a2acab9cf7089bf603713e1

Comment by Eliot Horowitz (Inactive) [ 28/Dec/09 ]

Yes - that's fine for now.
Its really a place holder in case there are things we need to do.

Comment by Aaron Staple [ 28/Dec/09 ]

Ok, I'm trying to make it work and I'd like to know what the semantics of a Scope reset() are. It looks like in the sm engine reset() doesn't actually do anything:

void reset(){
smlock;
assert( _convertor );
return;
<more lines after the return;>

Is that intentional? Is it ok to leave user data sitting in a scope in my v8 implementation?

Comment by Eliot Horowitz (Inactive) [ 28/Dec/09 ]

Currently its definitely expensive because it has to re-run all the js-helpers.
If we could handle that differently, then we can probable remove the cache

Comment by Aaron Staple [ 28/Dec/09 ]

I haven't investigated the details, but v8 expects v8::Context's to be entered in a hierarchical fashion and internally maintains a stack of the active contexts. When a v8::Context is exited, v8 automatically enters the next context in its stack. When we reuse V8Scope's, we can't completely reset the v8::Context's stored within them, and v8 asserts when the ordering relationships are not as expected.

I'm sure there's some way of making this work, I just wanted to make sure it's worth the time.

I assume scope caches were implemented because creating a scope is expensive in spidermonkey. Do we know the same is expensive in v8?

Comment by Eliot Horowitz (Inactive) [ 28/Dec/09 ]

The scope cache is a very big deal for a lot of things.

What's the problem?

For using v8 for testing it ok, but to use v8 as a spidermonkey replacement we need the caches to work.

Comment by Aaron Staple [ 28/Dec/09 ]

Is it ok if I disable the scope cache for v8? The current scope cache implementation doesn't work well with v8's management of entered/exited contexts.

Comment by Eliot Horowitz (Inactive) [ 23/Dec/09 ]

Need to get running in a buildbot environment.

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement gc() in v8 engine
http://github.com/mongodb/mongo/commit/770394a02d11fdf3b62be0c671083fb14826e1a0

Comment by Aaron Staple [ 22/Dec/09 ]

Updating my v8 code fixed the memory problem - must have gotten a bad rev. earlier.

Comment by Eliot Horowitz (Inactive) [ 22/Dec/09 ]

Is it running out of v8 memory or total memory? Is the v8 memory configurable?
If its system memory, then its probably a leak in the v8 side...

Comment by Aaron Staple [ 22/Dec/09 ]

Last failure in the basic tests is that in mr1.js the db process runs out of memory with v8 engine. I can try to investigate myself, but please let me know first if you have any thoughts about the cause of this.

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 clean up whitespace when creating js function in v8 engine
http://github.com/mongodb/mongo/commit/556a55420536d63159c762e377d4d1f640e54340

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 support multi update mode v8
http://github.com/mongodb/mongo/commit/80bd8d169d617a90cd515fd720ba38720066617d

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 make type1.js work with v8 double/int scheme as well as sm
http://github.com/mongodb/mongo/commit/76275b5a6bae403289ab0dba760a3f5022761362

Comment by auto [ 22/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 support storefunc
http://github.com/mongodb/mongo/commit/39967b06090dab2e55d71ab520708e5900b4c3d2

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement new Object.bsonsize( x ) syntax in sm engine
http://github.com/mongodb/mongo/commit/6c7cc48dcd1788a97878f7c7a9ee6e4ea3092f63

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement bsonsize() in v8
http://github.com/mongodb/mongo/commit/643ec0261b21c8053dfabe9a9446d7bdc3ef788d

Comment by Aaron Staple [ 21/Dec/09 ]

Thanks for letting me know - will do

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 save hostname in mongo object
http://github.com/mongodb/mongo/commit/793dfc134b24030ed19e29f4654077dfe56a75ce

Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ]

That's fine - just make sure to update the website docs with a note about 1.3

Comment by Aaron Staple [ 21/Dec/09 ]

Gotcha. Is it ok if we change the syntax to Object.bsonsize( x ) ? That would be much easier to implement in v8 unless we don't care about polluting Object.prototype with an extra key.

Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ]

we definitely need that and cannot get rid of it
used for client debugging in eval, etc...

Comment by Aaron Staple [ 21/Dec/09 ]

Also getting rid of the bsonsize() function – it's only used in one test, and can use the datasize command there instead.

If it's a problem, just let me know.

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 use generic object id validation in v8 engine ObjectId constructor
http://github.com/mongodb/mongo/commit/0e045e2fe1f9dede478a2980d6f4f75b6bc7b0ad

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 extract object id string validation to general engine layer
http://github.com/mongodb/mongo/commit/bbcc6c1d0c80305396f051efdc34935505b614be

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 handles NumberLong type
http://github.com/mongodb/mongo/commit/de9e185d3d0ce38da0da0ff82509a90c383c4870

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 adjust test since v8 reorders numeric named fields
http://github.com/mongodb/mongo/commit/7c005a3fee2104ab00fb1c7c42c7cb5c0bd755b9

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 populate 'args' object on v8 invoke
http://github.com/mongodb/mongo/commit/5d9f70180b38379b45450fdf27ef97319e97745d

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 don't drop undefined elements
http://github.com/mongodb/mongo/commit/ad8698567078da9b0d085dd33e02ca98147afe60

Comment by Aaron Staple [ 21/Dec/09 ]

v8 reorders fields with numeric names

In v8:
> z =

{ "x":1,"4":2 }

;

{ "4" : 2, "x" : 1 }

while in sm:
> z =

{ "x":1,"4":2 } { "x" : 1, "4" : 2 }

I'm assuming this isn't a big deal for us. I'm just going to make the tests more flexible where they are failing currently.

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 allow js client access to MaxKey and MinKey values
http://github.com/mongodb/mongo/commit/ad93048085caf8d79b7850a61b2d0669f2b972e1

Comment by Eliot Horowitz (Inactive) [ 21/Dec/09 ]

lets make v8 work like spider monkey for now

Comment by Aaron Staple [ 21/Dec/09 ]

An alternative solution would be to have both engines just drop undefined values.

Comment by Aaron Staple [ 21/Dec/09 ]

It appears that when spidermonkey is invoked to return a value from a native c++ callback, values which the callback returned as 'undefined' type are converted to 'null' type. As a result, undefined values are converted to null when the db is queried.

This conversion is not performed by v8, and as a result one (or more) of the js tests don't pass, producing obscure output.

Should I make v8 perform the same conversion undefined->null? Alternatively I could just fix the test(s) individually.

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement keySet w/o array comprehension, since v8 doesn't support that
http://github.com/mongodb/mongo/commit/e7e790ee960c7e455b6655064f15a7547d38b039

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 js implementation of keySet()
http://github.com/mongodb/mongo/commit/81b7e051b2c42e47566b9005e3db13e6a59ea68b

Comment by Aaron Staple [ 21/Dec/09 ]

I'm implementing a js version of keySet and adding it to Object rather than Object.prototype to avoid having a new and confusing 'keySet' key in all objects when doing manual iteration. This requires that keySet be called explicitly via Object.keySet() rather than simply doing obj.keySet().

Hopefully this new interface is ok.

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 MINOR enhance BinData round trip test to check data strings containing null bytes
http://github.com/mongodb/mongo/commit/499cb647b65ca9d1b0cbdc27f72359b30bf2bf39

Comment by auto [ 21/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 use js BinData type from v8 engine
http://github.com/mongodb/mongo/commit/455bf0c0492421f8467f88e1f2371cb3e29dc5d9

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 test that BinData js object is being used
http://github.com/mongodb/mongo/commit/15971b9ad92f8c82897046c92b7bf55683f8c95b

Comment by Eliot Horowitz (Inactive) [ 16/Dec/09 ]

Yeah - probably can do something like that actually.
At least can't think of any problems right now.

Comment by Aaron Staple [ 16/Dec/09 ]

Would something like this work for grabbing the keys that would be in keyset?

for( i in z ) { if ( !( i in z._proto_ ) || z[i] != z._proto_[i] )

{ print( i ); }

}

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 don't attempt to convert c++ backed 'external' objects to BSON
http://github.com/mongodb/mongo/commit/c3ea73736f3971adde40f41dfb7fdc9cff9b0a9a

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 fix v8 type() bug involving type hierarchy
http://github.com/mongodb/mongo/commit/4cf6964975f5e7a87c2ac4a0f1e59a98c07f1666

Comment by Eliot Horowitz (Inactive) [ 16/Dec/09 ]

I believe its impossible.
i'm pretty sure keySet() is supposed to only return fields on the current object, not including prototype fields.
and i don't think there is a way to do that in pure js.
if there is - then sure.

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 add DBPointer to v8
http://github.com/mongodb/mongo/commit/643ad02d77769b59a458116b123f3902f8867cc6

Comment by Aaron Staple [ 16/Dec/09 ]

It looks like the keySet() function is hardcoded in the spidermonkey engine currently. Would it make sense to write this in javascript instead?

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 don't attempt to write protect arrays in v8
http://github.com/mongodb/mongo/commit/78a9e77c33a0276893ec2412bc4305195a5b5665

Comment by Eliot Horowitz (Inactive) [ 16/Dec/09 ]

Yeah - just don't worry about write protection for arrays
not that important.

Comment by Aaron Staple [ 16/Dec/09 ]

It looks like it's impossible in v8 to implement access interceptors on a value which v8 represents internally as an array. Right now I'm using access interceptors to implement write protection, but to do so I am converting BSON arrays to v8 objects instead of v8 arrays (and then back into BSON arrays round-trip). However, with this implementation we can't treat the BSON arrays as arrays in js code (for example forEach() won't work, since v8 thinks it's dealing with a non-array object). I could conceivably use v8 arrays instead of v8 objects in cases where write protection isn't required, but then we'd have two different array formats and that could be confusing to a js programmer.

Inheriting from Array doesn't work correctly in v8, so that isn't an option for getting around this problem. I could post a question on the v8 user group, but I may not get any helpful feedback.

What should I do? Just disable write protection for arrays?

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 support bindata round trip
http://github.com/mongodb/mongo/commit/385bb326413e01ba27e62ab582e9b57f9251c269

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement old style dbref support in v8
http://github.com/mongodb/mongo/commit/df01c097d655133080dabb4b40f7c1bc8754b522

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 support conversion of Code, CodeWScope to v8 function
http://github.com/mongodb/mongo/commit/25b56fbba919cba1c370fc3dd9ee72b7deeef7ef

Comment by auto [ 16/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 v8 round trip array support
http://github.com/mongodb/mongo/commit/b9ddc3e9ef3b1e234bc84dd6a80326f0196a249d

Comment by Eliot Horowitz (Inactive) [ 15/Dec/09 ]

i think this is ok.
i can theoretically see a problem this would could cause, but don't think its relevant.

Comment by Aaron Staple [ 15/Dec/09 ]

Right now in sm, if a double that could be represented as an int is moved to sm and back, it remains a double. Right now in v8, in the same situation, the double is returned as an int without loss of information. Is this ok?

I'm just asking because the jstests don't check this behavior in TypeConservation though they test several other cases (I added a commented out -D- section to demonstrate this particular issue) and the implementation for the current v8 behavior is nicer than would be required if we needed to keep track of the original type.

Comment by auto [ 15/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 simple int/double differentiation
http://github.com/mongodb/mongo/commit/bec620a87a8ea0974431952d0aba291372002558

Comment by auto [ 15/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 Implemented special db types round trip support for v8
http://github.com/mongodb/mongo/commit/ebdb8282a2150c22a280705e99240d97b68e46d0

Comment by auto [ 15/Dec/09 ]

Author:

{'name': 'Aaron', 'email': 'aaron@10gen.com'}

Message: SERVER-446 implement readOnly mode in v8 engine
http://github.com/mongodb/mongo/commit/8add9a17b9fe9d4a0a647c982326a2279aac2d23

Comment by Eliot Horowitz (Inactive) [ 09/Dec/09 ]

No - that APi is not really usable, and I think v8 and safari are neck and neck for speed.
safari also doesn't support a lot of the platforms we need.

Comment by Alexander Kellett [ 09/Dec/09 ]

AFAIU v8 is still quite a bit slower than the C++ based engine in Safari. Have you considered using that instead?

Generated at Thu Feb 08 02:54:08 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.