[SERVER-30009] MapReduce failure: missing ) in parenthetical Created: 06/Jul/17  Updated: 30/Oct/23  Resolved: 31/Aug/17

Status: Closed
Project: Core Server
Component/s: MapReduce
Affects Version/s: 3.2.14, 3.4.6
Fix Version/s: 3.2.18, 3.4.11, 3.5.13

Type: Bug Priority: Major - P3
Reporter: Ben Calev Assignee: Spencer Jackson
Resolution: Fixed Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File server-30009-repro.tgz    
Issue Links:
Backports
Depends
Duplicate
is duplicated by SERVER-31374 If map function string ends in a semi... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4, v3.2
Sprint: Platforms 2017-07-31, Platforms 2017-08-21, Platforms 2017-09-11
Participants:

 Description   
Issue Status as of Jul 13, 2017

ISSUE DESCRIPTION AND IMPACT
SERVER-28323 added strict checks on conversions between Javascript strings and functions. After upgrading to MongoDB 3.4.6, 3.2.14, or 3.2.15 users may encounter the following error message when using mapReduce:

2017-07-10T16:53:32.121-0400 E QUERY    [thread1] Error: map reduce failed:{
    "ok" : 0,
    "errmsg" : "SyntaxError: missing ) in parenthetical @:1:118\n",
    "code" : 139,
    "codeName" : "JSInterpreterFailure"
} :

AFFECTED VERSIONS
MongoDB 3.2.14, 3.2.15, and 3.4.6.

ISSUE STATUS
This ticket tracks work to relax some of these constraints in future versions of MongoDB and provide a more user-friendly error message.

DIAGNOSIS AND WORKAROUNDS
Users encountering this error should examine the inputs to the mapReduce command and ensure they are functions.

As an example, consider the following mapReduce command, which runs on 3.4.5:

> var mapFunction1 = "function() {emit(this.cust_id, this.price);};"
> var reduceFunction1 = "function(keyCustId, valuesPrices) {return Array.sum(valuesPrices);};"
> db.orders.insert({ cust_id: "abc123", ord_date: new Date("Oct 04, 2012"), status: 'A', price: 25, items: [ { sku: "mmm", qty: 5, price: 2.5 }, { sku: "nnn", qty: 5, price: 2.5 } ] })
WriteResult({ "nInserted" : 1 })
> db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
{
    "result" : "map_reduce_example",
    "timeMillis" : 226,
    "counts" : {
        "input" : 2,
        "emit" : 2,
        "reduce" : 1,
        "output" : 1
    },
    "ok" : 1
}

Running the same map-reduce command against 3.4.6 results in:

> db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
2017-07-10T16:53:31.933-0400 I NETWORK  [thread1] trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2017-07-10T16:53:31.934-0400 I NETWORK  [thread1] reconnect 127.0.0.1:27017 (127.0.0.1) ok
2017-07-10T16:53:32.121-0400 E QUERY    [thread1] Error: map reduce failed:{
    "ok" : 0,
    "errmsg" : "SyntaxError: missing ) in parenthetical @:1:118\n",
    "code" : 139,
    "codeName" : "JSInterpreterFailure"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DBCollection.prototype.mapReduce@src/mongo/shell/collection.js:1352:1
@(shell):1:1

The error message appears because the mapReduce functions are strings, not functions. Fixing the types of the mapReduce functions resolves the error:

> var reduceFunction1 = function(keyCustId, valuesPrices) {return Array.sum(valuesPrices);};
> var mapFunction1 = function() {emit(this.cust_id, this.price);};
> db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
{
    "result" : "map_reduce_example",
    "timeMillis" : 247,
    "counts" : {
        "input" : 2,
        "emit" : 2,
        "reduce" : 1,
        "output" : 1
    },
    "ok" : 1
}

Users should consider using native Javascript objects in their application code if the driver supports them.

Original Summary

MapReduce failure with 3.4.6

Original Description

When performing a mapreduce that works on all previous versions including 3.4.5, we are now receiving the error "missing ) in parenthetical map reduce @15:17". A rollback to 3.4.5 from 3.4.6 was required to resolve this.



 Comments   
Comment by Fory Horio [ 30/Nov/17 ]

We just updated to 3.2.18. We are still getting the same error.

2017-11-30T12:07:42.490-0800 I COMMAND [conn13] mr failed, removing collection :: caused by :: 139 SyntaxError: missing ) in parenthetical @:112:0

Comment by Githook User [ 21/Nov/17 ]

Author:

{'name': 'Spencer Jackson', 'username': 'spencerjackson', 'email': 'spencer.jackson@mongodb.com'}

Message: SERVER-30009: Allow JS functions to be terminated with ';'

(cherry picked from commit 8024561b6a73b5b0b56200bdfa3233219ff7fb18)
Branch: v3.4
https://github.com/mongodb/mongo/commit/51e4c2a68701fb00b51beb15c8cf8868c057f035

Comment by Githook User [ 21/Nov/17 ]

Author:

{'name': 'Spencer Jackson', 'username': 'spencerjackson', 'email': 'spencer.jackson@mongodb.com'}

Message: SERVER-30009: Allow JS functions to be terminated with ';'

(cherry picked from commit 8024561b6a73b5b0b56200bdfa3233219ff7fb18)
Branch: v3.2
https://github.com/mongodb/mongo/commit/4c1bae566c0c00f996a2feb16febf84936ecaf6f

Comment by Githook User [ 31/Aug/17 ]

Author:

{'username': 'spencerjackson', 'name': 'Spencer Jackson', 'email': 'spencer.jackson@mongodb.com'}

Message: SERVER-30009: Allow JS functions to be terminated with ';'
Branch: master
https://github.com/mongodb/mongo/commit/8024561b6a73b5b0b56200bdfa3233219ff7fb18

Comment by Edouard Griffiths [X] [ 21/Jul/17 ]

Thanks for pointing out the trailing semicolon thing. Indeed the culprit was the finalizer function

finalizer = Code("""
function(key, value) {
    value = <blah blah blah>;
    return value;
};
""")

Removing the semicolon after the function ending brace solves the problem

Comment by Shane Harvey [ 20/Jul/17 ]

egriffiths1A - Wrapping the function with a bson.Code object does not work around this issue. Whether a string or bson.Code, the JavaScript code still needs to be a valid function definition according to the server. For example, adding a trailing semi-colon results in the same "SyntaxError: missing ) in parenthetical @:1:45" error:

>>> from pymongo import MongoClient
>>> from bson.code import Code
>>> db = MongoClient().aggregation_example
>>> db.orders.insert_one({ "cust_id": "abc123", "price": 25})
ObjectId('5970e49d947616f9815ceb98')
>>> map_str = "function() {emit(this.cust_id, this.price);}"
>>> reduce_str =  "function(id, prices) {return Array.sum(prices);}"
>>> db.orders.map_reduce(map_str, reduce_str, {"inline":1})
{u'counts': {u'input': 1, u'reduce': 0, u'emit': 1, u'output': 1}, u'timeMillis': 15, u'ok': 1.0, u'results': [{u'_id': u'abc123', u'value': 25.0}]}
>>> db.orders.map_reduce(map_str+';', reduce_str+';', {"inline":1})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pymongo/collection.py", line 2090, in map_reduce
    collation=collation)
  File "pymongo/collection.py", line 232, in _command
    collation=collation)
  File "pymongo/pool.py", line 440, in command
    collation=collation)
  File "pymongo/network.py", line 116, in command
    parse_write_concern_error=parse_write_concern_error)
  File "pymongo/helpers.py", line 210, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: SyntaxError: missing ) in parenthetical @:1:45
 
>>> db.orders.map_reduce(Code(map_str+';'), Code(reduce_str+';'), {"inline":1})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pymongo/collection.py", line 2090, in map_reduce
    collation=collation)
  File "pymongo/collection.py", line 232, in _command
    collation=collation)
  File "pymongo/pool.py", line 440, in command
    collation=collation)
  File "pymongo/network.py", line 116, in command
    parse_write_concern_error=parse_write_concern_error)
  File "pymongo/helpers.py", line 210, in _check_command_response
    raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: SyntaxError: missing ) in parenthetical @:1:45

Comment by Edouard Griffiths [X] [ 20/Jul/17 ]

Is there any work around when using PyMongo and define the functions inside a Code object from bson.code like:

reducer = Code("""
function(key, values)

{ ... }

""")

then use it as the reducer argument of PyMongo's map_reduce. Same for mapper and finalizer.

Thanks and best regards,
Edouard.

Comment by Ramon Fernandez Marina [ 11/Jul/17 ]

Thanks for the reproducer oleg@evergage.com; we're discussing whether there's something mongod can do in terms of relaxing these constraints or at least provide a more useful error message.

Regards,
Ramón.

Comment by Oleg Rekutin [ 10/Jul/17 ]

I have confirmed that removing an extra ";" in a failing instance of MapReduce made it work with 3.4.6 again. So I would recommend downgrading the severity of this bug at this point, to perhaps simply major? (Since it can be worked around with simple enough client updates).

Comment by Spencer Jackson [ 10/Jul/17 ]

SERVER-28323 made the server and shell more strict about conversions between strings and functions. I have made a small example which triggers this error message.

The following runs on 3.4.5:

MongoDB Enterprise > var mapFunction1 = "function() {                        emit(this.cust_id, this.price);                    };"
MongoDB Enterprise > var reduceFunction1 = "function(keyCustId, valuesPrices) {                           return Array.sum(valuesPrices);                       };"
MongoDB Enterprise > db.orders.insert({ cust_id: "abc123",      ord_date: new Date("Oct 04, 2012"),      status: 'A',      price: 25,      items: [ { sku: "mmm", qty: 5, price: 2.5 },               { sku: "nnn", qty: 5, price: 2.5 } ] })
WriteResult({ "nInserted" : 1 })
MongoDB Enterprise > db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
{
	"result" : "map_reduce_example",
	"timeMillis" : 226,
	"counts" : {
		"input" : 2,
		"emit" : 2,
		"reduce" : 1,
		"output" : 1
	},
	"ok" : 1
}

Running it against 3.4.6 results in:

MongoDB Enterprise > db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
2017-07-10T16:53:31.933-0400 I NETWORK  [thread1] trying reconnect to 127.0.0.1:27017 (127.0.0.1) failed
2017-07-10T16:53:31.934-0400 I NETWORK  [thread1] reconnect 127.0.0.1:27017 (127.0.0.1) ok
2017-07-10T16:53:32.121-0400 E QUERY    [thread1] Error: map reduce failed:{
	"ok" : 0,
	"errmsg" : "SyntaxError: missing ) in parenthetical @:1:118\n",
	"code" : 139,
	"codeName" : "JSInterpreterFailure"
} :
_getErrorWithCode@src/mongo/shell/utils.js:25:13
DBCollection.prototype.mapReduce@src/mongo/shell/collection.js:1352:1
@(shell):1:1

It's complaining because the map/reduce functions are strings, not functions. Admittedly, the error message is not clear. Fixing the types resolves the error.

MongoDB Enterprise > var reduceFunction1 = function(keyCustId, valuesPrices) {                           return Array.sum(valuesPrices);                       };
MongoDB Enterprise > var mapFunction1 = function() {                        emit(this.cust_id, this.price);                    };
MongoDB Enterprise > db.orders.mapReduce(mapFunction1, reduceFunction1, { out: "map_reduce_example" })
{
	"result" : "map_reduce_example",
	"timeMillis" : 247,
	"counts" : {
		"input" : 2,
		"emit" : 2,
		"reduce" : 1,
		"output" : 1
	},
	"ok" : 1
}

Take a look at your inputs to map reduce, and ensure they are functions.

Comment by Oleg Rekutin [ 10/Jul/17 ]

Attached is a reproducible test case written using Java driver 3.3.0 and Maven.

It appears to be that the problem is due to the "reduce" function having an extra semicolon after the function() {} definition. When I comment out line Server30009Test.java:32 in the attachment, the test passes with MongoDB 3.4.6. It fails with the semicolon on 3.4.6, but passes with it on 3.4.4.

Comment by Ian Springer [ 08/Jul/17 ]

The only issue I see in the 3.4.6 change log that mentions JavaScript or map-reduce is:

https://jira.mongodb.org/browse/SERVER-28323 (Don’t pass JavaScript scopes a function ID number)

Could that by any chance be the cause of the regression?

Comment by Ian Springer [ 08/Jul/17 ]

We are seeing this as well after upgrading to 3.4.6.

Comment by Michael O&#39;Keefe [ 07/Jul/17 ]

We are currently running into the same error on our system. MapReduce tasks which worked on all previous versions, including 3.4.5, are now failing with the same error message:

SyntaxError: missing ) in parenthetical @:653:700:01:01

I'm working on finding a minimal map-reduce command which reproduces this; I'll post it once I have one.

Comment by Kelsey Schubert [ 06/Jul/17 ]

Hi bcalev,

Thanks for reporting this issue. Would you please provide the problematic map reduce command, so we can reproduce and investigate the issue?

Kind regards,
Thomas

Generated at Thu Feb 08 04:22:24 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.