[SERVER-60770] $function does not accept native functions Created: 18/Oct/21  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: None
Affects Version/s: 5.0.3
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Wojciech Waga Assignee: Backlog - Query Execution
Resolution: Unresolved Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: File repro.js    
Assigned Teams:
Query Execution
Operating System: ALL
Participants:

 Description   

For the following pipeline:

db.listings.updateMany({}, [{ $set: { "host_verifications": { $function: {args: ["$host_verifications"], lang: "js", body: JSON.parse } } } }])

the "$function" operator does not seem to work correctly and returns a rather hard to decipher error: "MongoServerError: SyntaxError: missing ] after element list" . JSON.parse is a widely used function and certainly a valid one for lang type "js". The problem here is that instead of treating  functions as first-class functions (the way Javascript does it) and being able to pass them as an argument Mongo seems to call toString() method instead. For this particular function calling toString() returns: "function parse() { [native code] }" which inserted into the pipeline as body yields the same error.

There is a possible workaround for this:

body: function(x){return JSON.parse(x);}

Here the server has no issues with calling native code function.

Looks like little extra logic should be able to handle such cases even if hard to fix in the core. Making the error message more readable would certainly help as well. 



 Comments   
Comment by Kyle Suarez [ 09/Nov/21 ]

Adding this ticket to the backlog, to examine the function parsing code and improve the error message here to be more clear. Please continue to watch this ticket for updates.

Comment by Joe Kanaan [ 04/Nov/21 ]

Moving this to Query Execution for review.

Comment by Edwin Zhou [ 03/Nov/21 ]

Hi wojciech.waga@gmail.com,

Thanks for your report. I've been able to reproduce this issue and while I believe the workaround is a sound solution, I agree that we can clarify the error message and also improve the documentation around $function.

I've uploaded repro.js which will attempt to pass 3 different functions expressions into the body field of $function

1. function (x) {return JSON.stringify(x)},
2. JSON.stringify,
3. (x) => JSON.stringify(x),

These are the respective results:

1. [ { testField: '{"a":"string","b":"string"}' } ]
2. MongoServerError: SyntaxError: missing ] after element list
    at /usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/operations/update.js:110:33
    at /usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/cmap/connection_pool.js:253:25
    at handleOperationResult (/usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/sdam/server.js:397:9)
    at operationDescription.cb (/usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/cmap/connection.js:571:17)
    at MessageStream.messageHandler (/usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/cmap/connection.js:514:9)
    at MessageStream.emit (events.js:400:28)
    at MessageStream.emit (domain.js:532:15)
    at processIncomingData (/usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/cmap/message_stream.js:108:16)
    at MessageStream._write (/usr/local/Cellar/mongosh/1.0.3/libexec/lib/node_modules/@mongosh/cli-repl/node_modules/mongodb/lib/cmap/message_stream.js:28:9)
    at writeOrBuffer (internal/streams/writable.js:358:12) {
  index: 0,
  code: 139
}
3. [ { testField: Code("x => JSON.stringify(x)") } ]

I will pass this along to the query team to further investigate.

Best,
Edwin

Generated at Thu Feb 08 05:50:42 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.