[SERVER-17453] warn that db.eval() / eval command is deprecated Created: 03/Mar/15  Updated: 14/Jan/16  Resolved: 30/Mar/15

Status: Closed
Project: Core Server
Component/s: JavaScript
Affects Version/s: None
Fix Version/s: 3.0.3, 3.1.1

Type: Task Priority: Major - P3
Reporter: Daniel Pasette (Inactive) Assignee: Samantha Ritter (Inactive)
Resolution: Done Votes: 2
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-12742 remove collection.copyTo test shell h... Closed
Duplicate
duplicates DOCS-4899 document eval is deprecated Closed
Related
is related to DRIVERS-225 Deprecate eval helpers Closed
Backwards Compatibility: Minor Change
Backport Completed:
Sprint: Platform 1 04/03/15
Participants:

 Description   
Issue Status as of Apr 29, 2015

ISSUE SUMMARY
The eval operation that permits clients to execute is deprecated in 3.0 and is currently planned for removal in an upcoming release. This change adds additional warning messages regarding the deprecation to the mongo shell and the eval command.

USER IMPACT
This change just adds additional logging, but users depending on eval should start planning for its eventual removal.

FIX VERSION
The change is included in the 3.0.3 production release.

Original description

Add a warning msg to server and mongo repl log when invoking the eval command, aka $eval or db.eval().



 Comments   
Comment by Sercan Özdemir [ 14/Jan/16 ]

@Ramon Fernandez I know you still have the eval functionality but it's madness to use it since you are about to remove this command, thank you for clarifying.

@Ian Beaver thank you I'm going to watch it.

Comment by Ian Beaver [ 13/Jan/16 ]

rsercano, see SERVER-20510 for the discussion on an eval replacement.

Comment by Ramon Fernandez Marina [ 13/Jan/16 ]

rsercano, we're discussing replacements that can improve on the limitations of eval, but there are no further details I can provide. Note that eval is part of our latest stable release, MongoDB 3.2.

Regards,
Ramón.

Comment by Sercan Özdemir [ 29/Dec/15 ]

Hello,

Are you planning to serve a similar functionality ?
I'm developing a mongodb client which works with meteor-js (internally node-js mongodb driver) and eval was a perfect helper for me.

Regards,
Sercan

Comment by Ramon Fernandez Marina [ 25/Jun/15 ]

h.lassiege@gmail.com, looks like the latest update to Mongeez predates MongoDB 3.0, so there will be other changes that Mongeez may need to undertake to 3.0 before having any issues with eval. Note that eval will continue to work with MongoDB 2.6.

Regards,
Ramón.

Comment by Ian Beaver [ 23/Jun/15 ]

Are you planning any alternatives to eval? Mongo needs to have an equivalent to stored procedures or it will be unusable for many of our use cases. M/R and the aggregation pipelines are not alternatives as they are not designed for individual document operations but group and aggregation operations. Stored procedures exist because there are many cases where changes need to be made that are dependent on existing data and it is not efficient to do that outside of the DB.

A very common use case is you need to modify every document in a collection based on some attribute of each document. For example: each document has a rank score or other numerical attr of some kind, and you want to normalize that score based on the max value in the collection. To use client side code, as you recommend, means pulling down every single document or specific fields of each in the collection, reading the score and dividing by the max, and writing the result back. We have many collections with tens to hundreds of millions of documents and do many similar operation with them. To do this client side is a non-starter with the IO and bandwidth needed to ship data back and forth between the client and servers. Running some basic tests with only ~100k documents this simple normalization operation is 10x-20x slower running client side, not to mention all the additional bandwidth used. This is in AWS with r3.8xlarge DB and client instances and SSD volumes. We re-wrote lots of our applications to use $eval because of this performance issue.

If Mongo wants to position itself as a scaleable database for analytics, it has to provide some mechanism for executing arbitrary functions on the data within the server, and ideally one that works with shards. It doesn't need to be JS and it doesn't have to be embedded in the storage engine, but even a streaming model like Hadoop where each node executes a script on its partition of the data just using stdin/stdout would be fine. Pulling all of the data out of the database over the network to write it back again with some minor changes is not a usable strategy. One of the major wins of horizontal scaling is to be able to do the processing where the data lives and have that processing power scale with the data storage.

Comment by Hugo Lassiège [ 22/Jun/15 ]

Hi,

Regarding this issue, I guess that tools like Mongeez will not work anymore when eval will be removed, is it correct ?
(https://github.com/secondmarket/mongeez)

Do you know any alternative ?

Comment by Githook User [ 20/Apr/15 ]

Author:

{u'username': u'samantharitter', u'name': u'Samantha Ritter', u'email': u'samantha.ritter@10gen.com'}

Message: SERVER-17453 eval command is deprecated

(cherry picked from commit 465bb26c0fb0f4731f4dbb5e09e0a791177bbc64)
Branch: v3.0
https://github.com/mongodb/mongo/commit/ad6f455452db9dd395a0ba71ec38d34848770f3d

Comment by Githook User [ 30/Mar/15 ]

Author:

{u'username': u'samantharitter', u'name': u'Samantha Ritter', u'email': u'samantha.ritter@10gen.com'}

Message: SERVER-17453 eval command is deprecated
Branch: master
https://github.com/mongodb/mongo/commit/465bb26c0fb0f4731f4dbb5e09e0a791177bbc64

Comment by Dave Workman [ 26/Mar/15 ]

> If it turns out that the difference in performance is the extra latency (since both your code and eval passed in code run JS) - in which case you could achieve better performance running your node closer to the server.
The tests I've been running have been on an SSD with Node and Mongo on my machine. I can't see where there I could reduce the latency any further in that. It could be overhead of the complexity of the app, but I'm not sure. I attempted to use the C++ connector but gave up in frustration trying to solve boost linking problems.

> The schema choice you link to is not one that will be able to give you an efficient way to fetch all children of a node (i.e. it requires multiple queries at the tradeoff of less maintenance on changes in hierarchy).
Yeah, I said that. Changes in hierarchy are done much more frequently than removal of nodes in our system. Doesn't mean it never gets done just much less frequently. I'd rather have a performance hit 10% of the time than 90% of the time.

Is there any plan to introduce an API that can match this performance?

Comment by Asya Kamsky [ 20/Mar/15 ]

If it turns out that the difference in performance is the extra latency (since both your code and eval passed in code run JS) - in which case you could achieve better performance running your node closer to the server. The schema choice you link to is not one that will be able to give you an efficient way to fetch all children of a node (i.e. it requires multiple queries at the tradeoff of less maintenance on changes in hierarchy).

Comment by Dave Workman [ 20/Mar/15 ]

Not recommended? This page begs to differ: http://docs.mongodb.org/manual/tutorial/model-tree-structures-with-parent-references/

When we first started building our app 2 years ago, we took a serious look at all of the different approaches (outlined on docs.mongodb.org) and decided to go with parent references, as it gave us the best performance for what our users do most often, move branches around. If we picked ancestors, child references or nested sets(all of the other options outlined on docs.mongodb.org) we'd have exactly the same performance issues, but more often! There is a drawback to each one of the hierarchy schema designs, we picked the one that, for our use case, performed the best for everyday tasks.

I'm not opposed to removing eval, I'm opposed to not replacing it with something that gives equal performance, forgetting my use-case, as there are a thousand others. If I have some scheduled data reduction that takes on average 12 hours to complete on a client, but 2 with eval, why remove such a useful feature without giving us an equally performant (or better) option?

Comment by Asya Kamsky [ 20/Mar/15 ]

davebarbarian24

That's not a recommended way to solve this use case - docs.mongodb.org have several examples of schema design for representing hierarchy/trees that allow you to get this sort of information in a single query. I would recommend a different schema to improve performance.

Comment by Dave Workman [ 19/Mar/15 ]

I'm currently using eval for performance reasons. My database has a tree structure with parent references. I often need to find out all the children of a particular branch of the tree (using a recursive call). Doing these queries with the Node.js connector and concatenating their IDs together takes on average 6x times longer than my stored Javascript function that does the same thing. Without this my app doesn't scale very well.

Comment by rgpublic [ 18/Mar/15 ]

Sigh. Another beloved Mongo feature goes away... Though I fully understand the reasoning behind it. I'd like to add here that at least SERVER-458 should be implemented before eval is removed. When you want to do complex updating on lots of rows simple client code just isn't fast enough. And a client process (think of web server process) can much easier be interrupted - leaving you with half of the rows changed and half of the rows not. We're using eval right now to avoid this. Also, it's sometimes great to be able to use javascript for some changes, not your usual client language (PHP etc)...

Comment by Matt Parlane [ 05/Mar/15 ]

Personally, nothing – I'm not using eval at all, but I would like the option in the future.

The way I see it, server-side logic, in the form of stored procedures, is a big deal in the enterprise world, which you've made no secret that you're going after. I think you guys should look seriously at solving the problems and making eval a first-class citizen rather than just removing it.

You've got a tight, high-performance integration with (arguably) the fastest implementation of (arguably) the most popular programming language in the world, that sounds like something you should embrace and promote, not shun.

Comment by Daniel Pasette (Inactive) [ 04/Mar/15 ]

Hi Matt,
There are a number of issues with the eval command.

  1. Compatibility: eval was developed before the introduction of sharding. The eval command does not work on sharded clusters at all.
  2. Performance: Takes a global lock by default. If eval is with the noLock option, it implies that the javascript could just as easily be run from a client program.
  3. Security: You must have access to all actions on all resources in order to run eval. Security best practices depend on being able to segment privileges to actions, and eval gives too broad a range of powers.
  4. Maintenance: The server must move to a model where javascript is executed "out of process" for a number of reasons; maintaining support for eval makes this very difficult.

I should point out that this does not mean a drop of support for all server side javascript. $where and map/reduce are still supported.

Can you tell me what you're using eval to accomplish?

Comment by Matt Parlane [ 04/Mar/15 ]

Anyone care to provide some rationale behind the deprecation?

Comment by Andy Schwerin [ 04/Mar/15 ]

This ticket covers the db.eval() shell helper and the eval/$eval command. The latter form the basis for the implementation of the former.

Comment by Matt Parlane [ 03/Mar/15 ]

Is the $eval command also deprecated?

Generated at Thu Feb 08 03:44:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.