[SERVER-17453] warn that db.eval() / eval command is deprecated Created: 03/Mar/15 Updated: 14/Jan/16 Resolved: 30/Mar/15 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | JavaScript |
| Affects Version/s: | None |
| Fix Version/s: | 3.0.3, 3.1.1 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Daniel Pasette (Inactive) | Assignee: | Samantha Ritter (Inactive) |
| Resolution: | Done | Votes: | 2 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||||||||||
| Backport Completed: | |||||||||||||||||||||||||
| Sprint: | Platform 1 04/03/15 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Description |
| Comments |
| Comment by Sercan Özdemir [ 14/Jan/16 ] |
|
@Ramon Fernandez I know you still have the eval functionality but it's madness to use it since you are about to remove this command, thank you for clarifying. @Ian Beaver thank you I'm going to watch it. |
| Comment by Ian Beaver [ 13/Jan/16 ] |
|
rsercano, see |
| Comment by Ramon Fernandez Marina [ 13/Jan/16 ] |
|
rsercano, we're discussing replacements that can improve on the limitations of eval, but there are no further details I can provide. Note that eval is part of our latest stable release, MongoDB 3.2. Regards, |
| Comment by Sercan Özdemir [ 29/Dec/15 ] |
|
Hello, Are you planning to serve a similar functionality ? Regards, |
| Comment by Ramon Fernandez Marina [ 25/Jun/15 ] |
|
h.lassiege@gmail.com, looks like the latest update to Mongeez predates MongoDB 3.0, so there will be other changes that Mongeez may need to undertake to 3.0 before having any issues with eval. Note that eval will continue to work with MongoDB 2.6. Regards, |
| Comment by Ian Beaver [ 23/Jun/15 ] |
|
Are you planning any alternatives to eval? Mongo needs to have an equivalent to stored procedures or it will be unusable for many of our use cases. M/R and the aggregation pipelines are not alternatives as they are not designed for individual document operations but group and aggregation operations. Stored procedures exist because there are many cases where changes need to be made that are dependent on existing data and it is not efficient to do that outside of the DB. A very common use case is you need to modify every document in a collection based on some attribute of each document. For example: each document has a rank score or other numerical attr of some kind, and you want to normalize that score based on the max value in the collection. To use client side code, as you recommend, means pulling down every single document or specific fields of each in the collection, reading the score and dividing by the max, and writing the result back. We have many collections with tens to hundreds of millions of documents and do many similar operation with them. To do this client side is a non-starter with the IO and bandwidth needed to ship data back and forth between the client and servers. Running some basic tests with only ~100k documents this simple normalization operation is 10x-20x slower running client side, not to mention all the additional bandwidth used. This is in AWS with r3.8xlarge DB and client instances and SSD volumes. We re-wrote lots of our applications to use $eval because of this performance issue. If Mongo wants to position itself as a scaleable database for analytics, it has to provide some mechanism for executing arbitrary functions on the data within the server, and ideally one that works with shards. It doesn't need to be JS and it doesn't have to be embedded in the storage engine, but even a streaming model like Hadoop where each node executes a script on its partition of the data just using stdin/stdout would be fine. Pulling all of the data out of the database over the network to write it back again with some minor changes is not a usable strategy. One of the major wins of horizontal scaling is to be able to do the processing where the data lives and have that processing power scale with the data storage. |
| Comment by Hugo Lassiège [ 22/Jun/15 ] |
|
Hi, Regarding this issue, I guess that tools like Mongeez will not work anymore when eval will be removed, is it correct ? Do you know any alternative ? |
| Comment by Githook User [ 20/Apr/15 ] |
|
Author: {u'username': u'samantharitter', u'name': u'Samantha Ritter', u'email': u'samantha.ritter@10gen.com'}Message: (cherry picked from commit 465bb26c0fb0f4731f4dbb5e09e0a791177bbc64) |
| Comment by Githook User [ 30/Mar/15 ] |
|
Author: {u'username': u'samantharitter', u'name': u'Samantha Ritter', u'email': u'samantha.ritter@10gen.com'}Message: |
| Comment by Dave Workman [ 26/Mar/15 ] |
|
> If it turns out that the difference in performance is the extra latency (since both your code and eval passed in code run JS) - in which case you could achieve better performance running your node closer to the server. > The schema choice you link to is not one that will be able to give you an efficient way to fetch all children of a node (i.e. it requires multiple queries at the tradeoff of less maintenance on changes in hierarchy). Is there any plan to introduce an API that can match this performance? |
| Comment by Asya Kamsky [ 20/Mar/15 ] |
|
If it turns out that the difference in performance is the extra latency (since both your code and eval passed in code run JS) - in which case you could achieve better performance running your node closer to the server. The schema choice you link to is not one that will be able to give you an efficient way to fetch all children of a node (i.e. it requires multiple queries at the tradeoff of less maintenance on changes in hierarchy). |
| Comment by Dave Workman [ 20/Mar/15 ] |
|
Not recommended? This page begs to differ: http://docs.mongodb.org/manual/tutorial/model-tree-structures-with-parent-references/ When we first started building our app 2 years ago, we took a serious look at all of the different approaches (outlined on docs.mongodb.org) and decided to go with parent references, as it gave us the best performance for what our users do most often, move branches around. If we picked ancestors, child references or nested sets(all of the other options outlined on docs.mongodb.org) we'd have exactly the same performance issues, but more often! There is a drawback to each one of the hierarchy schema designs, we picked the one that, for our use case, performed the best for everyday tasks. I'm not opposed to removing eval, I'm opposed to not replacing it with something that gives equal performance, forgetting my use-case, as there are a thousand others. If I have some scheduled data reduction that takes on average 12 hours to complete on a client, but 2 with eval, why remove such a useful feature without giving us an equally performant (or better) option? |
| Comment by Asya Kamsky [ 20/Mar/15 ] |
|
That's not a recommended way to solve this use case - docs.mongodb.org have several examples of schema design for representing hierarchy/trees that allow you to get this sort of information in a single query. I would recommend a different schema to improve performance. |
| Comment by Dave Workman [ 19/Mar/15 ] |
|
I'm currently using eval for performance reasons. My database has a tree structure with parent references. I often need to find out all the children of a particular branch of the tree (using a recursive call). Doing these queries with the Node.js connector and concatenating their IDs together takes on average 6x times longer than my stored Javascript function that does the same thing. Without this my app doesn't scale very well. |
| Comment by rgpublic [ 18/Mar/15 ] |
|
Sigh. Another beloved Mongo feature goes away... Though I fully understand the reasoning behind it. I'd like to add here that at least |
| Comment by Matt Parlane [ 05/Mar/15 ] |
|
Personally, nothing – I'm not using eval at all, but I would like the option in the future. The way I see it, server-side logic, in the form of stored procedures, is a big deal in the enterprise world, which you've made no secret that you're going after. I think you guys should look seriously at solving the problems and making eval a first-class citizen rather than just removing it. You've got a tight, high-performance integration with (arguably) the fastest implementation of (arguably) the most popular programming language in the world, that sounds like something you should embrace and promote, not shun. |
| Comment by Daniel Pasette (Inactive) [ 04/Mar/15 ] |
|
Hi Matt,
I should point out that this does not mean a drop of support for all server side javascript. $where and map/reduce are still supported. Can you tell me what you're using eval to accomplish? |
| Comment by Matt Parlane [ 04/Mar/15 ] |
|
Anyone care to provide some rationale behind the deprecation? |
| Comment by Andy Schwerin [ 04/Mar/15 ] |
|
This ticket covers the db.eval() shell helper and the eval/$eval command. The latter form the basis for the implementation of the former. |
| Comment by Matt Parlane [ 03/Mar/15 ] |
|
Is the $eval command also deprecated? |