[SERVER-432] server side expansion of DBRefs Created: 20/Nov/09  Updated: 17/Jul/15  Resolved: 24/Jun/15

Status: Closed
Project: Core Server
Component/s: Usability
Affects Version/s: None
Fix Version/s: None

Type: New Feature Priority: Minor - P4
Reporter: Eliot Horowitz (Inactive) Assignee: Unassigned
Resolution: Duplicate Votes: 64
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-19095 $lookup Closed
is duplicated by SERVER-19476 Referenced Values Closed
Related
related to SERVER-745 Embedded Document Expansion (Pseudo-J... Closed
related to SERVER-5937 Remove document size limits by transp... Open
related to SERVER-1632 improve find with using DBRef to supp... Closed
Participants:

 Description   

Option to expand dbrefs (both styles) on the server.
How it would work with sharding is undetermined at this time.



 Comments   
Comment by Ian Whalen (Inactive) [ 24/Jun/15 ]

Hi Eliot, thanks a lot for filing this feature request and my apologies for the time since it was last updated.

I'm going to close this as a Duplicate and link it to our upcoming $lookup feature - after careful consideration we’ve decided to provide users with the desired functionality via our aggregation pipeline. Please follow along in SERVER-19095 for further details of the $lookup implementation.

Comment by Ronald van der Feer [ 11/Oct/13 ]

@bishwa shrestha: +1, you have exactly described my situation

Seriously though;
I honestly don't understand how this is not receiving more votes, or why it's simply classified as a 'usability' issue.
To me this seems like a major feature for the product, or even a new class of product, and it has been a showstopper for using Mongo several times.

After almost 4 years it seems this is just not going to get any attention, perhaps someone on the dev team can comment so we can understand?

Comment by bishwa shrestha [ 11/Oct/13 ]

@Hatch: +1, you have exactly described my situation

Comment by Hatch [ 10/Oct/13 ]

In looking at MongoDB, especially reading the company tutorials and examples of "real-world" implementation strategies, they have examples of many types of implementations from blogs to inventory management where, often with quite a lot of work, you can structure your data in such a way that your application needs only to make a single request for a single document to provide a response to a user request.

On my attempts to replicate these examples however, time and time again I find that while the examples work as is, the moment you try to add a tiny piece of additional functionality (simple things commonly seen in similar systems in the wild) you start hitting walls that require radical redesign of your data models, if you can even figure out a way to implement the feature. You end up with a flexible data model, but a very rigid feature set.

In every system I've tried so far I've started off with pure MongoDB, eventually added a secondary database, usually a traditional RDBMS, to handle some complex relational/transactional data which is then replicated to MongoDB. Little by little MongoDB becomes effectively a cache layer handling only "lose-able", derived data with the important canonical data stored elsewhere. Which is fine I guess, and MongoDB's schema-free design makes this relatively easy to do without having to worry about keeping two schema sets synchronized.

Still, my biggest grip right now is the lack of join-like operations in MongoDB. It seems like on one hand MongoDB says "you don't need joins, just embed documents", but then turns around and talks about how cheap queries are and suggests making multiple calls to perform your own joins. The second approach is usually required...eventually. I am beginning to believe that nearly every non-trivial application does in fact need joins for its primary data set, even if it doesn't appear so at first glance. Unless the foreign side of the join is either completely static or owned solely by the primary document, you end up requiring periodic system-wide updates to lots and lots of documents.

This feature would be a huge improvement for MongoDB, and if it performed well enough it would greatly expand the role MongoDB could play in serious applications.

Comment by Vincent Noel [ 16/Jul/13 ]

As a user of Mongodb for big projects, This feature seems very important to me.

It's an improvement in usability, but it's the kind of improvements that change an interesting tool into a must have tool, a fairly used tool into a standard.

Mongodb use will be simpler, faster and more agile. Not only the queries since even the aggregation system will benefit from the feature. At a point in any project even a key/value oriented project has the need of such feature, and it's a real pain to do multiple queries or do complex and costly agregation just to retrieve obvious informations.

I'd like to be able to find the logs of users created since last week without going through the whole database in javascript or to make complex and "hack like" queries sequence to obtain such a simple thing. The other option of duplicating every information you might need is not feasible, you can't possibly know in advance all the information you will need (either for application logic or trace analysis), and adding them once you need them is just a pain and will result in a useless increase of the space taken by the data and what if the data duplicated change?
db.logs.find({"_user.createdAt": {$gt: myDate}}, {'populate': {_user: 1}});
Would be great, formidable, maybe even wonderful.

Because I've got a game, and I banned users for cheating, and i need to go through the logs to find a way to detect the cheaters. I'd like to get all logs of users that are banned right now.
db.logs.find({"_user.banned": {$exists: 1}}, {'populate': {_user: 1}});

Even adding deep population:
db.logs.find({"_user.banned": {$exists: 1}, "_user.guild.name": "the greatest guild of all time"}, {'populate': {_user: 1, "_user._guild": 1}});

...etc
I'm not into the development of mongodb, so maybe i'm wrong, but it does not sounds too complicated for me and the benefit sounds really great.

Comment by Colin Mollenhour [ 29/Oct/10 ]

Syntax proposal:

db.posts.find({_id:1}, {'$refs': {author: 1}}) // expand using author field as a dbref
db.posts.find({_id:1}, {'$refs': {author: 'users'}}) // expand using author field as _id for users collection

Comment by Eliot Horowitz (Inactive) [ 15/Mar/10 ]

not as designed - since you don't know which collection an _id belongs

Comment by Chris [ 15/Mar/10 ]

Could this also support expanding just an ObjectId as well?

Generated at Thu Feb 08 02:54:05 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.