[SERVER-19958] aggregate([{$out}]) produces a different collection than an insert() Created: 14/Aug/15  Updated: 20/Aug/15  Resolved: 17/Aug/15

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Lucas Geiger Assignee: Unassigned
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
Operating System: ALL
Steps To Reproduce:

Here is an app that can reproduce the error: https://github.com/keyscores/meteor-aggregation-issue

just clone,
> cd meteor-aggregation-issue
>meteor

Participants:

 Description   

There is a persistent issue using Mongo aggregate with reactive frameworks like Meteor. The $out operator invalidates reactive updates, which are otherwise possible with insert().

What could be different about the collections created aggregate([$out]) than insert()?

See here:
http://stackoverflow.com/questions/31996392/does-mongos-aggregate-with-out-create-invalid-collections

and here:
https://github.com/meteor/meteor/issues/4947

Seems like Mongo's aggregate() when using $out, somehow produces a different collection than an insert() of the same data. I'm only guessing this because I'm getting very different behaviors.

I'm testing this in Meteor, and collections created with aggregate([$out]) are not observable by the app. I'm not sure why this would be the case, since the collection looks fine in the Mongo console.



 Comments   
Comment by Charlie Swanson [ 17/Aug/15 ]

I will comment here just for the record, I do not recommend using that workaround I mentioned as a fix. I was just curious if that was actually what was going on.

Comment by Ramon Fernandez Marina [ 17/Aug/15 ]

lpg, if I understand the problem description correctly I think Charlie's answer above should explain the behavior you're seeing.

Please note that the SERVER project is for reporting bugs or feature suggestions for the MongoDB server. For MongoDB-related support discussion please post on the mongodb-user group or Stack Overflow with the mongodb tag, where your question will reach a larger audience. A question like this involving more discussion would be best posted on the mongodb-user group. See also our Technical Support page for additional support resources.

Regards,
Ramón.

Comment by Charlie Swanson [ 14/Aug/15 ]

An aggregate command with an $out stage will create a temporary collection while the aggregation pipeline is running, and then, once the pipeline is done, it will rename the temporary collection to the name specified in the $out stage. e.g. if you ran

db.foo.aggregate([{$out: 'bar'}]);

It would create a temporary bar.tmp.agg_out collection, and insert all the results from the pipeline into that collection. When the pipeline completes, it will rename bar.tmp.agg_out to bar, keeping all indices and options of the existing bar collection (if any).

This should explain the behavior you are seeing. If I understand your code correctly, Meteor is providing a framework to let you listen to events, such as a document being inserted into a collection, and it is not calling the callback when you use an aggregate() command? As a temporary workaround, could you verify that listening to those events on the ".tmp.agg_out" collection would work?

Comment by Lucas Geiger [ 14/Aug/15 ]

And here is a link to test it quickly:
http://meteor-aggregation-issue.meteor.com/

Comment by Lucas Geiger [ 14/Aug/15 ]

Does $out drop a collection and recreate it? Possibly remove({}) and insert() would be equally effective and not produce errors on reactive frameworks?

Generated at Thu Feb 08 03:52:41 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.