[JAVA-4222] Aggregation pipeline stage $out cannot be used with Atlas specific description Created: 06/Jul/21  Updated: 27/Oct/23  Resolved: 06/Jul/21

Status: Closed
Project: Java Driver
Component/s: API
Affects Version/s: 4.2.3
Fix Version/s: None

Type: Task Priority: Minor - P4
Reporter: Alperen Pulur Assignee: Jeffrey Yemin
Resolution: Works as Designed Votes: 0
Labels: external-user
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Documentation Changes: Not Needed

 Description   

Greetings,

 

According to the mongodb doc about $out stage (https://docs.mongodb.com/datalake/reference/pipeline/out/#-out) it is possible to define out stage composed with a document which contains fields different than db and coll.

 

However, in latest driver (4.2.3), it only allows to use $out stage with db and coll fields (if it is used as document). Here is the places I spotted for the error is thrown:

  1. https://github.com/mongodb/mongo-java-driver/blob/0ce5c4f7977aa7a0a8a2b10b0f22477a8231dce5/driver-reactive-streams/src/main/com/mongodb/reactivestreams/client/internal/AggregatePublisherImpl.java#L209
  2. https://github.com/mongodb/mongo-java-driver/blob/0ce5c4f7977aa7a0a8a2b10b0f22477a8231dce5/driver-sync/src/main/com/mongodb/client/internal/AggregateIterableImpl.java#L232

 

Please consider to support Atlas specific $out stage as given in the mongodb doc.

Best,



 Comments   
Comment by Jeffrey Yemin [ 06/Jul/21 ]

You are right to assume that. We don't intend to support streaming from the output collection outside of the basic support that exists.

Comment by Alperen Pulur [ 06/Jul/21 ]

I see, I kinda rely on cursor at the moment to show some of the documents those are being copied into new collection. But this seems to be a very edge case maybe I need to add that extra find at the end of the operation.

So in that regard, I take your response as this will not be added to the driver at any point to cover the case. Am I right to assume that? Or you may prefer to take into account that case in the future?

Comment by Jeffrey Yemin [ 06/Jul/21 ]

Yes toCollection() is specific for $out and $merge. If you need to iterate over that collection subsequent to the aggregation finishing, you can do so with a separate call to MongoCollection#find

Comment by Alperen Pulur [ 06/Jul/21 ]

Hello jeff.yemin,

Indeed, I was using AggregateIterable#iterator. Is use of toCollection() specific for $out and $merge? I can give a try if it works or not, but would like to understand if this is the expected way of implementing such pipelines.

 

Comment by Jeffrey Yemin [ 06/Jul/21 ]

Hi alperen.pulur@3tsoftwarelabs.com,

I suspect you're using AggregateIterable#cursor or AggregateIterable#iterator to initiate the aggregation. Have you tried using the AggregateIterable#toCollection method? 

Generated at Thu Feb 08 09:01:32 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.