Uploaded image for project: 'Java Driver'
  1. Java Driver
  2. JAVA-5862

Reconsider the toCollection methods and aggregation behavior when $out, $merge is used

    • Type: Icon: Task Task
    • Resolution: Unresolved
    • Priority: Icon: Unknown Unknown
    • 6.0.0
    • Affects Version/s: None
    • Component/s: API
    • None
    • None
    • Java Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      com.mongodb.client.AggregateIterable.iterator (and similar API program elements, see https://github.com/mongodb/mongo-java-driver/pull/1677 to observe all of them) implicitly finds all documents if the pipeline ends with the $out/$merge stage. This is neither how users may expect the method to behave based on its normal behavior, nor is this helpful: in the rare situations when this is needed, a user may trivially execute the find himself, but in situations when this is not needed, it may cause an unexpected performance issue:

      Tests run fine and it’s only on the production system that you just see abnormal network and CPU traffic.

      Our production servers collapsed last year because of that. We deploy a change to use the aggregation framework to recompute some denormalisations (instead of a manual client side aggregation) We failed to find the cause of the problem and reverted that. And the same story happened very recently in a different context (link), fortunately not on prod.

            Assignee:
            Unassigned Unassigned
            Reporter:
            valentin.kovalenko@mongodb.com Valentin Kavalenka
            None
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: