Details

    • Type: New Feature New Feature
    • Status: Open Open
    • Priority: Minor - P4 Minor - P4
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: planned but not scheduled
    • Component/s: Usability
    • Labels:
      None
    • Backport:
      No
    • # Replies:
      4
    • Last comment by Customer:
      true

      Description

      Create a way to allow multiple commands to be executed in a group (batch).

        Issue Links

          Activity

          Hide
          Caleb Jones
          added a comment - - edited

          I disagree with the "Minor" Priority. This would be a huge feature add similar in impact to the aggregation framework.

          Probably the #1 complaint or deterrent people cite with MongoDB is the lack of JOINs. Yes, that is part of the trade-offs with the solution. But when de-normalization is undesirable, being able to perform a batch of operations would avoid the bursting of one-by-one commands in loops and could overcome some of the reservations people have in adopting MongoDB.

          Another use case impacted by this:
          I'm currently working on processing a stream of data to convert into mongo insert/update/delete commands. I was disappointed that I have to fire off the commands one at at time instead of queuing them up into batches to reduce round-trips to the server.

          Hopefully this can get a bump in priority.

          Show
          Caleb Jones
          added a comment - - edited I disagree with the "Minor" Priority. This would be a huge feature add similar in impact to the aggregation framework. Probably the #1 complaint or deterrent people cite with MongoDB is the lack of JOINs. Yes, that is part of the trade-offs with the solution. But when de-normalization is undesirable, being able to perform a batch of operations would avoid the bursting of one-by-one commands in loops and could overcome some of the reservations people have in adopting MongoDB. Another use case impacted by this: I'm currently working on processing a stream of data to convert into mongo insert/update/delete commands. I was disappointed that I have to fire off the commands one at at time instead of queuing them up into batches to reduce round-trips to the server. Hopefully this can get a bump in priority.
          Hide
          Moshe
          added a comment - - edited

          @Caleb, I agree. Fail-safe systems are a nightmare to build with MongoDB. Here described a problem that is similar to mine: can't roll back a set of writes (insertions/updates) when one failed. I needed this in several applications, but I gave up and left them non-fail-safe. I imagine that's what most people do when they encounter this problem, but they don't when the worst possible failure is a disaster (like a payment that was not properly rewarded). Then they do a workaround.

          The workarounds are terrible. The most complex part of the application becomes the database, which was the simplest before.

          Why won't you admit that's a serious problem?

          Show
          Moshe
          added a comment - - edited @Caleb, I agree. Fail-safe systems are a nightmare to build with MongoDB. Here described a problem that is similar to mine: can't roll back a set of writes (insertions/updates) when one failed. I needed this in several applications, but I gave up and left them non-fail-safe. I imagine that's what most people do when they encounter this problem, but they don't when the worst possible failure is a disaster (like a payment that was not properly rewarded). Then they do a workaround. The workarounds are terrible. The most complex part of the application becomes the database, which was the simplest before. Why won't you admit that's a serious problem?
          Hide
          John Wood
          added a comment -

          Allowing batch queries would also resolve SERVER-432 (server side expansion of DbRefs), and make it much easier to create a query preprocessor that would efficiently allow for more complex queries and eliminate some limitations (eg. SERVER-5937 (splitting documents to overcome size limitations)).

          I can imagine it would be hard because you would need to support multiple cursors per connection, but even if initially you just limited it to returning all the data in one BSON it would still reap a lot of benefits.

          Show
          John Wood
          added a comment - Allowing batch queries would also resolve SERVER-432 (server side expansion of DbRefs), and make it much easier to create a query preprocessor that would efficiently allow for more complex queries and eliminate some limitations (eg. SERVER-5937 (splitting documents to overcome size limitations)). I can imagine it would be hard because you would need to support multiple cursors per connection, but even if initially you just limited it to returning all the data in one BSON it would still reap a lot of benefits.
          Hide
          Kevin J. Rice
          added a comment -

          I would like to vote for a simple version of this: BATCH UPDATES. That is, I'm issuing thousands of update statements a second. I would like to send them as a list of updates rather than do them one by one, with the latency involved in that.

          Currently there is support for batch inserts. This feature may tie-in with that functionality.

          Playing devil's advocate to my own argument, I can see a possible complication. I have a sharded, replicated collection. So, let's say I send in a list of updates (each of a single separate document indexed by _id). The router (mongos) would have to split the updates to separate lists to go to their respective shards.

          Other than that, it seems a straightforward performance increaser.

          Re: failures, handle it the same as batch inserts, insert all possible records/documents and return data on which ones failed, or whatever is the easiest functionality to implement and I'll cope with the downsides.

          Show
          Kevin J. Rice
          added a comment - I would like to vote for a simple version of this: BATCH UPDATES. That is, I'm issuing thousands of update statements a second. I would like to send them as a list of updates rather than do them one by one, with the latency involved in that. Currently there is support for batch inserts. This feature may tie-in with that functionality. Playing devil's advocate to my own argument, I can see a possible complication. I have a sharded, replicated collection. So, let's say I send in a list of updates (each of a single separate document indexed by _id). The router (mongos) would have to split the updates to separate lists to go to their respective shards. Other than that, it seems a straightforward performance increaser. Re: failures, handle it the same as batch inserts, insert all possible records/documents and return data on which ones failed, or whatever is the easiest functionality to implement and I'll cope with the downsides.

            People

            • Votes:
              48 Vote for this issue
              Watchers:
              33 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since reply:
                1 year, 7 weeks, 3 days ago
                Date of 1st Reply: