Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8624

Remove read-only constraint on Reduce function values

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • 2.3.0, 2.3.1, 2.3.2, 2.4.0-rc0
    • 2.4.0-rc2
    • MapReduce
    • V8

    Description

      In the V8 engine switch, values passed to the reduce function have been changed to be read-only. There are several situations were read only values impose problems in the reduce step. For example trying to merge lists of objects with some common properties where you increment a count in an existing object when you see a duplicate in another list. Since that object is read only, you cant increment the count inside it to persist in future reduce steps. Or you want to actually merge two objects where the result is a composite of the two objects, you can't because the initial object is read only. So then you have to try and make a copy of the initial object to then write your changes into and return, which adds overhead and lots of duplicate code in each reduce function. Also as this is JavaScript, implementing a deep copy on a generic object is a hard thing to get right (See http://stackoverflow.com/questions/728360/copying-an-object-in-javascript for a detailed discussion).

      I understand this was done as a performance optimization, but maybe there should be a reconsideration of the impact on the M/R engine flexibility and on all the existing code written against the <2.4 engine behaviour?

      Attachments

        Issue Links

          Activity

            People

              benjamin.becker Ben Becker
              ibeaver Ian Beaver
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: