Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8624

Remove read-only constraint on Reduce function values

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 2.4.0-rc2
    • Affects Version/s: 2.3.0, 2.3.1, 2.3.2, 2.4.0-rc0
    • Component/s: MapReduce
    • Labels:
    • Environment:

      In the V8 engine switch, values passed to the reduce function have been changed to be read-only. There are several situations were read only values impose problems in the reduce step. For example trying to merge lists of objects with some common properties where you increment a count in an existing object when you see a duplicate in another list. Since that object is read only, you cant increment the count inside it to persist in future reduce steps. Or you want to actually merge two objects where the result is a composite of the two objects, you can't because the initial object is read only. So then you have to try and make a copy of the initial object to then write your changes into and return, which adds overhead and lots of duplicate code in each reduce function. Also as this is JavaScript, implementing a deep copy on a generic object is a hard thing to get right (See http://stackoverflow.com/questions/728360/copying-an-object-in-javascript for a detailed discussion).

      I understand this was done as a performance optimization, but maybe there should be a reconsideration of the impact on the M/R engine flexibility and on all the existing code written against the <2.4 engine behaviour?

            benjamin.becker Ben Becker
            ibeaver Ian Beaver
            0 Vote for this issue
            6 Start watching this issue