Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-2398

for inline mapreduce, all emitted objects are kept in RAM before the 1st reduce, potential high memory usage

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 1.7.5
    • None
    • None
    • None
    • ALL

    Description

      during map phase, checkSize() is called to do reduceInMemory and potentially dumpToInc.
      But if inline mode, checkSize doesnt do anything.
      All objects will be emitted before 1st attempt to reduce.
      Instead reduceInMemory should be called if map is over a certain size, or if there is potential for reduce.

      Attachments

        Activity

          People

            antoine Antoine Girbal
            antoine Antoine Girbal
            Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: