Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor - P4
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: planned but not scheduled
    • Component/s: MapReduce
    • Labels:
      None

      Description

      instead of just 1

      at least as an option

        Issue Links

          Activity

          Hide
          penfold Pen Fold added a comment -

          Is it possible to schedule this improvement for an upcoming release?

          This should give me a serious performance boost. Especially in a sharded setup.

          Show
          penfold Pen Fold added a comment - Is it possible to schedule this improvement for an upcoming release? This should give me a serious performance boost. Especially in a sharded setup.
          Hide
          ukd1 Russell Smith added a comment -

          This is important to me, as it will significantly improve performance on my machine (I'm cpu bound on a single core, on a 8 core server).

          Show
          ukd1 Russell Smith added a comment - This is important to me, as it will significantly improve performance on my machine (I'm cpu bound on a single core, on a 8 core server).
          Hide
          plasma Andrew Armstrong added a comment -

          Would be good to have I think for sharded setups; given I think group() does not work when sharded for example either?

          Show
          plasma Andrew Armstrong added a comment - Would be good to have I think for sharded setups; given I think group() does not work when sharded for example either?
          Hide
          chengas123 Ben McCann added a comment -

          I'm surprised this is listed as "planned but not scheduled". Won't this be fixed as part of moving to v8 in 2.4?

          Show
          chengas123 Ben McCann added a comment - I'm surprised this is listed as "planned but not scheduled". Won't this be fixed as part of moving to v8 in 2.4?
          Hide
          tad Tad Marshall added a comment -

          The move to V8 does enable additional multi-threading options, but MapReduce itself needs some restructuring to take advantage of this properly. This won't be fully realized for 2.4, but it should probably be placed on the schedule and not left as "planned but not scheduled".

          Show
          tad Tad Marshall added a comment - The move to V8 does enable additional multi-threading options, but MapReduce itself needs some restructuring to take advantage of this properly. This won't be fully realized for 2.4, but it should probably be placed on the schedule and not left as "planned but not scheduled".
          Hide
          benjamin.becker Ben Becker added a comment -

          To clarify a bit further, v2.4 will allow multiple MapReduce commands to execute JavaScript concurrently. A single MapReduce command will only use one core.

          Show
          benjamin.becker Ben Becker added a comment - To clarify a bit further, v2.4 will allow multiple MapReduce commands to execute JavaScript concurrently. A single MapReduce command will only use one core.
          Hide
          antoineguiral Antoine Guiral added a comment -

          Hum, so instead of 1 M/R at a time we could do "numCPU" M/R? I would prefer that my M/R become faster. Why they will still use only one CPU? When you perform a big M/R there is in fact a lot of M/R, they should be parallelized no? Or maybe I miss something..

          Show
          antoineguiral Antoine Guiral added a comment - Hum, so instead of 1 M/R at a time we could do "numCPU" M/R? I would prefer that my M/R become faster. Why they will still use only one CPU? When you perform a big M/R there is in fact a lot of M/R, they should be parallelized no? Or maybe I miss something..
          Hide
          chengas123 Ben McCann added a comment -

          I think there is agreement that M/R should use multiple cores. It's just a question of fixing it to do that now that we have v8.

          Show
          chengas123 Ben McCann added a comment - I think there is agreement that M/R should use multiple cores. It's just a question of fixing it to do that now that we have v8.
          Hide
          benjamin.becker Ben Becker added a comment -

          Hi Antoine,

          The goal of this ticket is to allow a single MapReduce job to use multiple cores, which is not yet implemented. This requires several new features; e.g. a storage system for intermediate results that doesn't acquire a db-level write lock, and global CPU/memory resource management.

          Show
          benjamin.becker Ben Becker added a comment - Hi Antoine, The goal of this ticket is to allow a single MapReduce job to use multiple cores, which is not yet implemented. This requires several new features; e.g. a storage system for intermediate results that doesn't acquire a db-level write lock, and global CPU/memory resource management.

            People

            • Votes:
              159 Vote for this issue
              Watchers:
              91 Start watching this issue

              Dates

              • Created:
                Updated:
                Days since reply:
                2 years, 13 weeks, 2 days ago
                Date of 1st Reply: