Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-8228

Map Reduce Data Will Sometimes Not Get Inserted

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Incomplete
    • Affects Version/s: 2.2.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide
      • Run a map/reduce/finalize on a sharded or unsharded, but replicated data set.
      • Delete some documents from output collection.
      • Rerun map reduces.

      This error, as far as we know, wasn't seen until version 2.2 of mongoDB. We were running these map reduces for months without major issue on 2.0, but we've never used anything besides 2.0.4 and 2.2.2

      Show
      Run a map/reduce/finalize on a sharded or unsharded, but replicated data set. Delete some documents from output collection. Rerun map reduces. This error, as far as we know, wasn't seen until version 2.2 of mongoDB. We were running these map reduces for months without major issue on 2.0, but we've never used anything besides 2.0.4 and 2.2.2

      Description

      We are able to run a series of map reduces that write data into an output collection without issue. However, we noticed that our results were not consistent so in testing and trying to find the issue, I have tried each of the following scenarios:

      Sharded database/input/output collections:

      • Run map reduces
      • See bad results
      • Drop and recreate output collection
      • Run all map reduces

      AND

      Unsharded database:

      • Run map reduces
      • See bad results
      • Delete documents for one of the map reduces
      • Rerun the one map reduce

      The results were the same no matter what we did:
      The map reduces after documents were deleted did not insert any documents, but ran successfully. We have checked the queries that the map/reduces run on and they return thousands of documents. Some map reduces take up to 2 minutes which is normal on our weak cluster and yet put nothing in the output collection.

      We started with everything sharded, moved to everything unsharded, saw the error within 2 days (didn't happen at first), then we built a MASSIVE (128 GB RAM, 16 core) unsharded, unreplicated mongodb server and have not seen the error yet.

        Attachments

          Activity

            People

            Assignee:
            Unassigned Unassigned
            Reporter:
            wwatson William Watson
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: