[SERVER-8228] Map Reduce Data Will Sometimes Not Get Inserted Created: 18/Jan/13  Updated: 10/Dec/14  Resolved: 28/Oct/13

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 2.2.2
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: William Watson Assignee: Unassigned
Resolution: Incomplete Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Steps To Reproduce:
  • Run a map/reduce/finalize on a sharded or unsharded, but replicated data set.
  • Delete some documents from output collection.
  • Rerun map reduces.

This error, as far as we know, wasn't seen until version 2.2 of mongoDB. We were running these map reduces for months without major issue on 2.0, but we've never used anything besides 2.0.4 and 2.2.2

Participants:

 Description   

We are able to run a series of map reduces that write data into an output collection without issue. However, we noticed that our results were not consistent so in testing and trying to find the issue, I have tried each of the following scenarios:

Sharded database/input/output collections:

  • Run map reduces
  • See bad results
  • Drop and recreate output collection
  • Run all map reduces

AND

Unsharded database:

  • Run map reduces
  • See bad results
  • Delete documents for one of the map reduces
  • Rerun the one map reduce

The results were the same no matter what we did:
The map reduces after documents were deleted did not insert any documents, but ran successfully. We have checked the queries that the map/reduces run on and they return thousands of documents. Some map reduces take up to 2 minutes which is normal on our weak cluster and yet put nothing in the output collection.

We started with everything sharded, moved to everything unsharded, saw the error within 2 days (didn't happen at first), then we built a MASSIVE (128 GB RAM, 16 core) unsharded, unreplicated mongodb server and have not seen the error yet.



 Comments   
Comment by Stennie Steneker (Inactive) [ 28/Oct/13 ]

Hi William,

I'm closing this issue due to inactivity.

If you are still seeing this issue (particularly with a newer version of MongoDB, such as 2.4.x) please feel free to open a new issue or comment on this one with the relevant details.

Thanks,
Stephen

Comment by Barrie Segal [ 18/Jan/13 ]

William, you can open a ticket in the Community Private project and attach the logs there. Only you, as the reporter of the ticket, and the dev team here will have access to it.

Comment by William Watson [ 18/Jan/13 ]

As long as you agree to not post the code to any ticket or public forum, and do dont disclose the code outside of 10gen, we will gladly email it.

Comment by William Watson [ 18/Jan/13 ]

We can email the code to a 10gen employee, but we will not post it here.

Comment by Eliot Horowitz (Inactive) [ 18/Jan/13 ]

Can you send the map/reduce jobs you are running?

Generated at Thu Feb 08 03:16:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.