[SERVER-37653] data inconsistent Created: 17/Oct/18 Updated: 07/Apr/23 Resolved: 23/Oct/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | 3.2.20 |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Critical - P2 |
| Reporter: | chensi | Assignee: | Kelsey Schubert |
| Resolution: | Cannot Reproduce | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Steps To Reproduce: | know no way to reproduce |
| Participants: |
| Description |
|
data can be found by some query on primary: shard01:PRIMARY> db.XXXXXXXXXX.find({"task_ids" : [ NumberLong("6566381934111359492") ]}) { "_id" : ObjectId("5b20be13e1803d1f2529dbe0"), "count" : 1, "task_ids" : [ NumberLong("6566381934111359492") ],....}
but cannot found by _id: shard01:PRIMARY> db.XXXXXXXXXX.find({"_id":ObjectId("5b20be13e1803d1f2529dbe0")}).count() the other secondaries are ok.
|
| Comments |
| Comment by Kelsey Schubert [ 23/Oct/18 ] |
|
Thanks for the additional information. Unfortunately, in cases like this where the issue could have occurred some time ago without any hints to reproduce the issue, it's very challenging to investigate. As MongoDB 3.2 is no longer supported, I'm going to resolve this ticket. Since your secondaries are consistent, I would recommend clearing the $dbpath of the affected node and performing an initial sync. From the information provided, it appears that the _id index on this node is likely corrupt. While the index could be rebuilt, an initial sync would ensure that that the data is consistent across the replica set. Kind regards, |
| Comment by chensi [ 22/Oct/18 ] |
|
we don't use the momgos, just query against mongod. the following are explain results: A. data not found {{ "queryPlanner" : { "plannerVersion" : 1, "namespace" : "XXX", "indexFilterSet" : false, "parsedQuery" : { "_id" : { "$eq" : ObjectId("5b20be13e1803d1f2529dbe0") }}, "winningPlan" : { "stage" : "IDHACK" }, "rejectedPlans" : [ ] }, "executionStats" : { "executionSuccess" : true, "nReturned" : 0, "executionTimeMillis" : 0, "totalKeysExamined" : 0, "totalDocsExamined" : 0, "executionStages" : { "stage" : "IDHACK", "nReturned" : 0, "executionTimeMillisEstimate" : 0, "works" : 1, "advanced" : 0, "needTime" : 0, "needYield" : 0, "saveState" : 0, "restoreState" : 0, "isEOF" : 1, "invalidates" : 0, "keysExamined" : 0, "docsExamined" : 0 }},
B. data founded {{ "queryPlanner" : { "plannerVersion" : 1, "namespace" : "ops_aggdata.tcs_project_blindverify_not_same_hourly", "indexFilterSet" : false, "parsedQuery" : { "task_ids" : { "$eq" : [ NumberLong("6566381934111359492") ] }}, "winningPlan" : { "stage" : "COLLSCAN", "filter" : { "task_ids" : { "$eq" : [ NumberLong("6566381934111359492") ] }}, "direction" : "forward" }, "rejectedPlans" : [ ] }, "executionStats" : { "executionSuccess" : true, "nReturned" : 1, "executionTimeMillis" : 4031, "totalKeysExamined" : 0, "totalDocsExamined" : 1156773, "executionStages" : { "stage" : "COLLSCAN", "filter" : { "task_ids" : { "$eq" : [ NumberLong("6566381934111359492") ] }}, "nReturned" : 1, "executionTimeMillisEstimate" : 3911, "works" : 1156775, "advanced" : 1, "needTime" : 1156773, "needYield" : 0, "saveState" : 9037, "restoreState" : 9037, "isEOF" : 1, "invalidates" : 0, "direction" : "forward", "docsExamined" : 1156773 } },
BTW, data on the secondaries are consistent, we are moving to the 3.6 version |
| Comment by Kelsey Schubert [ 19/Oct/18 ] |
|
Thanks for reporting this issue. Please note that MongoDB 3.2 is no longer supported. I would recommend upgrading to a more recent version of MongoDB. Would you please clarify whether these queries reliably produce inconsistent results? If so, would you please provide the Thank you, |