[SERVER-76447] BSON validation should always attempt to find an _id Created: 24/Apr/23  Updated: 09/Jun/23

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: Louis Williams Assignee: Geert Bosch
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Storage Execution
Sprint: Execution Team 2023-05-15, Execution Team 2023-05-29, Execution Team 2023-06-12, Execution NAMR Team 2023-06-26
Participants:

 Description   

When BSON validation encounters an error in a document and the _id is not the first field in the document, we currently report "in object with unknown _id". We should try to find the _id when reporting the error.



 Comments   
Comment by Louis Williams [ 31/May/23 ]

I discussed this with Geert offline. In summary, it's risky to try to continue parsing BSON that we know is corrupt. This failure mode also requires us to have a document where the _id is not positionally first. It is not possible to insert a document like this since at least MongoDB 3.6, so there are increasingly fewer cases where this is a problem.

Comment by Louis Williams [ 26/Apr/23 ]

I was able to insert a document in an instrumented build where the _id was not first, and this was the result:

// With the server started normally
test> db.test.insert({ a: { b: { c: { d: { e: { f: { a: { b: { c: { d: { e: { f: { a: { b: { c: { d: { e: { f: {a: { b: { c: { d: { e: { f: {} } } } } }} } } } } } } } } } } } } } } } } }, _id: 1})
// After restarting with --setParamter maxBSONDepth=21
test> db.test.validate()
{
  ns: 'test.test',
  uuid: new UUID("ca26dd7a-c38c-4f17-a3e2-590ea5d8398d"),
  nInvalidDocuments: 1,
  nNonCompliantDocuments: 0,
  nrecords: 2,
  nIndexes: 1,
  keysPerIndex: { _id_: 0 },
  indexDetails: { _id_: { valid: true } },
  valid: false,
  repaired: false,
  warnings: [],
  errors: [ 'Detected one or more invalid documents. See logs.' ],
  extraIndexEntries: [],
  missingIndexEntries: [],
  corruptRecords: [ Long("3") ],
  advice: 'A corrupt namespace has been detected. See http://dochub.mongodb.org/core/data-recovery for recovery steps.',
  ok: 1
}

Logs:

{"t":{"$date":"2023-04-26T17:01:20.440+00:00"},"s":"I",  "c":"STORAGE",  "id":4835001, "ctx":"conn2","msg":"Document corruption details - Document validation failed with error","attr":{"recordId":"3","error":{"code
":15,"codeName":"Overflow","errmsg":"BSONObj exceeds maximum nested object depth in element with field name 'a.b.c.d.e.f.a.b.c.d.e.f.a.b.c.d.e.f.a.b.c.d' in object with unknown _id"}}}

This was what I changed to insert the document:

diff --git a/src/mongo/db/ops/insert.cpp b/src/mongo/db/ops/insert.cpp
index 814ee91d196..04d3853c417 100644
--- a/src/mongo/db/ops/insert.cpp
+++ b/src/mongo/db/ops/insert.cpp
@@ -143,6 +143,7 @@ StatusWith<BSONObj> fixDocumentForInsert(OperationContext* opCtx,
         }
     }
 
+    validationDisabled = true;
     if (validationDisabled || (firstElementIsId && !hasTimestampToFix))
         return StatusWith<BSONObj>(BSONObj());

So BSON validation does not report the _id if it is not first.

Comment by Louis Williams [ 26/Apr/23 ]

Note I haven't confirmed whether this is actually a problem or not. It seems likely not

Generated at Thu Feb 08 06:32:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.