[SERVER-19559] Document growth of "key too large" document makes it disappear from the index Created: 23/Jul/15  Updated: 12/Aug/15  Resolved: 28/Jul/15

Status: Closed
Project: Core Server
Component/s: Index Maintenance
Affects Version/s: 2.6.9
Fix Version/s: 2.6.11

Type: Bug Priority: Major - P3
Reporter: Ger Hartnett Assignee: Mathias Stearn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Steps To Reproduce:

The document

> db.users.find()
{ "_id" : ObjectId("54876097ebb87b0aeb001e66"), "email" : "someone@somewhere.com", "appId" : 101430, "userId" : "5485adffab2a47502832c735", "customData" : { "subField" : "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" } }

The indexes:

> db.users.getIndexes()
[
	{
		"v" : 1,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "test.users"
	},
	{
		"v" : 1,
		"key" : {
			"appId" : 1,
			"customData.subField" : 1
		},
		"name" : "appId_1_customData.subField_1",
		"ns" : "test.users"
	},
	{
		"v" : 1,
		"unique" : true,
		"key" : {
			"appId" : 1,
			"userId" : 1,
			"email" : 1
		},
		"name" : "appId_1_userId_1_email_1",
		"ns" : "test.users",
		"background" : true,
		"dropDups" : true
	}
]

Finding by _id works

> db.users.find({}, {_id:1}).showDiskLoc()
{ "_id" : ObjectId("54876097ebb87b0aeb001e66"), "$diskLoc" : { "file" : 0, "offset" : 16816 } }

Make the document grow, the update seems to fail:

> var s2k = "a"; for (i=0;i<11;i++) s2k += s2k; print(s2k.length);
2048
> db.users.update({email: "someone@somewhere.com", appId:101430}, { $set: {"somethingNew" : s2k} });
WriteResult({
	"nMatched" : 0,
	"nUpserted" : 0,
	"nModified" : 0,
	"writeError" : {
		"code" : 17280,
		"errmsg" : "Btree::insert: key too large to index, failing test.users.$appId_1_customData.subField_1 2071 { : 101430.0, : \"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa...\" }"
	}
})

Finding by index finds nothing:

> db.users.find({ "_id" : ObjectId("54876097ebb87b0aeb001e66") }, {_id:1})
>  

The document is still there:

> db.users.find({}, {_id:1}).showDiskLoc()
{ "_id" : ObjectId("54876097ebb87b0aeb001e66"), "$diskLoc" : { "file" : 0, "offset" : 16816 } }

You can now insert another document with the same _id:

> db.users.insert(newDoc);
WriteResult({ "nInserted" : 1 })
> db.users.find({}, {_id:1}).showDiskLoc()
{ "_id" : ObjectId("54876097ebb87b0aeb001e66"), "$diskLoc" : { "file" : 0, "offset" : 16816 } }
{ "_id" : ObjectId("54876097ebb87b0aeb001e66"), "$diskLoc" : { "file" : 0, "offset" : 16560 } }

Sprint: Quint Iteration 7
Participants:

 Description   

This could happen for example of you upgrade from 2.4 with a document that has a "key too large" and didn't run or fix db.upgradeCheckAllDBs().

Changing the document to make it grow causes the document to get cleared from the indexes. The document still exists but can't be found by _id.

This can have strange effects downstream and on secondaries. For example another document with the same _id can be created.

It also seems to result in the following on secondaries after a number of other operations:

[repl writer worker 3] ERROR: writer worker caught exception:  :: caused by :: 10287 btree: key+recloc already in index on...



 Comments   
Comment by Githook User [ 28/Jul/15 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'redbeard0531@gmail.com'}

Message: SERVER-19559 fix test to use GLE rather than assert.writeOK/Error
Branch: v2.6
https://github.com/mongodb/mongo/commit/1800d67e551de19d27cc7a477793c3ed32be9285

Comment by Githook User [ 27/Jul/15 ]

Author:

{u'username': u'RedBeard0531', u'name': u'Mathias Stearn', u'email': u'redbeard0531@gmail.com'}

Message: SERVER-19559 Ignore KeyTooLarge errors when rolling back failed updates
Branch: v2.6
https://github.com/mongodb/mongo/commit/0733ab9c06cabbd9d5d07f68d94714bfb0b2e913

Generated at Thu Feb 08 03:51:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.