Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-41213

Unique background index builds may produce inconsistent keys

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.4.21, 3.6.13, 4.0.10, 4.1.12
    • Affects Version/s: 4.0.6
    • Component/s: Storage
    • Labels:
      None
    • Fully Compatible
    • ALL
    • v4.0, v3.6, v3.4, v3.2
    • Hide 
      mongo --port 29523 --eval 'var array = []; for (i = 0; i < 1000; i++) { array.push(i); } for (j = 0; j < 25; j++) { db.hugeindex.insert({_id: j, a: array}); } db.hugeindex.createIndex({a: 1}, {background: true});'
mongo --port 29523 --eval 'db.hugeindex.createIndex({a: 1, _id: 1}, {unique: true, background: true});' &
mongo --port 29523 --eval 'db.hugeindex.createIndex({_id: 1, a: 1}, {unique: true, background: true});' &

mongo --port 29523 --eval 'for (j = 0; j < 25; j++) { try { db.hugeindex.findOneAndUpdate({_id: j}, {$pop: {"a": -1}}); db.hugeindex.findOneAndUpdate({_id: j}, {$pop: {"a": 1}}); }catch(exc) {} }' &

mongo --port 29523 --eval 'for (j = 0; j < 3; j++) { try { db.hugeindex.updateOne({_id: j * 10}, {$set: {a: 1}}); db.hugeindex.deleteOne({_id: j * 19}); }catch(exc) {} }' &
      Show
      mongo --port 29523 --eval 'var array = []; for (i = 0; i < 1000; i++) { array.push(i); } for (j = 0; j < 25; j++) { db.hugeindex.insert({_id: j, a: array}); } db.hugeindex.createIndex({a: 1}, {background: true});' mongo --port 29523 --eval 'db.hugeindex.createIndex({a: 1, _id: 1}, {unique: true, background: true});' & mongo --port 29523 --eval 'db.hugeindex.createIndex({_id: 1, a: 1}, {unique: true, background: true});' & mongo --port 29523 --eval 'for (j = 0; j < 25; j++) { try { db.hugeindex.findOneAndUpdate({_id: j}, {$pop: {"a": -1}}); db.hugeindex.findOneAndUpdate({_id: j}, {$pop: {"a": 1}}); }catch(exc) {} }' & mongo --port 29523 --eval 'for (j = 0; j < 3; j++) { try { db.hugeindex.updateOne({_id: j * 10}, {$set: {a: 1}}); db.hugeindex.deleteOne({_id: j * 19}); }catch(exc) {} }' &
    • Storage NYC 2019-05-20, Execution Team 2019-06-03

      Issue Status as of May 23, 2019

      ISSUE DESCRIPTION AND IMPACT
      Background, unique, non-partial indexes on WiredTiger may build indexes that have mismatched keys. As a result, there may be index keys that point to documents without respective matching keys. It is possible that query results that use such an index may return invalid results. Despite this issue, the uniqueness constraint of the index is still maintained and no documents in the collection will violate the constraint.

      DIAGNOSIS AND AFFECTED VERSIONS
      Any user running WiredTiger who has created a background, unique, non-partial index on a non-empty collection prior to MongoDB 3.4.21, 3.6.13, or 4.0.10 while performing deletes or updates to the indexed fields during the index build may have been affected. Users can check the settings used to create an index by calling getIndexes on all of their collections.

      Users who meet the above criteria should run validate against their collection to determine whether their indexes are invalid and need to be rebuilt on a version containing the fix for this issue. In MongoDB 3.4 and before, users should validate with the {full:true} option passed. In MongoDB 3.6 and 4.0, users need not specify {full:true} when invoking the validate command to identify this issue.

      REMEDIATION AND WORKAROUNDS
      The fix is included in the 3.4.21, 3.6.13, 4.0.10 production release. Affected users should upgrade to these versions and rebuild the invalid indexes.

      Original description
      {
	"ns" : "test.hugeindex",
	"nInvalidDocuments" : NumberLong(0),
	"nrecords" : 23,
	"nIndexes" : 4,
	"keysPerIndex" : {
		"_id_" : 23,
		"a_1" : 20960,
		"a_1__id_1" : 20960,
		"_id_1_a_1" : 22958
	},
	"valid" : false,
	"warnings" : [
		"Detected 1998 extra index entries.",
		"Some checks omitted for speed. use {full:true} option to do more thorough scan."
	],
	"errors" : [
		"Index with name '_id_1_a_1' has inconsistencies."
	],
	"extraIndexEntries" : [
		{
			"indexName" : "_id_1_a_1",
			"recordId" : NumberLong(11),
			"indexKey" : {
				"_id" : 10,
				"a" : 0
			}
		},
		{
			"indexName" : "_id_1_a_1",
			"recordId" : NumberLong(11),
			"indexKey" : {
				"_id" : 10,
				"a" : 2
			}
		},
		{
			"indexName" : "_id_1_a_1",
			"recordId" : NumberLong(11),
			"indexKey" : {
				"_id" : 10,
				"a" : 3
			}
		},
<snip>
		{
			"indexName" : "_id_1_a_1",
			"recordId" : NumberLong(21),
			"indexKey" : {
				"_id" : 20,
				"a" : 998
			}
		},
		{
			"indexName" : "_id_1_a_1",
			"recordId" : NumberLong(21),
			"indexKey" : {
				"_id" : 20,
				"a" : 999
			}
		}
	],
	"missingIndexEntries" : [ ],
	"advice" : "A corrupt namespace has been detected. See http://dochub.mongodb.org/core/data-recovery for recovery steps.",
	"ok" : 1
}

            Assignee:
            daniel.gottlieb@mongodb.com Daniel Gottlieb (Inactive)
            Reporter:
            daniel.gottlieb@mongodb.com Daniel Gottlieb (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            15 Start watching this issue

              Created:
              Updated:
              Resolved: