We have a different code path for handling writes while a column index is being created, and it's called a side-write. Unfortunately, there's a bug and it can break CSI consistency. Here is the reproduction on master:
coll = db.coll; coll.insert({foo:1}); db.adminCommand({configureFailPoint: 'hangAfterStartingIndexBuild', mode: 'alwaysOn'}); coll.createIndex({"$**": "columnstore"});
With that console hanging, do this:
coll = db.coll; coll.insert({cow: 1, pig: 2}); coll.update({cow:1}, {cow: 2}); db.adminCommand({configureFailPoint: 'hangAfterStartingIndexBuild', mode: 'off'});
Now run:
MongoDB Enterprise > coll.find({cow:2}, {cow:1}); { "_id" : ObjectId("63a37e9ed5c25556348142f3"), "cow" : 2 } MongoDB Enterprise > coll.find({cow:2}, {cow:1}).hint({"$**": "columnstore"}); MongoDB Enterprise >
It appears that the delete portion of the update access method is removing keys it shouldn't. See: https://github.com/10gen/mongo/blob/ebbb36a531d6a137d817c4f833050da3964411dd/src/mongo/db/index/columns_access_method.cpp#L378-L389
columnKeys appears to be picking up entries from both the delete and the update diffActions, and so when we run the delete for `pig` above, cow gets wiped out with it. I think you could move columnKeys https://github.com/10gen/mongo/blob/ebbb36a531d6a137d817c4f833050da3964411dd/src/mongo/db/index/columns_access_method.cpp#L371 into the lambda scope to prevent this from happening, but I haven't thought it through.