[SERVER-46994] checkDBHashesForReplSet is not robust to missing collections Created: 19/Mar/20  Updated: 29/Oct/23  Resolved: 23/Mar/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 4.4.0-rc0, 4.7.0

Type: Bug Priority: Major - P3
Reporter: Daniel Gottlieb (Inactive) Assignee: Lingzhi Deng
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
related to SERVER-39452 Add rollback via refetch logic for st... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Sprint: Repl 2020-04-06
Participants:
Linked BF Score: 0

 Description   

I might not have chased this bug fully[1] but what I've observed is the following:

When checking dbhashes across replica set nodes, a collection that's on the primary, but missing on a secondary will proceed to compare collStats for the missing collection. A collstats for a missing collection:

test> db.server848.stats()
{
	"ns" : "test.server848",
	"size" : 0,
	"count" : 0,
	"storageSize" : 0,
	"totalSize" : 0,
	"nindexes" : 0,
	"totalIndexSize" : 0,
	"indexSizes" : {
		
	},
	"scaleFactor" : 1,
	"ok" : 1
}

First observation of note is that calling collStats on a missing collection returns ok: 1 avoiding the early return.

Second, the indexBuilds field is (javascript) undefined.

If the primary's indexBuilds is undefined (leftArr), the code is robust.

If the secondary's indexBuilds is undefined (rightArr), a type error is hit.

[1] It's not obvious to me whether something changed in how collStats responds, or if this was always a failure waiting to happen.



 Comments   
Comment by Githook User [ 24/Mar/20 ]

Author:

{'email': 'lingzhi.deng@mongodb.com', 'name': 'Lingzhi Deng', 'username': 'ldennis'}

Message: SERVER-46994: Fix potential TypeError in checkDBHashesForReplSet

(cherry picked from commit 9a14bbb32c61293c47fe9c128059686996e4247c)
Branch: v4.4
https://github.com/mongodb/mongo/commit/1c9f5232447d66e2d5123d22d8a3fd29c985d33a

Comment by Githook User [ 23/Mar/20 ]

Author:

{'email': 'lingzhi.deng@mongodb.com', 'name': 'Lingzhi Deng', 'username': 'ldennis'}

Message: SERVER-46994: Fix potential TypeError in checkDBHashesForReplSet
Branch: master
https://github.com/mongodb/mongo/commit/9a14bbb32c61293c47fe9c128059686996e4247c

Comment by Louis Williams [ 23/Mar/20 ]

It looks compareSets should first check whether rightArr is undefined. But it also looks possible that there is a real validation failure.

As for the solution: intuitively, I think that "undefined" should be distinct from an empty set of "indexBuilds". That would indicate a collection is missing from a node. If we would prefer that error to be reported elsewhere, then we can treat undefined as equivalent to an empty set.

Comment by Lingzhi Deng [ 23/Mar/20 ]

louis.williams Even if we fix the typeError, would the dbhash still fail in this case? Do we ever expect an index build on primary but not on secondaries for checkDBHashesForReplSet?

Generated at Thu Feb 08 05:13:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.