[SERVER-34609] The db.upgradeCheck() helper silently fails to validate the documents on a collection Created: 23/Apr/18  Updated: 30/Apr/18  Resolved: 30/Apr/18

Status: Closed
Project: Core Server
Component/s: Shell
Affects Version/s: None
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Dmitry Ryabtsev Assignee: DO NOT USE - Backlog - Platform Team
Resolution: Won't Fix Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Operating System: ALL
Participants:
Case:

 Description   

I'm not sure if we still maintain these helpers but they often happen to become helpful in cases about Atlas migrations that are failing due to certain limits being violated on the source (e.g. index key size limit).

Similar to db.upgradeCheckAllDBs(), db.upgradeCheck() has an extra option that enforces document check (checkDocs).

Normally you'd need to call it as db.upgradeCheck(undefined, true); to execute the check on all of the documents in all collections in the database. For the scope of a single collection it would be:

db.upgradeCheck({ collection: "<name>"}, true);

However that does not enforces the document check. The reason is that we don't pass the checkDocs argument into collUpgradeCheck():

146
 DB.prototype.upgradeCheck = function(obj, checkDocs) {
147
        var self = this;
148
        // parse args if there are any
149
        if (obj) {
150
            // check collection if a collection is passed
151
            if (obj["collection"]) {
152
                // make sure a string was passed in for the collection
153
                if (typeof obj["collection"] !== "string") {
154
                    throw Error("The collection field must contain a string");
155
                } else {
156
                    print("Checking collection '" + self.getName() + '.' + obj["collection"] +
157
                          "' for 2.6 upgrade compatibility");
158
                    if (collUpgradeCheck(self.getCollection(obj["collection"]))) {
159
                        print("Everything in '" + self.getName() + '.' + obj["collection"] +
160
                              "' is ready for the upgrade!");
161
                        return true;
162
                    }
163
                    print("To fix the problems above please consult " +
164
                          "http://dochub.mongodb.org/core/upgrade_checker_help");
165
                    return false;
166
                }
167
            } else {
168
                throw Error(
169
                    "When providing an argument to upgradeCheck, it must be of the form " +
170
                    "{collection: <collectionNameString>}. Otherwise, it will check every " +
171
                    "collection in the database. If you would like to check all databases, " +
172
                    "run db.upgradeCheckAllDBs() from the admin database.");
173
            }
174
        }
175
		<...>

It was tested that modifying the collUpgradeCheck() call in the line #158 in upgrade_check.js to have checkDocs fixes the issue.



 Comments   
Comment by Gregory McKeon (Inactive) [ 30/Apr/18 ]

We removed support for these helpers in 3.6, and won't be adding them back to the shell. 

Comment by Dmitry Ryabtsev [ 26/Apr/18 ]

I don't have a clear understanding of the impact

The impact is that there is no way to check the documents in a single collection.

The workaround is to run db.upgradeCheck(undefined, true); - this will check all of the collections in the DB including the one the user wants to check. That may not be optimal however.

The alternative is to apply the fix to the upgrade_check.js file and then have it re-loaded in the shell.

how frequently is this an issue for Support?

Not very frequently but every now and then Atlas users raise a ticket about failed migration due to documents violating the limits. Like in 00491662 where the user runs a source MongoDB deployment with failIndexKeyTooLong set to false (could be b/c of a poor advice that people can on StackOverflow).

If the fix is a single line, maybe the next step to open a pull request that we can review?

It is indeed a single line.

                    if (collUpgradeCheck(self.getCollection(obj["collection"]), checkDocs)) {

Comment by Dmitry Ryabtsev [ 24/Apr/18 ]

Forgot to mention that the code snippet is taken from the 3.4 branch.

Comment by Daniel Pasette (Inactive) [ 23/Apr/18 ]

these helpers have been removed before 3.6 in SERVER-30743

Generated at Thu Feb 08 04:37:15 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.