-
Type:
Task
-
Resolution: Won't Do
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Environment:DocumentDB to Mongo
-
Tools and Replicator
-
0.25
Problem Statement/Rationale
Starting around early June 2024 Amazon added an option to collections in DocumentDB that is not supported by Mongorestore. Below is a code snippet we have tested locally and resolves the issue, however it is too generic for widespread deployment and ought to be restricted by an optional tag passed into Mongorestore.
Likely Epic: Cross-version dump/restore
Steps to Reproduce
Run Mongodump on a DocumentDB database. (specifically
mongodump --uri "mongodb://USERNAME:PASSWORD@HOST:PORT/TARGET_DB?ssl=true&ssl_ca_certs=global-bundle.pem&retryWrites=false&authSource=CREDENTIAL_DB" --gzip --archive="FILE_LOCATION"
)
Attempt to restore with Mongorestore. (specifically
mongorestore --drop --username USERNAME --nsInclude=DATABASE_NAME.* --authenticationDatabase CREDENTIAL_DB --archive=/home/path/backup.archive --gzip -p PASSWORD
)
Expected Results
Regardless of what storage engine the source had, restoration should be into a compatible engine.
Actual Results
Get error (retyped from snip): "error running create command: (InvalidOptions) documentDB is not a registered storage engine for this server"
Additional Notes
Cause: Around early June 2024 Amazon added the documentDB option to all collections. Although this option ought to be persisted on dump, it should be ignored when not restoring to DocumentDB. Specifcally the command db.getCollectionInfos() shows (retyped from snip) -
type: 'collection', options: { autoIndexId: true, capped: false, storageEngine: {documentDB: { compression: {enable: false }}} }, info: { readonly: false},
Partial Fix:
It should be noted that I am not a GO developer and have only spent around 4 hours learning GO to create this fix. That said, this fix does compile and the result of the make command does resolve the issue locally. The down side is that it is an inelegant solution which ought to be limited to only running when a new option is passed into mongorestore such as "mongorestore --ignoreStorageEngine"
cleanOptions := bson.D{} for index, option := range options{ if(option.Key == "storageEngine"){ log.Logv(log.Info, fmt.Sprintf("Skipping option 'storaged engine': %d", index)) }else{ cleanOptions = append(cleanOptions, option) } } options = cleanOptions
as of commit a9eeb2c20a0568ffec5534f94b7b6b47af651355 by John DuBois on Wed June 26, 2024 this code should be inserted in restore.go at line 328:
325 // first create the collection with options from the metadata file 326 uuid := intent.UUID 327 options := intent.Options //snippet goes here to fix. 328 if len(options) == 0 { 329 logMessageSuffix = "with no metadata" 330 }
- is related to
-
TOOLS-3614 Mongorestore should ignore InvalidOptions error if unknown option is documentDB specific
-
- Needs Scheduling
-