Uploaded image for project: 'MongoDB ETL Tools'
  1. MongoDB ETL Tools
  2. TOOLS-2038

mongorestore fails when source is from the encrypted storage engine

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Works as Designed
    • Affects Version/s: 3.2.20, 3.6.4, 3.4.15
    • Fix Version/s: None
    • Component/s: mongodump, mongorestore
    • Labels:
      None
    • Environment:
      MacOS
    • Case:

      Description

      Requested a config dump and was unable to restore due to the following errors:

      mongorestore --drop --gzip dump/mongorestore --drop --gzip dump/
      2018-05-16T09:09:47.124-0400 Failed: config.chunks: error creating collection config.chunks: error running create command: 22: Invalid argument
      

      The mongodb log file shows the following:

      2018-05-16T09:09:47.159-0400 E STORAGE  [conn25] WiredTiger error (22) [1526476187:159247][37561:0x70000feb4000], file:collection-54--1620569275930459397.wt, WT_SESSION.create: unknown encryptor 'AES256-CBC': Invalid argument
      2018-05-16T09:09:47.159-0400 I -        [conn25] Assertion: 2:22: Invalid argument src/mongo/db/catalog/database.cpp 545
      

      The source was encrypted at rest but the backup is not encrypted however the metadata for the dump contains the encryption specifier in the configString (N.B. Pretty printed below for readability)

      version.metadata.json
      {  
         "options":{  
            "flags":1,
            "storageEngine":{  
               "wiredTiger":{  
                  "configString":"access_pattern_hint=none,allocation_size=4KB,app_metadata=(formatVersion=1),block_allocation=best,block_compressor=snappy,cache_resident=false,checksum=on,colgroups=,collator=,columns=,dictionary=0,encryption=(keyid=\"config\",name=AES256-CBC),exclusive=false,extractor=,format=btree,huffman_key=,huffman_value=,ignore_in_memory_cache_size=false,immutable=false,internal_item_max=0,internal_key_max=0,internal_key_truncate=true,internal_page_max=4KB,key_format=q,key_gap=10,leaf_item_max=0,leaf_key_max=0,leaf_page_max=32KB,leaf_value_max=64MB,log=(enabled=true),lsm=(auto_throttle=true,bloom=true,bloom_bit_count=16,bloom_config=,bloom_hash_count=8,bloom_oldest=false,chunk_count_limit=0,chunk_max=5GB,chunk_size=10MB,merge_max=15,merge_min=0),memory_page_max=10m,os_cache_dirty_max=0,os_cache_max=0,prefix_compression=false,prefix_compression_min=4,source=,split_deepen_min_child=0,split_deepen_per_child=0,split_pct=90,type=file,value_format=u,type=file"
               }
            }
         },
         "indexes":[  
            {  
               "v":2,
               "key":{  
                  "_id":1
               },
               "name":"_id_",
               "ns":"config.version"
            }
         ]
      }
      

      Workaround

      Hacking the metadata to remove the encryption key allows me to successfully restore the dump.

      sed -i "s/encryption.*exclusive/exclusive/" *.metadata.json
      

        Attachments

          Activity

            People

            • Assignee:
              david.golden David Golden
              Reporter:
              vick.mena Vick Mena
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: