Unable to start queryable mongod because it failed to regenerate index for admin.system.users

XMLWordPrintableJSON

    • Fully Compatible
    • ALL
    • v3.6, v3.4
    • Platforms 2018-02-12
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None

      Unable to start queryable mongod for the backup of mongod version 3.4.10.

      indexes on admin.system.user on mongod 3.4.10:

      db.system.users.getIndexes()
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "admin.system.users"
	}
]

      indexes on admin.system.user on mongod 3.4.11:

      db.system.users.getIndexes()
[
	{
		"v" : 2,
		"key" : {
			"_id" : 1
		},
		"name" : "_id_",
		"ns" : "admin.system.users"
	},
	{
		"v" : 2,
		"unique" : true,
		"key" : {
			"user" : 1,
			"db" : 1
		},
		"name" : "user_1_db_1",
		"ns" : "admin.system.users"
	}
]

      Queryable restore uses the latest mongod available to start queryable mongod. So for the backup of mongod 3.4.10 it will use queryable mongod 3.4.11. While starting queryable mongod, it tries to create the extra index introduced in mongod 3.4.11, and it fails because queryable mongod is started in read only mode. Following is the error encountered:

      2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] MongoDB starting : pid=20564 port=27812 dbpath=/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath/ 64-bit host=brs-qr-2.vpc8.10gen.cc
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] db version v3.4.11
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] git version: 34f5bec2c9d827d71828fe858167f89a28b29a2a
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] allocator: tcmalloc
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] modules: enterprise
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] build environment:
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten]     distmod: rhel62
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten]     distarch: x86_64
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten]     target_arch: x86_64
2018-02-02T08:19:12.240+0000 I CONTROL  [initandlisten] options: { config: "/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath/conf.yaml", net: { bindIp: "0.0.0.0", port: 27812, ssl: { CAFile: "/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath/ca.pem", PEMKeyFile: "/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath/serverIdentity.pem", mode: "requireSSL" } }, queryableBackup: { apiUri: "127.0.0.1:8095", memoryQuotaMB: 8192.0, snapshotId: "5a741effdf9db17ed7c5b844" }, security: { authorization: "enabled" }, setParameter: { authenticationMechanisms: "MONGODB-X509" }, storage: { dbPath: "/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath/", engine: "queryable_wt", queryableBackupMode: true, wiredTiger: { engineConfig: { cacheSizeGB: 6.0 } } }, systemLog: { destination: "file", path: "/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/mongod.log" } }
2018-02-02T08:19:12.258+0000 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=6144M,session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),extensions=[local={entry=queryableWtFsCreate,early_load=true,config={apiUri="127.0.0.1:8095",snapshotId="5a741effdf9db17ed7c5b844",dbpath="/srv/10gen/bgrid-prod/head01/queryable/5a741effdf9db17ed7c5b844/dbpath"}},],readonly=true,
2018-02-02T08:19:18.792+0000 W STORAGE  [initandlisten] Detected configuration for non-active storage engine wiredTiger when current storage engine is queryable_wt
2018-02-02T08:19:18.955+0000 I ACCESS   [initandlisten] No authorization index detected on admin.system.users collection. Attempting to recover by creating an index with spec: { ns: "admin.system.users", v: 2, name: "user_1_db_1", key: { user: 1, db: 1 }, unique: true }
2018-02-02T08:19:18.955+0000 F ACCESS   [initandlisten] Failed to regenerate index for admin.system.users. Exception: Cannot execute a write operation in read-only mode
2018-02-02T08:19:18.955+0000 I STORAGE  [initandlisten] IllegalOperation: Cannot execute a write operation in read-only mode

            Assignee:
            Spencer Jackson
            Reporter:
            Ankur Srivastava (Inactive)
            Votes:
            1 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved: