[SERVER-14556] Default dbpath for mongod --configsvr changes in 2.6 Created: 15/Jul/14  Updated: 11/Mar/15  Resolved: 17/Jul/14

Status: Closed
Project: Core Server
Component/s: Admin
Affects Version/s: 2.6.3
Fix Version/s: 2.6.4, 2.7.4

Type: Bug Priority: Critical - P2
Reporter: Jason R. Coombs Assignee: Matt Kangas
Resolution: Done Votes: 0
Labels: community-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
Tested
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Completed:
Steps To Reproduce:

1. Deploy sharded cluster using MongoDB 2.4, not specifying dbpath for the mongod --configsvr instance.
2. Follow upgrade procedure for MongoDB 2.6.

Sprint: Server 2.7.4
Participants:

 Description   
Issue Status as of Jul 18, 2014

ISSUE SUMMARY
When a config server is started with mongod --configsvr and no explicit dbpath is specified with --dbpath, MongoDB should use the default /data/configdb instead of /data/db for regular mongod nodes. A bug in the option parser causes config servers to also use the default /data/db.

USER IMPACT
Sharded clusters that do not set an explicit dbpath for their config server mongod processes and rely on the default values are affected by this issue.

The issue is exacerbated in the case where a config server and a regular mongod are co-located on the same host, and both processes rely on default dbpaths. In that case, the config server might read from the regular mongod dbpath, which can lead to undefined behavior and cluster instability.

WORKAROUNDS
Always specify the dbpath explicitly with the --dbpath option for all mongod processes, including config servers.

AFFECTED VERSIONS
Production release versions from 2.6.0 to 2.6.3 are affected by this issue.

FIX VERSION
The fix is included in the 2.6.4 production release.

RESOLUTION DETAILS
The config parser's params.count() did not distinguish between default and non-default options. This has been fixed to restore the default dbpath /data/configdb when using the --configsvr option.

Original description

We've discovered an undocumented backward-incompatible change when upgrading our sharded cluster from MongoDB 2.4 to 2.6.

On 2.4, mongod defaults to `/data/configdb` for the dbpath.

On 2.6, mongod defaults to `/data/db` for the dbpath.

This change must be documented in the Upgrade Documentation but is not currently. If the deployment relies on the default path for the configdb, following the upgrade procedure will take down the entire cluster.



 Comments   
Comment by Githook User [ 22/Jul/14 ]

Author:

{u'username': u'kangas', u'name': u'Matt Kangas', u'email': u'matt.kangas@mongodb.com'}

Message: SERVER-14556 mongod --configsvr default dbpath should be same as 2.4

(cherry picked from commit 7797c0ed4fc2ce61979ae5d8b0931d73e08855bc)

Conflicts:
src/mongo/db/mongod_options.cpp
src/mongo/db/storage_options.h
Branch: v2.6
https://github.com/mongodb/mongo/commit/01c029084ca7866d40fc649022c54d06dddbdd82

Comment by Githook User [ 17/Jul/14 ]

Author:

{u'username': u'kangas', u'name': u'Matt Kangas', u'email': u'matt.kangas@mongodb.com'}

Message: SERVER-14556 mongod --configsvr default dbpath should be same as 2.4
Branch: master
https://github.com/mongodb/mongo/commit/7797c0ed4fc2ce61979ae5d8b0931d73e08855bc

Comment by Thomas Rueckstiess [ 15/Jul/14 ]

Hi Jason,

Thanks for reporting this issue.

I can confirm the different behavior of 2.4.10 vs. 2.6.3 regarding the dbpath, as shown in the log snippets below

2.4.10 startup log messages

mongod --configsvr
Tue Jul 15 14:02:02.802 [initandlisten] MongoDB starting : pid=7337 port=27019 dbpath=/data/configdb master=1 64-bit host=enter.local
Tue Jul 15 14:02:02.802 [initandlisten] db version v2.4.10
Tue Jul 15 14:02:02.802 [initandlisten] git version: e3d78955d181e475345ebd60053a4738a4c5268a
Tue Jul 15 14:02:02.802 [initandlisten] build info: Darwin bs-osx-106-x86-64-2.10gen.cc 10.8.0 Darwin Kernel Version 10.8.0: Tue Jun  7 16:32:41 PDT 2011; root:xnu-1504.15.3~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
Tue Jul 15 14:02:02.802 [initandlisten] allocator: system
Tue Jul 15 14:02:02.802 [initandlisten] options: { configsvr: true }

2.6.3 startup log messages

mongod --configsvr
2014-07-15T14:02:18.255+0200 [initandlisten] MongoDB starting : pid=7352 port=27019 dbpath=/data/db master=1 64-bit host=enter.local
2014-07-15T14:02:18.255+0200 [initandlisten] db version v2.6.3
2014-07-15T14:02:18.255+0200 [initandlisten] git version: 255f67a66f9603c59380b2a389e386910bbb52cb
2014-07-15T14:02:18.255+0200 [initandlisten] build info: Darwin mci-osx108-3.build.10gen.cc 12.3.0 Darwin Kernel Version 12.3.0: Sun Jan  6 22:37:10 PST 2013; root:xnu-2050.22.13~1/RELEASE_X86_64 x86_64 BOOST_LIB_VERSION=1_49
2014-07-15T14:02:18.255+0200 [initandlisten] allocator: system
2014-07-15T14:02:18.255+0200 [initandlisten] options: { sharding: { clusterRole: "configsvr" } }

Based on the 2.6 documentation of the mongod --configsvr option, I believe this is an unintentional change in behavior.

The default port for a mongod with this option is 27019 and the default --dbpath directory is /data/configdb, unless specified.

We'll investigate further and post updates to the ticket once we know more.

In the mean time, the easiest work-around is to ensure that you explicitly set the --dbpath to the desired path.

Regards,
Thomas

Comment by Jason R. Coombs [ 15/Jul/14 ]

Given the mitigating factor that /data/db must exist to take down the cluster, I'd say this is critical but not a blocker.

Comment by Jason R. Coombs [ 15/Jul/14 ]

Note: if /data/db exists, the config server upgrade will complete, creating blank config instances. If /data/db does not exist, the config server upgrade will fail because the new instances will refuse to start. In our case, the former happened because the config servers were built with the same puppet recipes as our shard instances, and that recipe creates /data/db.

Generated at Thu Feb 08 03:35:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.