Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-24583

Running "service mongod start" more than once on RHEL doesn't allow "service mongod stop" to stop the instance

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 3.2.5
    • Component/s: Build, Shell
    • Labels:
      None
    • Build
    • ALL
    • Hide

      1. sudo yum install -y mongodb-org-3.2.5
      2. run "sudo service mongod start"
      3. run "sudo service mongod start"
      4. run "sudo service mongod stop"

      Show
      1. sudo yum install -y mongodb-org-3.2.5 2. run "sudo service mongod start" 3. run "sudo service mongod start" 4. run "sudo service mongod stop"

      Running service mongod start on a running instance shows failure on terminal but tries to spawn a new process (see mongod log) and fails due to socket already in use.

      This causes service mongod stop to inturn fail in stopping the instance although it shows no error.

      $ uname -ar
Linux ip-172-31-27-237.ap-southeast-1.compute.internal 2.6.32-504.16.2.el6.x86_64 #1 SMP Tue Mar 10 17:01:00 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux

$ sudo service mongod start

$ ps -ef|grep mongo
mongod    2007     1  2 02:30 ?        00:00:00 /usr/bin/mongod -f /etc/mongod.conf
ec2-user  2026  1421  0 02:30 pts/0    00:00:00 grep mongo

$ sudo service mongod start
Starting mongod:                                           [FAILED]

$ ps -ef|grep mongo
mongod    2007     1  1 02:30 ?        00:00:00 /usr/bin/mongod -f /etc/mongod.conf
ec2-user  2053  1421  0 02:30 pts/0    00:00:00 grep mongo

$ sudo service mongod stop
Stopping mongod:                                           [  OK  ]

$ ps -ef|grep mongo
mongod    2007     1  0 02:30 ?        00:00:02 /usr/bin/mongod -f /etc/mongod.conf
ec2-user  2110  1421  0 02:39 pts/0    00:00:00 grep mongo

      mongod log

      2016-06-15T02:30:30.831-0400 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-06-15T02:30:30.837-0400 I CONTROL  [initandlisten] MongoDB starting : pid=2007 port=27017 dbpath=/var/lib/mongo 64-bit host=ip-172-31-27-237.ap-southeast-1.compute.internal
2016-06-15T02:30:30.837-0400 I CONTROL  [initandlisten] db version v3.2.5
2016-06-15T02:30:30.837-0400 I CONTROL  [initandlisten] git version: 34e65e5383f7ea1726332cb175b73077ec4a1b02
2016-06-15T02:30:30.837-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten] allocator: tcmalloc
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten] modules: none
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten] build environment:
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten]     distmod: rhel62
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten]     distarch: x86_64
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2016-06-15T02:30:30.838-0400 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongo", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-06-15T02:30:30.862-0400 I -        [initandlisten] Detected data files in /var/lib/mongo created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-06-15T02:30:30.862-0400 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 1024 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files.
2016-06-15T02:30:31.037-0400 I CONTROL  [initandlisten] 
2016-06-15T02:30:31.039-0400 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongo/diagnostic.data'
2016-06-15T02:30:31.040-0400 I NETWORK  [initandlisten] waiting for connections on port 27017
2016-06-15T02:30:31.042-0400 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-06-15T02:30:44.216-0400 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] MongoDB starting : pid=2049 port=27017 dbpath=/var/lib/mongo 64-bit host=ip-172-31-27-237.ap-southeast-1.compute.internal
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] db version v3.2.5
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] git version: 34e65e5383f7ea1726332cb175b73077ec4a1b02
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] allocator: tcmalloc
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] modules: none
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] build environment:
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten]     distmod: rhel62
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten]     distarch: x86_64
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten]     target_arch: x86_64
2016-06-15T02:30:44.222-0400 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongo", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-06-15T02:30:44.243-0400 E NETWORK  [initandlisten] listen(): bind() failed errno:98 Address already in use for socket: 127.0.0.1:27017
2016-06-15T02:30:44.243-0400 E NETWORK  [initandlisten]   addr already in use
2016-06-15T02:30:44.243-0400 E STORAGE  [initandlisten] Failed to set up sockets during startup.
2016-06-15T02:30:44.243-0400 I CONTROL  [initandlisten] dbexit:  rc: 48

            Assignee:
            backlog-server-build Backlog - Build Team (Inactive)
            Reporter:
            ankur.raina@mongodb.com Ankur Raina (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: