Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-42085

Mongo Server get restarted when running mongodump

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Gone away
    • Affects Version/s: 3.4.2
    • Fix Version/s: None
    • Component/s: Networking, Shell, Storage
    • Labels:
      None
    • Operating System:
      ALL

      Description

      When I try to run below command in the shell,

       

       _./mongodump -h localhost:27017 -d dfnbi_config -o "/home/directfn/AML/mongodb/bin/dump/d" --gzip_
      

       

      shell is showing

      {{Failed: error connecting to db server: no reachable servers}}
      

      later, when I check the mongo log, I can find an exception in the log .

       

      2019-07-04T12:41:05.734+0300 I CONTROL [main] ***** SERVER RESTARTED *****
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] MongoDB starting : pid=6670 port=27017 dbpath=/home/dfnuat/data/mongodb 64-bit host=localhost
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] db version v3.4.2
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] allocator: tcmalloc
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] modules: none
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] build environment:
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] distmod: rhel62
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] distarch: x86_64
      2019-07-04T12:41:05.743+0300 I CONTROL [initandlisten] target_arch: x86_64
      2019-07-04T12:41:05.744+0300 I CONTROL [initandlisten] options: \{ config: "/home/dfnuat/config/mongod.conf", net: { port: 27017 }, processManagement: \{ fork: true, pidFilePath: "/home/dfnuat/data/mongodb/mongod.pid" }, storage: \{ dbPath: "/home/dfnuat/data/mongodb", journal: { enabled: true } }, systemLog: \{ destination: "file", logAppend: true, path: "/home/dfnuat/logs/mongodb/mongod.log" } }
      2019-07-04T12:41:05.744+0300 W - [initandlisten] Detected unclean shutdown - /home/dfnuat/data/mongodb/mongod.lock is not empty.
      2019-07-04T12:41:05.775+0300 I - [initandlisten] Detected data files in /home/dfnuat/data/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
      2019-07-04T12:41:05.775+0300 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
      2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] 
      2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
      2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
      2019-07-04T12:41:05.775+0300 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15539M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2019-07-04T12:41:06.166+0300 I CONTROL [initandlisten] 
      2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
      2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
      2019-07-04T12:41:06.167+0300 I CONTROL [initandlisten] 
      2019-07-04T12:41:06.196+0300 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/home/dfnuat/data/mongodb/diagnostic.data'
      2019-07-04T12:41:06.197+0300 I NETWORK [thread1] waiting for connections on port 27017
      2019-07-04T12:41:07.016+0300 I FTDC [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK
      2019-07-04T12:41:11.575+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56256 #1 (1 connection now open)
      2019-07-04T12:41:11.575+0300 I NETWORK [conn1] received client metadata from 10.160.192.63:56256 conn1: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" }
      2019-07-04T12:41:13.618+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56258 #2 (2 connections now open)
      2019-07-04T12:41:13.618+0300 I NETWORK [conn2] received client metadata from 10.160.192.63:56258 conn2: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" }
      2019-07-04T12:43:29.970+0300 I NETWORK [thread1] connection accepted from 10.160.192.95:49202 #3 (3 connections now open)
      2019-07-04T12:43:29.971+0300 E - [conn3] InputStreamSecureRandom failed to generate random bytes
      2019-07-04T12:43:29.971+0300 I - [conn3] Fatal Assertion 28840 at src/mongo/platform/random.cpp 167
      2019-07-04T12:43:29.971+0300 I - [conn3]
       
      ***aborting after fassert() failure
       
       
      2019-07-04T12:43:29.985+0300 F - [conn3] Got signal: 6 (Aborted).
       
      0x5563705afd21 0x5563705aee19 0x5563705af2fd 0x7fc4e28787e0 0x7fc4e25074f5 0x7fc4e2508cd5 0x55636f837c9d 0x556370353dc9 0x55636fa78b96 0x55636fa964d7 0x55636fa97870 0x5563700b354d 0x55636fcb43e2 0x55636fcb6356 0x55636f8a64fd 0x55636f8a6e3d 0x556370515232 0x7fc4e2870aa1 0x7fc4e25bdc4d
      ----- BEGIN BACKTRACE -----
      {"backtrace":[\{"b":"55636F026000","o":"1589D21","s":"_ZN5mongo15printStackTraceERSo"},\{"b":"55636F026000","o":"1588E19"},\{"b":"55636F026000","o":"15892FD"},\{"b":"7FC4E2869000","o":"F7E0"},\{"b":"7FC4E24D5000","o":"324F5","s":"gsignal"},\{"b":"7FC4E24D5000","o":"33CD5","s":"abort"},\{"b":"55636F026000","o":"811C9D","s":"_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj"},\{"b":"55636F026000","o":"132DDC9","s":"_ZN5mongo23InputStreamSecureRandom9nextInt64Ev"},\{"b":"55636F026000","o":"A52B96","s":"_ZN5mongo11CmdGetNonce3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderE"},\{"b":"55636F026000","o":"A704D7","s":"_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"A71870","s":"_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"108D54D","s":"_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE"},\{"b":"55636F026000","o":"C8E3E2"},\{"b":"55636F026000","o":"C90356","s":"_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE"},\{"b":"55636F026000","o":"8804FD","s":"_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE"},\{"b":"55636F026000","o":"880E3D"},\{"b":"55636F026000","o":"14EF232"},\{"b":"7FC4E2869000","o":"7AA1"},\{"b":"7FC4E24D5000","o":"E8C4D","s":"clone"}],"processInfo":\{ "mongodbVersion" : "3.4.2", "gitVersion" : "3f76e40c105fc223b3e5aac3e20dcd026b83b38b", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "2.6.32-754.12.1.el6.x86_64", "version" : "#1 SMP Thu Mar 7 22:07:44 EST 2019", "machine" : "x86_64" }, "somap" : [ \{ "b" : "55636F026000", "elfType" : 3, "buildId" : "0409C529A50A34D3E255B4350462A560B78F8892" }, \{ "b" : "7FFD2E49D000", "elfType" : 3, "buildId" : "5E4B2841D2BD1BAB1DF5DC73366730BCE4C83264" }, \{ "b" : "7F8E05911000", "path" : "/usr/lib64/libssl.so.10", "elfType" : 3, "buildId" : "D256E285C5E11D9A99EB04CA7651003A8F67B64E" }, \{ "b" : "7F8E09D2C000", "path" : "/usr/lib64/libcrypto.so.10", "elfType" : 3, "buildId" : "1EDB45C205A844A75EBBB4F0075E705803FFB85B" }, \{ "b" : "7F8E0E724000", "path" : "/lib64/librt.so.1", "elfType" : 3, "buildId" : "552CEC3216281CCFD7FA6432C723D50163255823" }, \{ "b" : "7F8E0ED20000", "path" : "/lib64/libdl.so.2", "elfType" : 3, "buildId" : "2AF795BFFD122309BA3359FEBABB5D0967403D17" }, \{ "b" : "7F8E0EE9C000", "path" : "/lib64/libm.so.6", "elfType" : 3, "buildId" : "4AAEE970B045D8BF946578B9C7F3AB5CDE9AB44A" }, \{ "b" : "7F8E0C886000", "path" : "/lib64/libgcc_s.so.1", "elfType" : 3, "buildId" : "EDC925E58FE28DCA536993EB13179C739F1E6566" }, \{ "b" : "7F8E0E269000", "path" : "/lib64/libpthread.so.0", "elfType" : 3, "buildId" : "4EA475CD3FD3B69B6C95D9381FA74B36DB4992EF" }, \{ "b" : "7F8E0EAD5000", "path" : "/lib64/libc.so.6", "elfType" : 3, "buildId" : "A5F93642576869A169BACF9771C674E0CB91FCBB" }, \{ "b" : "7F8E1037D000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "97AF4B77212F74CFF72B6C013E6AA2D74A97EF60" }, \{ "b" : "7F8E08091000", "path" : "/lib64/libgssapi_krb5.so.2", "elfType" : 3, "buildId" : "0C249DF4D77989253CCD859956BF50749308A16A" }, \{ "b" : "7F8E079AA000", "path" : "/lib64/libkrb5.so.3", "elfType" : 3, "buildId" : "624C7056B8BBE6BA758DEF557F516FBDBD01E1FD" }, \{ "b" : "7F8E097A6000", "path" : "/lib64/libcom_err.so.2", "elfType" : 3, "buildId" : "57F77704A7F1F4E3689D028D3F9ADD4E77486EC9" }, \{ "b" : "7F8E0897A000", "path" : "/lib64/libk5crypto.so.3", "elfType" : 3, "buildId" : "C81673692EEF670BC951EE726490F5D1CAB822F4" }, \{ "b" : "7F8E0BF60000", "path" : "/lib64/libresolv.so.2", "elfType" : 3, "buildId" : "4786A2A5D30B121601958E84D643C70C13C4FBA5" }, \{ "b" : "7F8E0C94A000", "path" : "/lib64/libz.so.1", "elfType" : 3, "buildId" : "D053BB4FF0C2FC983842F81598813B9B931AD0D1" }, \{ "b" : "7F8E0673F000", "path" : "/lib64/libkrb5support.so.0", "elfType" : 3, "buildId" : "03B69EEB8998AC9CA7519A27571BAD976BA4C56D" }, \{ "b" : "7F8E0693C000", "path" : "/lib64/libkeyutils.so.1", "elfType" : 3, "buildId" : "3BCCABE75DC61BBA81AAE45D164E26EF4F9F55DB" }, \{ "b" : "7F8E0BF1D000", "path" : "/lib64/libselinux.so.1", "elfType" : 3, "buildId" : "B4576BE308DDCF7BC31F7304E4734C3D846D0236" } ] }}
       mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x5563705afd21]
       mongod(+0x1588E19) [0x5563705aee19]
       mongod(+0x15892FD) [0x5563705af2fd]
       libpthread.so.0(+0xF7E0) [0x7fc4e28787e0]
       libc.so.6(gsignal+0x35) [0x7fc4e25074f5]
       libc.so.6(abort+0x175) [0x7fc4e2508cd5]
       mongod(_ZN5mongo32fassertFailedNoTraceWithLocationEiPKcj+0x0) [0x55636f837c9d]
       mongod(_ZN5mongo23InputStreamSecureRandom9nextInt64Ev+0x109) [0x556370353dc9]
       mongod(_ZN5mongo11CmdGetNonce3runEPNS_16OperationContextERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERNS_7BSONObjEiRS8_RNS_14BSONObjBuilderE+0x56) [0x55636fa78b96]
       mongod(_ZN5mongo7Command3runEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS3_21ReplyBuilderInterfaceE+0x527) [0x55636fa964d7]
       mongod(_ZN5mongo7Command11execCommandEPNS_16OperationContextEPS0_RKNS_3rpc16RequestInterfaceEPNS4_21ReplyBuilderInterfaceE+0x9E0) [0x55636fa97870]
       mongod(_ZN5mongo11runCommandsEPNS_16OperationContextERKNS_3rpc16RequestInterfaceEPNS2_21ReplyBuilderInterfaceE+0x26D) [0x5563700b354d]
       mongod(+0xC8E3E2) [0x55636fcb43e2]
       mongod(_ZN5mongo16assembleResponseEPNS_16OperationContextERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x6D6) [0x55636fcb6356]
       mongod(_ZN5mongo23ServiceEntryPointMongod12_sessionLoopERKSt10shared_ptrINS_9transport7SessionEE+0x1ED) [0x55636f8a64fd]
       mongod(+0x880E3D) [0x55636f8a6e3d]
       mongod(+0x14EF232) [0x556370515232]
       libpthread.so.0(+0x7AA1) [0x7fc4e2870aa1]
       libc.so.6(clone+0x6D) [0x7fc4e25bdc4d]
      ----- END BACKTRACE -----
      2019-07-04T12:44:29.200+0300 I CONTROL [main] ***** SERVER RESTARTED *****
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] MongoDB starting : pid=7110 port=27017 dbpath=/home/dfnuat/data/mongodb 64-bit host=localhost
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] db version v3.4.2
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] allocator: tcmalloc
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] modules: none
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] build environment:
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] distmod: rhel62
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] distarch: x86_64
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] target_arch: x86_64
      2019-07-04T12:44:29.211+0300 I CONTROL [initandlisten] options: \{ config: "/home/dfnuat/config/mongod.conf", net: { port: 27017 }, processManagement: \{ fork: true, pidFilePath: "/home/dfnuat/data/mongodb/mongod.pid" }, storage: \{ dbPath: "/home/dfnuat/data/mongodb", journal: { enabled: true } }, systemLog: \{ destination: "file", logAppend: true, path: "/home/dfnuat/logs/mongodb/mongod.log" } }
      2019-07-04T12:44:29.211+0300 W - [initandlisten] Detected unclean shutdown - /home/dfnuat/data/mongodb/mongod.lock is not empty.
      2019-07-04T12:44:29.232+0300 I - [initandlisten] Detected data files in /home/dfnuat/data/mongodb created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
      2019-07-04T12:44:29.232+0300 W STORAGE [initandlisten] Recovering data from the last clean checkpoint.
      2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] 
      2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
      2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
      2019-07-04T12:44:29.232+0300 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=15539M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
      2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] 
      2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
      2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
      2019-07-04T12:44:29.589+0300 I CONTROL [initandlisten] 
      2019-07-04T12:44:29.618+0300 I FTDC [initandlisten] Initializing full-time diagnostic data capture with directory '/home/dfnuat/data/mongodb/diagnostic.data'
      2019-07-04T12:44:29.619+0300 I NETWORK [thread1] waiting for connections on port 27017
      2019-07-04T12:44:30.026+0300 I FTDC [ftdc] Unclean full-time diagnostic data capture shutdown detected, found interim file, some metrics may have been lost. OK
      2019-07-04T12:44:31.595+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56408 #1 (1 connection now open)
      2019-07-04T12:44:31.596+0300 I NETWORK [conn1] received client metadata from 10.160.192.63:56408 conn1: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" }
      2019-07-04T12:44:33.637+0300 I NETWORK [thread1] connection accepted from 10.160.192.63:56410 #2 (2 connections now open)
      2019-07-04T12:44:33.637+0300 I NETWORK [conn2] received client metadata from 10.160.192.63:56410 conn2: \{ driver: { name: "mongo-java-driver", version: "3.4.2" }, os: \{ type: "Linux", name: "Linux", architecture: "amd64", version: "2.6.32-754.12.1.el6.x86_64" }, platform: "Java/Oracle Corporation/1.8.0_121-b13" }
      2019-07-04T13:06:22.777+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:49505 #3 (3 connections now open)
      2019-07-04T13:06:37.684+0300 I - [conn3] end connection 10.237.149.228:49505 (3 connections now open)
      2019-07-04T13:13:51.695+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50554 #4 (3 connections now open)
      2019-07-04T13:13:51.698+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50555 #5 (4 connections now open)
      2019-07-04T13:13:51.699+0300 I - [conn4] end connection 10.237.149.228:50554 (4 connections now open)
      2019-07-04T13:13:51.915+0300 I - [conn5] end connection 10.237.149.228:50555 (3 connections now open)
      2019-07-04T13:17:22.203+0300 I NETWORK [thread1] connection accepted from 10.237.149.228:50865 #6 (3 connections now open)
      2019-07-04T13:17:27.355+0300 I - [conn6] end connection 10.237.149.228:50865 (3 connections now open)
      

       

        Attachments

          Activity

            People

            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: