Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-34746

Segmentation fault when shard is started with --shardsvr before being added to a shard

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 3.6.5
    • Affects Version/s: 3.6.4
    • Component/s: Sharding
    • None
    • Fully Compatible
    • ALL
    • Hide

      Set up in docker a replica set with:

      mongo1:
    container_name: mongo1
    image: mongo:3.6.3
    command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27017
    ports:
      - 27017:27017
    expose:
      - 27017
    environment:
      TERM: xterm
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/mongo-rs/mongo1:/data/db
  mongo2:
    container_name: mongo2
    image: mongo:3.6.3
    command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27027
    ports:
      - 27027:27027
    expose:
      - 27027
    environment:
      TERM: xterm
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/mongo-rs/mongo2:/data/db
  mongo3:
    container_name: mongo3
    image: mongo:3.6.3
    command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27037
    ports:
      - 27037:27037
    expose:
      - 27037
    environment:
      TERM: xterm
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/docker/mongo-rs/mongo3:/data/db

      Replica set in configured with:

      config = {
    "_id" : "RS1",
    "members" : [
        {
            "_id" : 0,
            "host" : "mongo1:27017"
        },
        {
            "_id" : 1,
            "host" : "mongo2:27027"
        },
        {
            "_id" : 2,
            "host" : "mongo3:27037"
        }
    ]
}
rs.initiate(config);

      Collection is set with:

      db.adminCommand({
    setFeatureCompatibilityVersion: "3.6"
});
databaseName = "myDatabase";
collectionName = "people";

db = db.getSiblingDB(databaseName);
collection = db.getCollection(collectionName);
collection.drop();db.createCollection(collectionName, {
   validator: { $jsonSchema: {
      bsonType: "object",
      required: ["name","description","address"],
      properties: {
         name: {
            bsonType: "string",
            description: "must be a string and is required"
         },
         description: {
            bsonType: "string",
            description: "must be a string and is required"
         },
         address: {
            bsonType: "string",
            description: "must be a string and is required"
         }
      }
   }}
});
collection = db.getCollection(collectionName);
collection.createIndex({
    name: -1,
    description: "text"
});

      After that, after a random amount of time the whole set of nodes crash.

      Let me know if there's further data I can provide.

      Cheers

       

      Show
      Set up in docker a replica set with: mongo1: container_name: mongo1 image: mongo:3.6.3 command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27017 ports: - 27017:27017 expose: - 27017 environment: TERM: xterm volumes: - /etc/localtime:/etc/localtime:ro - /opt/docker/mongo-rs/mongo1:/data/db mongo2: container_name: mongo2 image: mongo:3.6.3 command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27027 ports: - 27027:27027 expose: - 27027 environment: TERM: xterm volumes: - /etc/localtime:/etc/localtime:ro - /opt/docker/mongo-rs/mongo2:/data/db mongo3: container_name: mongo3 image: mongo:3.6.3 command: mongod --shardsvr --replSet RS1 --dbpath /data/db --port 27037 ports: - 27037:27037 expose: - 27037 environment: TERM: xterm volumes: - /etc/localtime:/etc/localtime:ro - /opt/docker/mongo-rs/mongo3:/data/db Replica set in configured with: config = { "_id" : "RS1" , "members" : [ { "_id" : 0, "host" : "mongo1:27017" }, { "_id" : 1, "host" : "mongo2:27027" }, { "_id" : 2, "host" : "mongo3:27037" } ] } rs.initiate(config); Collection is set with: db.adminCommand({ setFeatureCompatibilityVersion: "3.6" }); databaseName = "myDatabase" ; collectionName = "people" ; db = db.getSiblingDB(databaseName); collection = db.getCollection(collectionName); collection.drop();db.createCollection(collectionName, { validator: { $jsonSchema: { bsonType: "object" , required: [ "name" , "description" , "address" ], properties: { name: { bsonType: "string" , description: "must be a string and is required" }, description: { bsonType: "string" , description: "must be a string and is required" }, address: { bsonType: "string" , description: "must be a string and is required" } } }} }); collection = db.getCollection(collectionName); collection.createIndex({ name: -1, description: "text" }); After that, after a random amount of time the whole set of nodes crash. Let me know if there's further data I can provide. Cheers  
    • Sharding 2018-05-21

      After some time the replica set is running in Docker all nodes of my replica set crash due to segmentation fault.

      This happened to me every time I use JSON schema validation in a collection. When I don't, all nodes work fine.

      Here I am pasting the logs:

      mongo2            | 2018-04-30T11:10:25.905+0200 F -        [thread74] Invalid access at address: 0x18
mongo2            | 2018-04-30T11:10:25.947+0200 F -        [thread74] Got signal: 11 (Segmentation fault).
mongo2            | 
mongo2            |  0x55f86f905f11 0x55f86f905129 0x55f86f905796 0x7fe038fea890 0x7fe038fe5234 0x55f86f0e0306 0x55f86f0e5091 0x55f86e7741fe 0x55f86e774520 0x55f86f1e3b8a 0x55f86f1e5038 0x55f86e75b5a2 0x55f86f3e01fa 0x55f86f3e0bbc 0x55f86f3e0f94 0x55f86f4569f9 0x55f86f456c41 0x55f86e75a4ad 0x55f86fa14fd0 0x7fe038fe3064 0x7fe038d1862d
mongo2            | ----- BEGIN BACKTRACE -----
mongo2            | {"backtrace":[{"b":"55F86D6F4000","o":"2211F11","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55F86D6F4000","o":"2211129"},{"b":"55F86D6F4000","o":"2211796"},{"b":"7FE038FDB000","o":"F890"},{"b":"7FE038FDB000","o":"A234","s":"pthread_mutex_lock"},{"b":"55F86D6F4000","o":"19EC306","s":"_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE"},{"b":"55F86D6F4000","o":"19F1091","s":"_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE"},{"b":"55F86D6F4000","o":"10801FE","s":"_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE"},{"b":"55F86D6F4000","o":"1080520","s":"_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE"},{"b":"55F86D6F4000","o":"1AEFB8A","s":"_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE"},{"b":"55F86D6F4000","o":"1AF1038","s":"_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE"},{"b":"55F86D6F4000","o":"10675A2"},{"b":"55F86D6F4000","o":"1CEC1FA","s":"_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_"},{"b":"55F86D6F4000","o":"1CECBBC","s":"_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_"},{"b":"55F86D6F4000","o":"1CECF94","s":"_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m"},{"b":"55F86D6F4000","o":"1D629F9","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"55F86D6F4000","o":"1D62C41","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"55F86D6F4000","o":"10664AD"},{"b":"55F86D6F4000","o":"2320FD0"},{"b":"7FE038FDB000","o":"8064"},{"b":"7FE038C30000","o":"E862D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.3", "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-112-generic", "version" : "#135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "55F86D6F4000", "elfType" : 3, "buildId" : "F9BA3BD99A1212BD7BCFA9EDA7EB7CC05FEBB67C" }, { "b" : "7FFDB03DF000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "FA97F4849697BBE252BA1F7FB2316979E93E61DE" }, { "b" : "7FE03A178000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "C0E9A6CE03F960E690EA8F72575FFA29570E4A0B" }, { "b" : "7FE039F17000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "21115992A1F885E1ACE88AADA60F126AD9759D03" }, { "b" : "7FE039B1B000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "FD6376149047833953B0269E84DE181CA45DBE90" }, { "b" : "7FE039913000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "A63C95FB33CCA970E141D2E13774B997C1CF0565" }, { "b" : "7FE03970F000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D70B531D672A34D71DB42EB32B68E63F2DCC5B6A" }, { "b" : "7FE03940E000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "152C93BA3E8590F7ED0BCDDF868600D55EC4DD6F" }, { "b" : "7FE0391F8000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "BAC839560495859598E8515CBAED73C7799AE1FF" }, { "b" : "7FE038FDB000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "9DA9387A60FFC196AEDB9526275552AFEF499C44" }, { "b" : "7FE038C30000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "48C48BC6ABB794461B8A558DD76B29876A0551F0" }, { "b" : "7FE03A38F000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "1D98D41FBB1EABA7EC05D0FD7624B85D6F51C03C" } ] }}
mongo2            |  mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55f86f905f11]
mongo2            |  mongod(+0x2211129) [0x55f86f905129]
mongo2            |  mongod(+0x2211796) [0x55f86f905796]
mongo2            |  libpthread.so.0(+0xF890) [0x7fe038fea890]
mongo2            |  libpthread.so.0(pthread_mutex_lock+0x4) [0x7fe038fe5234]
mongo2            |  mongod(_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE+0x46) [0x55f86f0e0306]
mongo2            |  mongod(_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE+0x41) [0x55f86f0e5091]
mongo2            |  mongod(_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE+0x10E) [0x55f86e7741fe]
mongo2            |  mongod(_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE+0x20) [0x55f86e774520]
mongo2            |  mongod(_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE+0x12A) [0x55f86f1e3b8a]
mongo2            |  mongod(_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE+0x28) [0x55f86f1e5038]
mongo2            |  mongod(+0x10675A2) [0x55f86e75b5a2]
mongo2            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_+0x7A) [0x55f86f3e01fa]
mongo2            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_+0x3AC) [0x55f86f3e0bbc]
mongo2            |  mongod(_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m+0x164) [0x55f86f3e0f94]
mongo2            |  mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code+0x389) [0x55f86f4569f9]
mongo2            |  mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xD1) [0x55f86f456c41]
mongo2            |  mongod(+0x10664AD) [0x55f86e75a4ad]
mongo2            |  mongod(+0x2320FD0) [0x55f86fa14fd0]
mongo2            |  libpthread.so.0(+0x8064) [0x7fe038fe3064]
mongo2            |  libc.so.6(clone+0x6D) [0x7fe038d1862d]
mongo2            | -----  END BACKTRACE  -----
 
      mongo3            | 2018-04-30T11:10:26.083+0200 I NETWORK  [conn10] end connection 172.19.0.8:34336 (9 connections now open)
mongo3            | 2018-04-30T11:10:26.083+0200 I NETWORK  [conn5] end connection 172.19.0.8:34290 (8 connections now open)
mongo3            | 2018-04-30T11:10:26.083+0200 I NETWORK  [conn11] end connection 172.19.0.8:34338 (7 connections now open)
mongo3            | 2018-04-30T11:10:26.084+0200 I NETWORK  [conn15] end connection 172.19.0.8:34346 (6 connections now open)
mongo2 exited with code 139
mongo3            | 2018-04-30T11:10:26.945+0200 I NETWORK  [listener] connection accepted from 172.19.0.2:42980 #75 (7 connections now open)
mongo3            | 2018-04-30T11:10:26.955+0200 F -        [thread76] Invalid access at address: 0x18
mongo3            | 2018-04-30T11:10:26.982+0200 I NETWORK  [conn75] end connection 172.19.0.2:42980 (6 connections now open)
mongo3            | 2018-04-30T11:10:26.985+0200 F -        [thread76] Got signal: 11 (Segmentation fault).
mongo3            | 
mongo3            |  0x561674202f11 0x561674202129 0x561674202796 0x7f7ac94db890 0x7f7ac94d6234 0x5616739dd306 0x5616739e2091 0x5616730711fe 0x561673071520 0x561673ae0b8a 0x561673ae2038 0x5616730585a2 0x561673cdd1fa 0x561673cddbbc 0x561673cddf94 0x561673d539f9 0x561673d53c41 0x5616730574ad 0x561674311fd0 0x7f7ac94d4064 0x7f7ac920962d
mongo3            | ----- BEGIN BACKTRACE -----
mongo3            | {"backtrace":[{"b":"561671FF1000","o":"2211F11","s":"_ZN5mongo15printStackTraceERSo"},{"b":"561671FF1000","o":"2211129"},{"b":"561671FF1000","o":"2211796"},{"b":"7F7AC94CC000","o":"F890"},{"b":"7F7AC94CC000","o":"A234","s":"pthread_mutex_lock"},{"b":"561671FF1000","o":"19EC306","s":"_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE"},{"b":"561671FF1000","o":"19F1091","s":"_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE"},{"b":"561671FF1000","o":"10801FE","s":"_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE"},{"b":"561671FF1000","o":"1080520","s":"_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE"},{"b":"561671FF1000","o":"1AEFB8A","s":"_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE"},{"b":"561671FF1000","o":"1AF1038","s":"_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE"},{"b":"561671FF1000","o":"10675A2"},{"b":"561671FF1000","o":"1CEC1FA","s":"_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_"},{"b":"561671FF1000","o":"1CECBBC","s":"_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_"},{"b":"561671FF1000","o":"1CECF94","s":"_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m"},{"b":"561671FF1000","o":"1D629F9","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"561671FF1000","o":"1D62C41","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"561671FF1000","o":"10664AD"},{"b":"561671FF1000","o":"2320FD0"},{"b":"7F7AC94CC000","o":"8064"},{"b":"7F7AC9121000","o":"E862D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.3", "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-112-generic", "version" : "#135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "561671FF1000", "elfType" : 3, "buildId" : "F9BA3BD99A1212BD7BCFA9EDA7EB7CC05FEBB67C" }, { "b" : "7FFFBA33C000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "FA97F4849697BBE252BA1F7FB2316979E93E61DE" }, { "b" : "7F7ACA669000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "C0E9A6CE03F960E690EA8F72575FFA29570E4A0B" }, { "b" : "7F7ACA408000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "21115992A1F885E1ACE88AADA60F126AD9759D03" }, { "b" : "7F7ACA00C000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "FD6376149047833953B0269E84DE181CA45DBE90" }, { "b" : "7F7AC9E04000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "A63C95FB33CCA970E141D2E13774B997C1CF0565" }, { "b" : "7F7AC9C00000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D70B531D672A34D71DB42EB32B68E63F2DCC5B6A" }, { "b" : "7F7AC98FF000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "152C93BA3E8590F7ED0BCDDF868600D55EC4DD6F" }, { "b" : "7F7AC96E9000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "BAC839560495859598E8515CBAED73C7799AE1FF" }, { "b" : "7F7AC94CC000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "9DA9387A60FFC196AEDB9526275552AFEF499C44" }, { "b" : "7F7AC9121000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "48C48BC6ABB794461B8A558DD76B29876A0551F0" }, { "b" : "7F7ACA880000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "1D98D41FBB1EABA7EC05D0FD7624B85D6F51C03C" } ] }}
mongo3            |  mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x561674202f11]
mongo3            |  mongod(+0x2211129) [0x561674202129]
mongo3            |  mongod(+0x2211796) [0x561674202796]
mongo3            |  libpthread.so.0(+0xF890) [0x7f7ac94db890]
mongo3            |  libpthread.so.0(pthread_mutex_lock+0x4) [0x7f7ac94d6234]
mongo3            |  mongod(_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE+0x46) [0x5616739dd306]
mongo3            |  mongod(_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE+0x41) [0x5616739e2091]
mongo3            |  mongod(_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE+0x10E) [0x5616730711fe]
mongo3            |  mongod(_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE+0x20) [0x561673071520]
mongo3            |  mongod(_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE+0x12A) [0x561673ae0b8a]
mongo3            |  mongod(_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE+0x28) [0x561673ae2038]
mongo3            |  mongod(+0x10675A2) [0x5616730585a2]
mongo3            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_+0x7A) [0x561673cdd1fa]
mongo3            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_+0x3AC) [0x561673cddbbc]
mongo3            |  mongod(_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m+0x164) [0x561673cddf94]
mongo3            |  mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code+0x389) [0x561673d539f9]
mongo3            |  mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xD1) [0x561673d53c41]
mongo3            |  mongod(+0x10664AD) [0x5616730574ad]
mongo3            |  mongod(+0x2320FD0) [0x561674311fd0]
mongo3            |  libpthread.so.0(+0x8064) [0x7f7ac94d4064]
mongo3            |  libc.so.6(clone+0x6D) [0x7f7ac920962d]
mongo3            | -----  END BACKTRACE  -----
 
      mongo1            | 2018-04-30T11:10:26.992+0200 F -        [thread71] Invalid access at address: 0x18
mongo1            | 2018-04-30T11:10:27.005+0200 F -        [thread71] Got signal: 11 (Segmentation fault).
mongo1            | 
mongo1            |  0x558e136f6f11 0x558e136f6129 0x558e136f6796 0x7fccace4b890 0x7fccace46234 0x558e12ed1306 0x558e12ed6091 0x558e125651fe 0x558e12565520 0x558e12fd4b8a 0x558e12fd6038 0x558e1254c5a2 0x558e131d11fa 0x558e131d1bbc 0x558e131d1f94 0x558e132479f9 0x558e13247c41 0x558e1254b4ad 0x558e13805fd0 0x7fccace44064 0x7fccacb7962d
mongo1            | ----- BEGIN BACKTRACE -----
mongo1            | {"backtrace":[{"b":"558E114E5000","o":"2211F11","s":"_ZN5mongo15printStackTraceERSo"},{"b":"558E114E5000","o":"2211129"},{"b":"558E114E5000","o":"2211796"},{"b":"7FCCACE3C000","o":"F890"},{"b":"7FCCACE3C000","o":"A234","s":"pthread_mutex_lock"},{"b":"558E114E5000","o":"19EC306","s":"_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE"},{"b":"558E114E5000","o":"19F1091","s":"_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE"},{"b":"558E114E5000","o":"10801FE","s":"_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE"},{"b":"558E114E5000","o":"1080520","s":"_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE"},{"b":"558E114E5000","o":"1AEFB8A","s":"_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE"},{"b":"558E114E5000","o":"1AF1038","s":"_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE"},{"b":"558E114E5000","o":"10675A2"},{"b":"558E114E5000","o":"1CEC1FA","s":"_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_"},{"b":"558E114E5000","o":"1CECBBC","s":"_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_"},{"b":"558E114E5000","o":"1CECF94","s":"_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m"},{"b":"558E114E5000","o":"1D629F9","s":"_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code"},{"b":"558E114E5000","o":"1D62C41","s":"_ZN4asio6detail9scheduler3runERSt10error_code"},{"b":"558E114E5000","o":"10664AD"},{"b":"558E114E5000","o":"2320FD0"},{"b":"7FCCACE3C000","o":"8064"},{"b":"7FCCACA91000","o":"E862D","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.3", "gitVersion" : "9586e557d54ef70f9ca4b43c26892cd55257e1a5", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.4.0-112-generic", "version" : "#135-Ubuntu SMP Fri Jan 19 11:48:36 UTC 2018", "machine" : "x86_64" }, "somap" : [ { "b" : "558E114E5000", "elfType" : 3, "buildId" : "F9BA3BD99A1212BD7BCFA9EDA7EB7CC05FEBB67C" }, { "b" : "7FFDA834D000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "FA97F4849697BBE252BA1F7FB2316979E93E61DE" }, { "b" : "7FCCADFD9000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "C0E9A6CE03F960E690EA8F72575FFA29570E4A0B" }, { "b" : "7FCCADD78000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.0.0", "elfType" : 3, "buildId" : "21115992A1F885E1ACE88AADA60F126AD9759D03" }, { "b" : "7FCCAD97C000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0", "elfType" : 3, "buildId" : "FD6376149047833953B0269E84DE181CA45DBE90" }, { "b" : "7FCCAD774000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "A63C95FB33CCA970E141D2E13774B997C1CF0565" }, { "b" : "7FCCAD570000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "D70B531D672A34D71DB42EB32B68E63F2DCC5B6A" }, { "b" : "7FCCAD26F000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "152C93BA3E8590F7ED0BCDDF868600D55EC4DD6F" }, { "b" : "7FCCAD059000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "BAC839560495859598E8515CBAED73C7799AE1FF" }, { "b" : "7FCCACE3C000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "9DA9387A60FFC196AEDB9526275552AFEF499C44" }, { "b" : "7FCCACA91000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "48C48BC6ABB794461B8A558DD76B29876A0551F0" }, { "b" : "7FCCAE1F0000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "1D98D41FBB1EABA7EC05D0FD7624B85D6F51C03C" } ] }}
mongo1            |  mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x558e136f6f11]
mongo1            |  mongod(+0x2211129) [0x558e136f6129]
mongo1            |  mongod(+0x2211796) [0x558e136f6796]
mongo1            |  libpthread.so.0(+0xF890) [0x7fccace4b890]
mongo1            |  libpthread.so.0(pthread_mutex_lock+0x4) [0x7fccace46234]
mongo1            |  mongod(_ZN5mongo12CatalogCache27invalidateShardedCollectionERKNS_15NamespaceStringE+0x46) [0x558e12ed1306]
mongo1            |  mongod(_ZN5mongo12CatalogCache42getShardedCollectionRoutingInfoWithRefreshEPNS_16OperationContextERKNS_15NamespaceStringE+0x41) [0x558e12ed6091]
mongo1            |  mongod(_ZN5mongo25SessionsCollectionSharded32_checkCacheForSessionsCollectionEPNS_16OperationContextE+0x10E) [0x558e125651fe]
mongo1            |  mongod(_ZN5mongo25SessionsCollectionSharded23setupSessionsCollectionEPNS_16OperationContextE+0x20) [0x558e12565520]
mongo1            |  mongod(_ZN5mongo23LogicalSessionCacheImpl8_refreshEPNS_6ClientE+0x12A) [0x558e12fd4b8a]
mongo1            |  mongod(_ZN5mongo23LogicalSessionCacheImpl16_periodicRefreshEPNS_6ClientE+0x28) [0x558e12fd6038]
mongo1            |  mongod(+0x10675A2) [0x558e1254c5a2]
mongo1            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_7binder1ISt8functionIFvSt10error_codeEES5_EEEEvRPNS1_11strand_implERT_+0x7A) [0x558e131d11fa]
mongo1            |  mongod(_ZN4asio6detail14strand_service8dispatchINS0_17rewrapped_handlerINS0_7binder1INS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEES9_EESB_EEEEvRPNS1_11strand_implERT_+0x3AC) [0x558e131d1bbc]
mongo1            |  mongod(_ZN4asio6detail12wait_handlerINS0_15wrapped_handlerINS_10io_context6strandESt8functionIFvSt10error_codeEENS0_26is_continuation_if_runningEEEE11do_completeEPvPNS0_19scheduler_operationERKS6_m+0x164) [0x558e131d1f94]
mongo1            |  mongod(_ZN4asio6detail9scheduler10do_run_oneERNS0_27conditionally_enabled_mutex11scoped_lockERNS0_21scheduler_thread_infoERKSt10error_code+0x389) [0x558e132479f9]
mongo1            |  mongod(_ZN4asio6detail9scheduler3runERSt10error_code+0xD1) [0x558e13247c41]
mongo1            |  mongod(+0x10664AD) [0x558e1254b4ad]
mongo1            |  mongod(+0x2320FD0) [0x558e13805fd0]
mongo1            |  libpthread.so.0(+0x8064) [0x7fccace44064]
mongo1            |  libc.so.6(clone+0x6D) [0x7fccacb7962d]
mongo1            | -----  END BACKTRACE  -----

       

      The error happens after a variable amount of time, independently from the activity happening in the replica set.

       

            Assignee:
            blake.oler@mongodb.com Blake Oler
            Reporter:
            pachico Mariano Benítez Mulet
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated:
              Resolved: