Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-7028

ReplSetConfig constructor using DBDirectClient can trigger page faults

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 2.3.0
    • None
    • Replication
    • ALL

    Description

      The ReplSetConfig constructor can perform database reads using a DBDirectClient:

                      // on startup, socket is not listening yet
                      DBDirectClient cli;
                      cfg = cli.findOne( rsConfigNs, Query() ).getOwned();
                      count = cli.count(rsConfigNs);

      This can trigger a PageFaultException (potentially from a simulated page fault in a debug build), which can trigger an abort in a debug build where constructor stack auditing on page faults is enabled.

      – Aaron

      ------------------------------------------------------------

      Buildbot test on Linux 64-bit debug dur off failed in jstests/replsets/fastsync.js:

      http://buildlogs.mongodb.org/Linux%2064-bit%20debug%20dur%20off/builds/1616/test/fastsync.js
      http://buildbot.mongodb.org/builders/Linux%2064-bit%20debug%20dur%20off/builds/1616/steps/test_8/logs/stdio

      Wed Sep 12 16:54:58 shell: started program /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod --port 31002 --dbpath /data/db/jstests_fastsync-s2 --nohttpinterface --noprealloc --smallfiles --replSet jstests_fastsync --fastsync --oplogSize 2
      m31002| note: noprealloc may hurt performance in many applications
      m31002| Wed Sep 12 16:54:58 [initandlisten] MongoDB starting : pid=29021 port=31002 dbpath=/data/db/jstests_fastsync-s2 64-bit host=tp1.10gen.cc
      m31002| Wed Sep 12 16:54:58 [initandlisten] _DEBUG build (which is slower)
      m31002| Wed Sep 12 16:54:58 [initandlisten] 
      m31002| Wed Sep 12 16:54:58 [initandlisten] ** NOTE: This is a development version (2.3.0-pre-) of MongoDB.
      m31002| Wed Sep 12 16:54:58 [initandlisten] ** Not recommended for production.
      m31002| Wed Sep 12 16:54:58 [initandlisten] 
      m31002| Wed Sep 12 16:54:58 [initandlisten] db version v2.3.0-pre-, pdfile version 4.5
      m31002| Wed Sep 12 16:54:58 [initandlisten] git version: 291b2e263b2ea3d578f183da7a9187c88d498d98
      m31002| Wed Sep 12 16:54:58 [initandlisten] build info: Linux tp1.10gen.cc 2.6.31-14-server #48-Ubuntu SMP Fri Oct 16 15:07:34 UTC 2009 x86_64 BOOST_LIB_VERSION=1_49
      m31002| Wed Sep 12 16:54:58 [initandlisten] options: { dbpath: "/data/db/jstests_fastsync-s2", fastsync: true, nohttpinterface: true, noprealloc: true, oplogSize: 2, port: 31002, replSet: "jstests_fastsync", smallfiles: true }
      m31002| Wed Sep 12 16:54:58 [initandlisten] Unable to check for journal files due to: boost::filesystem::basic_directory_iterator constructor: No such file or directory: "/data/db/jstests_fastsync-s2/journal"
      m31002| Wed Sep 12 16:54:58 [initandlisten] info PageFaultRetryableSection will not yield, already locked upon reaching
      m31002| Wed Sep 12 16:54:58 [initandlisten] _DEBUG ReadContext db wasn't open, will try to open local.system.namespaces
      m31002| Wed Sep 12 16:54:58 [initandlisten] opening db: local
      m31002| Wed Sep 12 16:54:58 [initandlisten] _DEBUG ReadContext db wasn't open, will try to open admin.system.namespaces
      m31002| Wed Sep 12 16:54:58 [initandlisten] opening db: admin
      m31002| Wed Sep 12 16:54:58 [initandlisten] _DEBUG ReadContext db wasn't open, will try to open foo.system.namespaces
      m31002| Wed Sep 12 16:54:58 [initandlisten] opening db: foo
      m31002| Wed Sep 12 16:54:58 [initandlisten] waiting for connections on port 31002
      m31002| Wed Sep 12 16:54:58 [initandlisten] connection accepted from 174.121.165.194:46575 #1 (1 connection now open)
      m31002| Wed Sep 12 16:54:58 [conn1] _DEBUG ReadContext db wasn't open, will try to open local.system.replset
      m31002| Wed Sep 12 16:54:58 [conn1] opening db: local
      m31002| Wed Sep 12 16:54:58 [rsStart] replSet load config ok from self
      m31002| Wed Sep 12 16:54:58 [rsStart] replSet REMOVED
      m31002| Wed Sep 12 16:54:58 [rsStart] replSet info self not present in the repl set configuration:
      m31002| Wed Sep 12 16:54:58 [rsStart] { _id: "jstests_fastsync", version: 2, members: [ { _id: 0, host: "tp1.10gen.cc:31000", priority: 2.0 }, { _id: 1, host: "tp1.10gen.cc:31001" } ] }
      m31002| Wed Sep 12 16:54:58 [initandlisten] connection accepted from 127.0.0.1:49570 #2 (2 connections now open)
      m31000| Wed Sep 12 16:54:59 [conn2] replSet replSetReconfig config object parses ok, 3 members specified
      m31002| found a constructor in the call tree: mongo::ReplSetConfig::ReplSetConfig(mongo::HostAndPort const&)
      m31002| _ZN5mongo13ReplSetConfigC1ERKNS_11HostAndPortE
      m31002| Wed Sep 12 16:54:59 [rsStart] local.system.replset Fatal Assertion 16236
      m31002| 0xd24f97 0xd02718 0xcf0d44 0x974ec3 0xbb6042 0x8deae8 0x8df1f9 0x9ecd87 0xb0b12f 0xb0b3a0 0xb0c694 0xb0d6a3 0xb0fb8b 0xa91751 0xa927e7 0xa95b69 0x926cd3 0x8fe2d9 0xa95d54 0x8fca6f 
      m31002| Wed Sep 12 16:54:59 [initandlisten] connection accepted from 174.121.165.194:46577 #3 (3 connections now open)
      m31000| Wed Sep 12 16:54:59 [conn2] replSet replSetReconfig [2]
      m31000| Wed Sep 12 16:54:59 [conn2] replSet info saving a newer config version to local.system.replset
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x27) [0xd24f97]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo10logContextEPKc+0x5e) [0xd02718]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo13fassertFailedEi+0x65) [0xcf0d44]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo7fassertEib+0x25) [0x974ec3]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x11c) [0xbb6042]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo6Record4dataEv+0x18) [0x8deae8]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo7BSONObj4makeEPKNS_6RecordE+0x21) [0x8df1f9]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo11BasicCursor7currentEv+0x3d) [0x9ecd87]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo21ResponseBuildStrategy7currentEb+0x229) [0xb0b12f]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo20OrderedBuildStrategy11handleMatchERbRNS_12MatchDetailsE+0x114) [0xb0b3a0]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo20QueryResponseBuilder8addMatchEv+0xd6) [0xb0c694]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_17ShardChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE+0x49d) [0xb0d6a3]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x113b) [0xb0fb8b]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod [0xa91751]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x33b) [0xa927e7]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x91) [0xa95b69]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo14DBClientCursor4initEv+0xdd) [0x926cd3]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0xe9) [0x8fe2d9]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo14DBDirectClient5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii+0x6e) [0xa95d54]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo17DBClientInterface5findNERSt6vectorINS_7BSONObjESaIS2_EERKSsNS_5QueryEiiPKS2_i+0xcb) [0x8fca6f]
      m31002| Wed Sep 12 16:54:59 [conn1] end connection 174.121.165.194:46575 (2 connections now open)
      m31002| Wed Sep 12 16:54:59 [rsStart] 
      m31002| 
      m31002| ***aborting after fassert() failure
      m31002| 
      m31002| 
      m31002| Wed Sep 12 16:54:59 Got signal: 6 (Aborted).
      m31002| 
      m31002| Wed Sep 12 16:54:59 [conn1] thread conn1 stack usage was 7456 bytes, which is the most so far
      m31002| Wed Sep 12 16:54:59 Backtrace:
      m31002| 0xd24f97 0x8b9d82 0x7fb7884e2530 0x7fb7884e24b5 0x7fb7884e5f50 0xcf0d85 0x974ec3 0xbb6042 0x8deae8 0x8df1f9 0x9ecd87 0xb0b12f 0xb0b3a0 0xb0c694 0xb0d6a3 0xb0fb8b 0xa91751 0xa927e7 0xa95b69 0x926cd3 
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo15printStackTraceERSo+0x27) [0xd24f97]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo10abruptQuitEi+0x130) [0x8b9d82]
      m31002| /lib/libc.so.6 [0x7fb7884e2530]
      m31002| /lib/libc.so.6(gsignal+0x35) [0x7fb7884e24b5]
      m31002| /lib/libc.so.6(abort+0x180) [0x7fb7884e5f50]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo9uassertedEiRKSs+0) [0xcf0d85]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo7fassertEib+0x25) [0x974ec3]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo6Record10_accessingEv+0x11c) [0xbb6042]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo6Record4dataEv+0x18) [0x8deae8]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo7BSONObj4makeEPKNS_6RecordE+0x21) [0x8df1f9]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo11BasicCursor7currentEv+0x3d) [0x9ecd87]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZNK5mongo21ResponseBuildStrategy7currentEb+0x229) [0xb0b12f]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo20OrderedBuildStrategy11handleMatchERbRNS_12MatchDetailsE+0x114) [0xb0b3a0]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo20QueryResponseBuilder8addMatchEv+0xd6) [0xb0c694]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo23queryWithQueryOptimizerEiRKSsRKNS_7BSONObjERNS_5CurOpES4_S4_RKN5boost10shared_ptrINS_11ParsedQueryEEES4_RKNS_17ShardChunkVersionERNS7_10scoped_ptrINS_25PageFaultRetryableSectionEEERNSG_INS_19NoPageFaultsAllowedEEERNS_7MessageE+0x49d) [0xb0d6a3]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1_+0x113b) [0xb0fb8b]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod [0xa91751]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_11HostAndPortE+0x33b) [0xa927e7]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo14DBDirectClient4callERNS_7MessageES2_bPSs+0x91) [0xa95b69]
      m31002| /home/yellow/buildslave/Linux_64bit_debug_dur_off/mongo/mongod(_ZN5mongo14DBClientCursor4initEv+0xdd) [0x926cd3]
      m31002| 

      Attachments

        Activity

          People

            aaron Aaron Staple
            tad Tad Marshall
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: