Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5081

getMostElectable can read freed memory

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 2.1.0
    • Fix Version/s: 2.0.4, 2.1.1
    • Component/s: Replication
    • Labels:
      None
    • Operating System:
      ALL

      Description

      If you run jstests/sharding/mongos_no_replica_set_refresh.js with Memcheck, you will get this:

       m31100| ==19834== Thread 15:
       m31100| ==19834== Invalid read of size 8
       m31100| ==19834==    at 0x303E86E860: std::_Rb_tree_increment(std::_Rb_tree_node_base*) (tree.cc:62)
       m31100| ==19834==    by 0x97EB15: mongo::ReplSetImpl::getMostElectable() (stl_tree.h:277)
       m31100| ==19834==    by 0x788992: mongo::Manager::checkElectableSet() (manager.cpp:97)
       m31100| ==19834==    by 0x789020: mongo::Manager::msgCheckNewState() (manager.cpp:171)
       m31100| ==19834==    by 0x6E269A: mongo::task::Server::doWork() (function_template.hpp:760)
       m31100| ==19834==    by 0x6E0D09: mongo::task::Task::run() (task.cpp:53)
       m31100| ==19834==    by 0x5CFDE8: mongo::BackgroundJob::jobBody(boost::shared_ptr<mongo::BackgroundJob::JobStatus>) (background.cpp:61)
       m31100| ==19834==    by 0x5D201A: boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf1<void, mongo::BackgroundJob, boost::shared_ptr<mongo::BackgroundJob::JobStatus> >, boost::_bi::list2<boost::_bi::value<mongo::BackgroundJob*>, boost::_bi::value<boost::shared_ptr<mongo::BackgroundJob::JobStatus> > > > >::run() (mem_fn_template.hpp:165)
       m31100| ==19834==    by 0x4E3AC38: thread_proxy (thread.cpp:121)
       m31100| ==19834==    by 0x3037007D8F: start_thread (pthread_create.c:309)
       m31100| ==19834==    by 0x3036CEF48C: clone (clone.S:115)
       m31100| ==19834==  Address 0x1ed87498 is 24 bytes inside a block of size 40 free'd
       m31100| ==19834==    at 0x4A06336: operator delete(void*) (vg_replace_malloc.c:457)
       m31100| ==19834==    by 0x78981F: std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::_M_erase(std::_Rb_tree_node<unsigned int>*) (new_allocator.h:98)
       m31100| ==19834==    by 0x789983: std::_Rb_tree<unsigned int, unsigned int, std::_Identity<unsigned int>, std::less<unsigned int>, std::allocator<unsigned int> >::erase(unsigned int const&) (stl_tree.h:799)
       m31100| ==19834==    by 0x97EBAF: mongo::ReplSetImpl::getMostElectable() (stl_set.h:528)
       m31100| ==19834==    by 0x788992: mongo::Manager::checkElectableSet() (manager.cpp:97)
       m31100| ==19834==    by 0x789020: mongo::Manager::msgCheckNewState() (manager.cpp:171)
       m31100| ==19834==    by 0x6E269A: mongo::task::Server::doWork() (function_template.hpp:760)
       m31100| ==19834==    by 0x6E0D09: mongo::task::Task::run() (task.cpp:53)
       m31100| ==19834==    by 0x5CFDE8: mongo::BackgroundJob::jobBody(boost::shared_ptr<mongo::BackgroundJob::JobStatus>) (background.cpp:61)
       m31100| ==19834==    by 0x5D201A: boost::detail::thread_data<boost::_bi::bind_t<void, boost::_mfi::mf1<void, mongo::BackgroundJob, boost::shared_ptr<mongo::BackgroundJob::JobStatus> >, boost::_bi::list2<boost::_bi::value<mongo::BackgroundJob*>, boost::_bi::value<boost::shared_ptr<mongo::BackgroundJob::JobStatus> > > > >::run() (mem_fn_template.hpp:165)
       m31100| ==19834==    by 0x4E3AC38: thread_proxy (thread.cpp:121)
       m31100| ==19834==    by 0x3037007D8F: start_thread (pthread_create.c:309)

      I actually ran into this on Windows first, then tried it with memcheck on Linux. On Windows, you get an assertion in the iterator code when you attempt to iterate over the freed memory in _electableSet – no memcheck required.
      This may indicate a thread safety problem, where we are reading a std::set (which is a red-black tree under the hood) in one thread, while another thread is adding/removing nodes and rebalancing the tree.

        Attachments

          Activity

            People

            Assignee:
            milkie Eric Milkie
            Reporter:
            milkie Eric Milkie
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: