Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5440

fd leak from opened and unclosed conns in webServerThread

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: 2.0.3
    • Component/s: None
    • Labels:
      None
    • ALL

      Apparently we're bad at closing connections somewhere in the web server thread. Valgrind --track-fds output from user with the problem:

      ==10784== Nulgrind, the minimal Valgrind tool
==10784== Copyright (C) 2002-2009, and GNU GPL'd, by Nicholas Nethercote.
==10784== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==10784== Command: /usr/bin/mongod --quiet -f /etc/mongodb.conf
==10784== Parent PID: 11981
==10784== 
==10784== 
==10784== FILE DESCRIPTORS: 5 open at exit.
==10784== Open file descriptor 4: /dev/urandom
==10784==    at 0x32D8CC5B40: __open_nocancel (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C6C2F2: _IO_file_open (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C6C43B: _IO_file_fopen@@GLIBC_2.2.5 (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C61763: __fopen_internal (in /lib64/libc-2.5.so)
==10784==    by 0x32DA0BBC33: std::__basic_file<char>::open(char const*, std::_Ios_Openmode, int) (in /usr/lib64/libstdc++.so.6.0.8)
==10784==    by 0x32DA06A522: std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.8)
==10784==    by 0x544F8C: mongo::Security::init() (in /usr/bin/mongod)
==10784==    by 0x54526B: ??? (in /usr/bin/mongod)
==10784==    by 0xACDCC5: ??? (in /usr/bin/mongod)
==10784==    by 0x4FFBD2: ??? (in /usr/bin/mongod)
==10784== 
==10784== Open file descriptor 3: /root/valgrind.log
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 2: /dev/pts/0
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 1: /dev/pts/0
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 0: /dev/pts/0
==10784==    <inherited from parent>
==10784== 
==10784== 
==10784== 
==10784== FILE DESCRIPTORS: 5 open at exit.
==10784== Open file descriptor 4: /dev/urandom
==10784==    at 0x32D8CC5B40: __open_nocancel (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C6C2F2: _IO_file_open (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C6C43B: _IO_file_fopen@@GLIBC_2.2.5 (in /lib64/libc-2.5.so)
==10784==    by 0x32D8C61763: __fopen_internal (in /lib64/libc-2.5.so)
==10784==    by 0x32DA0BBC33: std::__basic_file<char>::open(char const*, std::_Ios_Openmode, int) (in /usr/lib64/libstdc++.so.6.0.8)
==10784==    by 0x32DA06A522: std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.8)
==10784==    by 0x544F8C: mongo::Security::init() (in /usr/bin/mongod)
==10784==    by 0x54526B: ??? (in /usr/bin/mongod)
==10784==    by 0xACDCC5: ??? (in /usr/bin/mongod)
==10784==    by 0x4FFBD2: ??? (in /usr/bin/mongod)
==10784== 
==10784== Open file descriptor 3: /root/valgrind.log
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 2: /dev/pts/0
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 1: /dev/pts/0
==10784==    <inherited from parent>
==10784== 
==10784== Open file descriptor 0: /dev/pts/0==9895== 
==9895== FILE DESCRIPTORS: 14 open at exit.
==9895== Open AF_INET socket 51: 10.1.101.211:39569 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 190: 10.1.101.211:50059 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 192: 10.1.101.211:47191 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 39: 10.1.101.211:35835 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 95: 10.1.101.211:46725 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 194: 10.1.101.211:42425 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 189: 10.1.101.211:57489 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open AF_INET socket 183: 10.1.101.211:42136 <-> unbound
==9895==    at 0x32D940DB3B: ??? (in /lib64/libpthread-2.5.so)
==9895==    by 0x5AB50E: mongo::Listener::initAndListen() (in /usr/bin/mongod)
==9895==    by 0x65459D: mongo::webServerThread(mongo::AdminAccess const*) (in /usr/bin/mongod)
==9895==    by 0xAAB7BF: thread_proxy (in /usr/bin/mongod)
==9895==    by 0x32D940673C: start_thread (in /lib64/libpthread-2.5.so)
==9895==    by 0x32D8CD3F6C: clone (in /lib64/libc-2.5.so)
==9895== 
==9895== Open file descriptor 5: /mongodb/mongod.lock
==9895==    at 0x32D940E32F: open64 (in /lib64/libpthread-2.5.so)
==9895==    by 0x88FD2E: mongo::acquirePathLock(bool) (in /usr/bin/mongod)
==9895==    by 0xA94B91: mongo::_initAndListen(int) (in /usr/bin/mongod)
==9895==    by 0xA9558C: mongo::initAndListen(int) (in /usr/bin/mongod)
==9895==    by 0xA9F2B9: main (in /usr/bin/mongod)
==9895== 
==9895== Open file descriptor 1: /var/log/mongodb/mongodb.log
==9895==    at 0x32D8CC6AC7: dup2 (in /lib64/libc-2.5.so)
==9895==    by 0x586D3B: mongo::LoggingManager::rotate() (in /usr/bin/mongod)
==9895==    by 0x58718D: mongo::LoggingManager::start(std::string const&, bool) (in /usr/bin/mongod)
==9895==    by 0x8F8D91: mongo::CmdLine::store(int, char**, boost::program_options::options_description&, boost::program_options::options_description&, boost::program_options::positional_options_description&, boost::program_options::variables_map&) (in /usr/bin/mongod)
==9895==    by 0xA965C3: main (in /usr/bin/mongod)
==9895== 
==9895== Open file descriptor 2: /var/log/mongodb/mongodb.log
==9895==    at 0x32D8CC6AC7: dup2 (in /lib64/libc-2.5.so)
==9895==    by 0x586D4F: mongo::LoggingManager::rotate() (in /usr/bin/mongod)
==9895==    by 0x58718D: mongo::LoggingManager::start(std::string const&, bool) (in /usr/bin/mongod)
==9895==    by 0x8F8D91: mongo::CmdLine::store(int, char**, boost::program_options::options_description&, boost::program_options::options_description&, boost::program_options::positional_options_description&, boost::program_options::variables_map&) (in /usr/bin/mongod)
==9895==    by 0xA965C3: main (in /usr/bin/mongod)
==9895== 
==9895== Open file descriptor 0: /dev/null
==9895==    at 0x32D8CC5B7B: ??? (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C6C2F2: _IO_file_open (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C6C43B: _IO_file_fopen@@GLIBC_2.2.5 (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C69FA3: freopen64 (in /lib64/libc-2.5.so)
==9895==    by 0x8F8B5A: mongo::CmdLine::store(int, char**, boost::program_options::options_description&, boost::program_options::options_description&, boost::program_options::positional_options_description&, boost::program_options::variables_map&) (in /usr/bin/mongod)
==9895==    by 0xA965C3: main (in /usr/bin/mongod)
==9895== 
==9895== Open file descriptor 4: /dev/urandom
==9895==    at 0x32D8CC5B40: __open_nocancel (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C6C2F2: _IO_file_open (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C6C43B: _IO_file_fopen@@GLIBC_2.2.5 (in /lib64/libc-2.5.so)
==9895==    by 0x32D8C61763: __fopen_internal (in /lib64/libc-2.5.so)
==9895==    by 0x32DA0BBC33: std::__basic_file<char>::open(char const*, std::_Ios_Openmode, int) (in /usr/lib64/libstdc++.so.6.0.8)
==9895==    by 0x32DA06A522: std::basic_filebuf<char, std::char_traits<char> >::open(char const*, std::_Ios_Openmode) (in /usr/lib64/libstdc++.so.6.0.8)
==9895==    by 0x544F8C: mongo::Security::init() (in /usr/bin/mongod)
==9895==    by 0x54526B: ??? (in /usr/bin/mongod)
==9895==    by 0xACDCC5: ??? (in /usr/bin/mongod)
==9895==    by 0x4FFBD2: ??? (in /usr/bin/mongod)
==9895== 
==9895== Open file descriptor 3: /root/valgrind.log
==9895==    <inherited from parent>

      this results in lots of orphaned fds:

      none-amd6 9898 root  194u     sock                0,5            61517133 can't identify protocol

            Assignee:
            Unassigned Unassigned
            Reporter:
            greg_10gen Greg Studer
            Votes:
            1 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: