Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-1266

libmongoclient should not link with xulrunner's libmozjs, causes subtle memory errors

    • Type: Icon: Improvement Improvement
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 1.5.7
    • Affects Version/s: 1.5.3
    • Component/s: Internal Client
    • Labels:
      None
    • Environment:
      Ubuntu 10.04 x86_64

      An empty mainline, when linked with libmongoclient, then run under valgrind, reports a bad free at shutdown in __libc_freeres processing:

      mongo_libc_freeres$ cat ./main.cc
      #include <cstdio>

      int main(int argc, char* argv[])
      {
      std::printf("Hello, World\n");
      return 0;
      }

      mongo_libc_freeres$ g++ -ggdb ./main.cc -L /mongo.temp.install/lib64 -Wl,-rpath,/mongo.temp.install/lib64 -lmongoclient

      mongo_libc_freeres$ ls -la ./a.out
      -rwxrwxr-x 1 xxx xxx 12666 2010-06-21 12:08 ./a.out

      mongo_libc_freeres$ ldd ./a.out
      linux-vdso.so.1 => (0x00007fffc50ce000)
      libmongoclient.so.9.9.9 => ~/mongo.temp.install/lib64/libmongoclient.so.9.9.9 (0x00007f20cb1cb000)
      libstdc+.so.6 => /usr/lib/libstdc+.so.6 (0x00007f20cae97000)
      libm.so.6 => /lib/libm.so.6 (0x00007f20cac13000)
      libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f20ca9fc000)
      libc.so.6 => /lib/libc.so.6 (0x00007f20ca67a000)
      libpthread.so.0 => /lib/libpthread.so.0 (0x00007f20ca45c000)
      libboost_system.so.1.40.0 => /usr/lib/libboost_system.so.1.40.0 (0x00007f20ca258000)
      libboost_thread.so.1.40.0 => /usr/lib/libboost_thread.so.1.40.0 (0x00007f20ca042000)
      libboost_filesystem.so.1.40.0 => /usr/lib/libboost_filesystem.so.1.40.0 (0x00007f20c9e2c000)
      libboost_program_options.so.1.40.0 => /usr/lib/libboost_program_options.so.1.40.0 (0x00007f20c9bde000)
      libpcrecpp.so.0 => /usr/lib/libpcrecpp.so.0 (0x00007f20c99d5000)
      libpcre.so.3 => /lib/libpcre.so.3 (0x00007f20c97a6000)
      libmozjs.so => /usr/lib64/xulrunner-1.9.2.3/libmozjs.so (0x00007f20c948d000)
      /lib64/ld-linux-x86-64.so.2 (0x00007f20cb6a3000)
      librt.so.1 => /lib/librt.so.1 (0x00007f20c9284000)
      libplds4.so => /usr/lib/libplds4.so (0x00007f20c9080000)
      libplc4.so => /usr/lib/libplc4.so (0x00007f20c8e7a000)
      libnspr4.so => /usr/lib/libnspr4.so (0x00007f20c8c3e000)
      libdl.so.2 => /lib/libdl.so.2 (0x00007f20c8a3a000)

      mongo_libc_freeres$ valgrind --track-origins=yes --read-var-info=yes --leak-check=full ./a.out
      ==29614== Memcheck, a memory error detector
      ==29614== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
      ==29614== Using Valgrind-3.6.0.SVN and LibVEX; rerun with -h for copyright info
      ==29614== Command: ./a.out
      ==29614==
      Hello, World
      ==29614== Invalid free() / delete / delete[]
      ==29614== at 0x4C28D5E: free (vg_replace_malloc.c:381)
      ==29614== by 0x5BE404A: free_mem (in /lib/libc-2.11.1.so)
      ==29614== by 0x5BE3BE1: __libc_freeres (in /lib/libc-2.11.1.so)
      ==29614== by 0x4A236AB: _vgnU_freeres (vg_preloaded.c:62)
      ==29614== by 0x5AEC214: exit (exit.c:93)
      ==29614== by 0x5AD1C53: (below main) (libc-start.c:258)
      ==29614== Address 0x4043e88 is not stack'd, malloc'd or (recently) free'd
      ==29614==
      ==29614==
      ==29614== HEAP SUMMARY:
      ==29614== in use at exit: 10,512 bytes in 25 blocks
      ==29614== total heap usage: 421 allocs, 397 frees, 40,988 bytes allocated
      ==29614==
      ==29614== LEAK SUMMARY:
      ==29614== definitely lost: 0 bytes in 0 blocks
      ==29614== indirectly lost: 0 bytes in 0 blocks
      ==29614== possibly lost: 0 bytes in 0 blocks
      ==29614== still reachable: 10,512 bytes in 25 blocks
      ==29614== suppressed: 0 bytes in 0 blocks
      ==29614== Reachable blocks (those to which a pointer was found) are not shown.
      ==29614== To see them, rerun with: --leak-check=full --show-reachable=yes
      ==29614==
      ==29614== For counts of detected and suppressed errors, rerun with: -v
      ==29614== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 6 from 6)

      If we manually re-link libmongoclient.so without mozjs, the problem goes away (note lack of -lmozjs, and -L and -Wl,rpath arguments pointing to /usr/lib64/xulrunner-w.x.y.z:

      mongo$ g++ -o libmongoclient.so.9.9.9 -fPIC -pthread -rdynamic -Wl,-soname,libmongoclient.so.9.9.9 -shared pch.os buildinfo.os db/common.os db/jsobj.os db/json.os db/lasterror.os db/nonce.os db/queryutil.os shell/mongo.os util/background.os util/mmap.os util/ramstore.os util/sock.os util/util.os util/message.os util/assert_util.os util/httpclient.os util/md5main.os util/base64.os util/concurrency/vars.os util/concurrency/task.os util/debug_util.os util/concurrency/thread_pool.os util/password.os util/version.os util/histogram.os util/concurrency/spin_lock.os util/text.os util/md5.os client/connpool.os client/dbclient.os client/dbclientcursor.os client/model.os client/syncclusterconnection.os s/shardconnection.os util/mmap_posix.os util/processinfo_linux2.os client/clientOnly.os client/gridfs.os s/d_util.os -L/usr/lib64 -L/lib64 -lpthread -lstdc++ -lboost_system-mt -lboost_thread-mt -lboost_filesystem-mt -lboost_program_options-mt -lpcrecpp -lpcre

      mongo_libc_freeres$ ldd ./a.out
      linux-vdso.so.1 => (0x00007fff3693c000)
      libmongoclient.so.9.9.9 => ~/mongo.temp.install/lib64/libmongoclient.so.9.9.9 (0x00007f68fd175000)
      libstdc+.so.6 => /usr/lib/libstdc+.so.6 (0x00007f68fce41000)
      libm.so.6 => /lib/libm.so.6 (0x00007f68fcbbd000)
      libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f68fc9a6000)
      libc.so.6 => /lib/libc.so.6 (0x00007f68fc624000)
      libpthread.so.0 => /lib/libpthread.so.0 (0x00007f68fc406000)
      libboost_system.so.1.40.0 => /usr/lib/libboost_system.so.1.40.0 (0x00007f68fc202000)
      libboost_thread.so.1.40.0 => /usr/lib/libboost_thread.so.1.40.0 (0x00007f68fbfec000)
      libboost_filesystem.so.1.40.0 => /usr/lib/libboost_filesystem.so.1.40.0 (0x00007f68fbdd6000)
      libboost_program_options.so.1.40.0 => /usr/lib/libboost_program_options.so.1.40.0 (0x00007f68fbb88000)
      libpcrecpp.so.0 => /usr/lib/libpcrecpp.so.0 (0x00007f68fb97f000)
      libpcre.so.3 => /lib/libpcre.so.3 (0x00007f68fb750000)
      /lib64/ld-linux-x86-64.so.2 (0x00007f68fd64d000)
      librt.so.1 => /lib/librt.so.1 (0x00007f68fb548000)

      (note that there is no link to libmozjs listed above)

      Re-running valgrind:

      mongo_libc_freeres$ valgrind --track-origins=yes --read-var-info=yes --leak-check=full ./a.out
      ==29841== Memcheck, a memory error detector
      ==29841== Copyright (C) 2002-2010, and GNU GPL'd, by Julian Seward et al.
      ==29841== Using Valgrind-3.6.0.SVN and LibVEX; rerun with -h for copyright info
      ==29841== Command: ./a.out
      ==29841==
      Hello, World
      ==29841==
      ==29841== HEAP SUMMARY:
      ==29841== in use at exit: 10,512 bytes in 25 blocks
      ==29841== total heap usage: 421 allocs, 396 frees, 40,988 bytes allocated
      ==29841==
      ==29841== LEAK SUMMARY:
      ==29841== definitely lost: 0 bytes in 0 blocks
      ==29841== indirectly lost: 0 bytes in 0 blocks
      ==29841== possibly lost: 0 bytes in 0 blocks
      ==29841== still reachable: 10,512 bytes in 25 blocks
      ==29841== suppressed: 0 bytes in 0 blocks
      ==29841== Reachable blocks (those to which a pointer was found) are not shown.
      ==29841== To see them, rerun with: --leak-check=full --show-reachable=yes
      ==29841==
      ==29841== For counts of detected and suppressed errors, rerun with: -v
      ==29841== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 4 from 4)

      And our error goes away. Its pretty clear that libmongoclient doesn't actually depend on mozjs.so (since it linked), and forcing all clients that link against libmongoclient to pick up such a dependency seems to cause memory errors. It would be good if libmongoclient would drop this dependency.

      FWIW, libmongoclient links, on my system, the following libraries:

      -lpthread -lstdc++ -lboost_system-mt -lboost_thread-mt -lboost_filesystem-mt -lboost_program_options-mt -lpcrecpp -lpcre

      Many of these are not necessary:

      The explicit -lpthread is not needed since the g++ driver is being called with the -pthread option. Its better to let the compiler driver worry about where to put things like -lpthread and -lc on the link line.
      The -lstdc++ is not necessary since that is included by default by the g++ driver when linking
      The -lboost_program_options_mt library is not needed since the client library does not do any options parsing
      The library does not appear to depend on -lpcrecpp -lpcre either.

      I verified that all of these can be removed by manually linking the library with -Wl,-zdefs added to the link line and removing each of the libraries, one by one. The final link line looked like:

      g++ -o libmongoclient.so.9.9.9 -fPIC -pthread -rdynamic -Wl,-zdefs -Wl,-soname,libmongoclient.so.9.9.9 -shared pch.os buildinfo.os db/common.os db/jsobj.os db/json.os db/lasterror.os db/nonce.os db/queryutil.os shell/mongo.os util/background.os util/mmap.os util/ramstore.os util/sock.os util/util.os util/message.os util/assert_util.os util/httpclient.os util/md5main.os util/base64.os util/concurrency/vars.os util/concurrency/task.os util/debug_util.os util/concurrency/thread_pool.os util/password.os util/version.os util/histogram.os util/concurrency/spin_lock.os util/text.os util/md5.os client/connpool.os client/dbclient.os client/dbclientcursor.os client/model.os client/syncclusterconnection.os s/shardconnection.os util/mmap_posix.os util/processinfo_linux2.os db/commands.os client/clientOnly.os client/gridfs.os s/d_util.os -L/usr/lib64 -L/lib64 -lboost_system-mt -lboost_thread-mt -lboost_filesystem-mt

      After this, my test mainline still runs cleanly, even with LD_BIND_NOW in the environment:

      mongo_libc_freeres$ LD_BIND_NOW=1 ./a.out
      Hello, World

      And still shows no errors under valgrind.

            Assignee:
            mike Michael Dirolf
            Reporter:
            andrew.morrow@mongodb.com Andrew Morrow (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

              Created:
              Updated:
              Resolved: