Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5074

server crash when geoNear result grows past 64 MB

    • Type: Icon: Bug Bug
    • Resolution: Done
    • Priority: Icon: Critical - P2 Critical - P2
    • None
    • Affects Version/s: 2.0.2
    • Component/s: Stability
    • Labels:
      None
    • Environment:
      64 bit Ubuntu 11.04
      64 bit Mongodb 2.0.2
    • ALL

      We migrated our mongo instance to a sharded setup and because of that had to convert our $near queries to geoNear. Some of our queries return a lot of data and that crashes our db instances. Currently there is no way to skip records (documented in https://jira.mongodb.org/browse/SERVER-3925) so we don't have a good solution for getting the data we need while being restricted to geoNear.

      How to reproduce

      > use foo;
      > db.bar.ensureIndex(

      {loc:"2d"}

      );
      > for(i = 0; i < 2000000; i++) { db.bar.insert(

      { loc:[Math.random() * 100, Math.random() * 100] }

      ) };
      > db.runCommand(

      { geoNear : "bar" , near : [50,50], num : 2000000 }

      );

      Console error:

      Fri Feb 24 10:55:03 DBClientCursor::init call() failed
      Fri Feb 24 10:55:03 query failed : foo.$cmd

      { geoNear: "bar", near: [ 50.0, 50.0 ], num: 2000000.0 }

      to: 127.0.0.1
      Fri Feb 24 10:55:03 Error: error doing query: failed shell/collection.js:151
      Fri Feb 24 10:55:03 trying reconnect to 127.0.0.1
      Fri Feb 24 10:55:03 reconnect 127.0.0.1 ok
      Fri Feb 24 10:55:03 Socket recv() errno:104 Connection reset by peer 127.0.0.1:27017
      Fri Feb 24 10:55:03 SocketException: remote: 127.0.0.1:27017 error: 9001 socket exception [1] server [127.0.0.1:27017]
      Fri Feb 24 10:55:03 DBClientCursor::init call() failed

      Log error:

      Fri Feb 24 10:55:03 [conn1] Assertion: 13548:BufBuilder grow() > 64MB
      0x555758 0x4fb155 0x4fb1a3 0x72cf45 0x734f41 0x7356e2 0x616700 0x6190c0 0x6d4f9e 0x7c7276 0x7ff08db93ba9 0x7ff08e2b2efc 0x7ff08c5e589d
      /usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x138) [0x555758]
      /usr/bin/mongod(_ZN5mongo10BufBuilder15grow_reallocateEv+0x45) [0x4fb155]
      /usr/bin/mongod(_ZN5mongo10BufBuilder4growEi+0x23) [0x4fb1a3]
      /usr/bin/mongod(_ZN5mongo16Geo2dFindNearCmd3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0xcb5) [0x72cf45]
      /usr/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6b1) [0x734f41]
      /usr/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x332) [0x7356e2]
      /usr/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x30) [0x616700]
      /usr/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0xf50) [0x6190c0]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0xf5e) [0x6d4f9e]
      /usr/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x1e6) [0x7c7276]
      /usr/lib/libboost_thread.so.1.46.1(thread_proxy+0x69) [0x7ff08db93ba9]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc) [0x7ff08e2b2efc]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff08c5e589d]
      Fri Feb 24 10:55:03 [conn1] Assertion: 13548:BufBuilder grow() > 64MB
      0x555758 0x4fb155 0x4fb1a3 0x4f8f00 0x4fea8c 0x72da8d 0x734f41 0x7356e2 0x616700 0x6190c0 0x6d4f9e 0x7c7276 0x7ff08db93ba9 0x7ff08e2b2efc 0x7ff08c5e589d
      /usr/bin/mongod(_ZN5mongo11msgassertedEiPKc+0x138) [0x555758]
      /usr/bin/mongod(_ZN5mongo10BufBuilder15grow_reallocateEv+0x45) [0x4fb155]
      /usr/bin/mongod(_ZN5mongo10BufBuilder4growEi+0x23) [0x4fb1a3]
      /usr/bin/mongod() [0x4f8f00]
      /usr/bin/mongod(_ZN5mongo14BSONObjBuilderD1Ev+0x2c) [0x4fea8c]
      /usr/bin/mongod(_ZN5mongo16Geo2dFindNearCmd3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x17fd) [0x72da8d]
      /usr/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6b1) [0x734f41]
      /usr/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x332) [0x7356e2]
      /usr/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x30) [0x616700]
      /usr/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0xf50) [0x6190c0]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0xf5e) [0x6d4f9e]
      /usr/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x1e6) [0x7c7276]
      /usr/lib/libboost_thread.so.1.46.1(thread_proxy+0x69) [0x7ff08db93ba9]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc) [0x7ff08e2b2efc]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff08c5e589d]
      Fri Feb 24 10:55:03 terminate() called, printing stack:
      0x7cc43f 0x7c4c4a 0x7ff08e05df26 0x7ff08e05cf59 0x7ff08e05d84d 0x7ff08c8b26fb 0x7ff08c8b2b58 0x4feaeb 0x72da8d 0x734f41 0x7356e2 0x616700 0x6190c0 0x6d4f9e 0x7c7276 0x7ff08db93ba9 0x7ff08e2b2efc 0x7ff08c5e589d
      /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x1f) [0x7cc43f]
      /usr/bin/mongod(_ZN5mongo11myterminateEv+0x3a) [0x7c4c4a]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb9f26) [0x7ff08e05df26]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8f59) [0x7ff08e05cf59]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__gxx_personality_v0+0x22d) [0x7ff08e05d84d]
      /lib/x86_64-linux-gnu/libgcc_s.so.1(+0xf6fb) [0x7ff08c8b26fb]
      /lib/x86_64-linux-gnu/libgcc_s.so.1(_Unwind_Resume+0x68) [0x7ff08c8b2b58]
      /usr/bin/mongod(_ZN5mongo14BSONObjBuilderD1Ev+0x8b) [0x4feaeb]
      /usr/bin/mongod(_ZN5mongo16Geo2dFindNearCmd3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x17fd) [0x72da8d]
      /usr/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6b1) [0x734f41]
      /usr/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x332) [0x7356e2]
      /usr/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x30) [0x616700]
      /usr/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0xf50) [0x6190c0]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0xf5e) [0x6d4f9e]
      /usr/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x1e6) [0x7c7276]
      /usr/lib/libboost_thread.so.1.46.1(thread_proxy+0x69) [0x7ff08db93ba9]
      /lib/x86_64-linux-gnu/libpthread.so.0(+0x7efc) [0x7ff08e2b2efc]
      /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7ff08c5e589d]
      Fri Feb 24 10:55:03 Got signal: 6 (Aborted).

      Fri Feb 24 10:55:03 Backtrace:
      0x7cc43f 0x7c61cc 0x7ff08c53a420 0x7ff08c53a3a5 0x7ff08c53db0b 0x7c4c4f 0x7ff08e05df26 0x7ff08e05cf59 0x7ff08e05d84d 0x7ff08c8b26fb 0x7ff08c8b2b58 0x4feaeb 0x72da8d 0x734f41 0x7356e2 0x616700 0x6190c0 0x6d4f9e 0x7c7276 0x7ff08db93ba9
      /usr/bin/mongod(_ZN5mongo15printStackTraceERSo+0x1f) [0x7cc43f]
      /usr/bin/mongod(_ZN5mongo10abruptQuitEi+0x34c) [0x7c61cc]
      /lib/x86_64-linux-gnu/libc.so.6(+0x36420) [0x7ff08c53a420]
      /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x35) [0x7ff08c53a3a5]
      /lib/x86_64-linux-gnu/libc.so.6(abort+0x17b) [0x7ff08c53db0b]
      /usr/bin/mongod(_ZN5mongo11myterminateEv+0x3f) [0x7c4c4f]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb9f26) [0x7ff08e05df26]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8f59) [0x7ff08e05cf59]
      /usr/lib/x86_64-linux-gnu/libstdc++.so.6(__gxx_personality_v0+0x22d) [0x7ff08e05d84d]
      /lib/x86_64-linux-gnu/libgcc_s.so.1(+0xf6fb) [0x7ff08c8b26fb]
      /lib/x86_64-linux-gnu/libgcc_s.so.1(_Unwind_Resume+0x68) [0x7ff08c8b2b58]
      /usr/bin/mongod(_ZN5mongo14BSONObjBuilderD1Ev+0x8b) [0x4feaeb]
      /usr/bin/mongod(_ZN5mongo16Geo2dFindNearCmd3runERKSsRNS_7BSONObjERSsRNS_14BSONObjBuilderEb+0x17fd) [0x72da8d]
      /usr/bin/mongod(_ZN5mongo11execCommandEPNS_7CommandERNS_6ClientEiPKcRNS_7BSONObjERNS_14BSONObjBuilderEb+0x6b1) [0x734f41]
      /usr/bin/mongod(_ZN5mongo12_runCommandsEPKcRNS_7BSONObjERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x332) [0x7356e2]
      /usr/bin/mongod(_ZN5mongo11runCommandsEPKcRNS_7BSONObjERNS_5CurOpERNS_10BufBuilderERNS_14BSONObjBuilderEbi+0x30) [0x616700]
      /usr/bin/mongod(ZN5mongo8runQueryERNS_7MessageERNS_12QueryMessageERNS_5CurOpES1+0xf50) [0x6190c0]
      /usr/bin/mongod(_ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE+0xf5e) [0x6d4f9e]
      /usr/bin/mongod(_ZN5mongo10connThreadEPNS_13MessagingPortE+0x1e6) [0x7c7276]
      /usr/lib/libboost_thread.so.1.46.1(thread_proxy+0x69) [0x7ff08db93ba9]

            Assignee:
            randolph@mongodb.com Randolph Tan
            Reporter:
            jjshanks Joshua Shanks
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: