<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 02:59:59 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-2445] Assertion failure in replica set + durability while killing and restarting servers</title>
                <link>https://jira.mongodb.org/browse/SERVER-2445</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;I have a test case that&lt;br/&gt;
1. Starts up a replica set with 2 servers and 1 arbiter, all with --dur option.&lt;br/&gt;
2. Loads 10000 1K docs into a collection&lt;br/&gt;
3. Forks 5 client threads, each $pushes then $pulls its own id to/from an array in all document (multi-update)&lt;br/&gt;
5. A 6th thread kills a random server in the replica set every 0-60 secs then restarts it 0-30 secs later.&lt;/p&gt;

&lt;p&gt;After a few minutes I get the error below in one of the servers.&lt;/p&gt;

&lt;p&gt;You can run my testcase by downloading &lt;a href=&quot;http://github.com/TonyGen/mongoTest-cpp&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://github.com/TonyGen/mongoTest-cpp&lt;/a&gt; and running&lt;br/&gt;
&amp;gt; mongoTest 666 controller 127.0.0.1 mongoTest::One&lt;br/&gt;
The log of each server is written to the current directory and is named mongodX.out where X is 1, 2, or 3. I usually tail -F each in a separate terminal window while it is running.&lt;/p&gt;

&lt;p&gt;I get an assertion error on every run on my MacBook Pro.&lt;/p&gt;

&lt;p&gt;Log from first replica server where error starts:&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6&amp;#93;&lt;/span&gt; couldn&apos;t make room for new record (len: 144) in capped ns local.oplog.rs&lt;br/&gt;
  Extent 0 (capExtent)&lt;br/&gt;
    magic: 41424344 extent-&amp;gt;ns: local.oplog.rs&lt;br/&gt;
    fr: null lr: 2:3771fd0 extent-&amp;gt;len: 192000256&lt;br/&gt;
 local.oplog.rs Assertion failure len * 5 &amp;gt; lastExtentSize db/namespace.cpp 423&lt;br/&gt;
0x100037d3e 0x10003d9c6 0x100097ed8 0x10009b95d 0x10009773c 0x1002ca34a 0x100287691 0x1000914a2 0x1001f4f0f 0x1001f79cb 0x1002baef0 0x1002bdd86 0x1003c5c1d 0x1007c6254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x0000000100037d3e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongod                              0x000000010003d9c6 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongod                              0x0000000100097ed8 _ZNK5mongo16NamespaceDetails13maybeComplainEPKci + 1736&lt;br/&gt;
 3   mongod                              0x000000010009b95d _ZN5mongo16NamespaceDetails11cappedAllocEPKci + 2637&lt;br/&gt;
 4   mongod                              0x000000010009773c _ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE + 492&lt;br/&gt;
 5   mongod                              0x00000001002ca34a _ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci + 394&lt;br/&gt;
 6   mongod                              0x0000000100287691 _ZN5mongoL8_logOpRSEPKcS1_S1_RKNS_7BSONObjEPS2_Pb + 1761&lt;br/&gt;
 7   mongod                              0x00000001000914a2 _ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pb + 66&lt;br/&gt;
 8   mongod                              0x00000001001f4f0f _ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE + 4367&lt;br/&gt;
 9   mongod                              0x00000001001f79cb _ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE + 315&lt;br/&gt;
 10  mongod                              0x00000001002baef0 _ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE + 1120&lt;br/&gt;
 11  mongod                              0x00000001002bdd86 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 5734&lt;br/&gt;
 12  mongod                              0x00000001003c5c1d _ZN5mongo10connThreadEPNS_13MessagingPortE + 653&lt;br/&gt;
 13  libboost_thread-mt.dylib            0x00000001007c6254 thread_proxy + 132&lt;br/&gt;
 14  libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 15  libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn6&amp;#93;&lt;/span&gt; update test.col  query: {} exception 0 assertion db/namespace.cpp:423 31ms&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn7&amp;#93;&lt;/span&gt; couldn&apos;t make room for new record (len: 124) in capped ns local.oplog.rs&lt;br/&gt;
  Extent 0 (capExtent)&lt;br/&gt;
    magic: 41424344 extent-&amp;gt;ns: local.oplog.rs&lt;br/&gt;
    fr: null lr: 2:3771fd0 extent-&amp;gt;len: 192000256&lt;br/&gt;
 local.oplog.rs Assertion failure len * 5 &amp;gt; lastExtentSize db/namespace.cpp 423&lt;br/&gt;
0x100037d3e 0x10003d9c6 0x100097ed8 0x10009b95d 0x10009773c 0x1002ca34a 0x100287691 0x1000914a2 0x1001f4f0f 0x1001f79cb 0x1002baef0 0x1002bdd86 0x1003c5c1d 0x1007c6254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x0000000100037d3e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongod                              0x000000010003d9c6 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongod                              0x0000000100097ed8 _ZNK5mongo16NamespaceDetails13maybeComplainEPKci + 1736&lt;br/&gt;
 3   mongod                              0x000000010009b95d _ZN5mongo16NamespaceDetails11cappedAllocEPKci + 2637&lt;br/&gt;
 4   mongod                              0x000000010009773c _ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE + 492&lt;br/&gt;
 5   mongod                              0x00000001002ca34a _ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci + 394&lt;br/&gt;
 6   mongod                              0x0000000100287691 _ZN5mongoL8_logOpRSEPKcS1_S1_RKNS_7BSONObjEPS2_Pb + 1761&lt;br/&gt;
 7   mongod                              0x00000001000914a2 _ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pb + 66&lt;br/&gt;
 8   mongod                              0x00000001001f4f0f _ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE + 4367&lt;br/&gt;
 9   mongod                              0x00000001001f79cb _ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE + 315&lt;br/&gt;
 10  mongod                              0x00000001002baef0 _ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE + 1120&lt;br/&gt;
 11  mongod                              0x00000001002bdd86 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 5734&lt;br/&gt;
 12  mongod                              0x00000001003c5c1d _ZN5mongo10connThreadEPNS_13MessagingPortE + 653&lt;br/&gt;
 13  libboost_thread-mt.dylib            0x00000001007c6254 thread_proxy + 132&lt;br/&gt;
 14  libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 15  libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn7&amp;#93;&lt;/span&gt; update test.col  query: {} exception 0 assertion db/namespace.cpp:423 171ms&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn10&amp;#93;&lt;/span&gt; couldn&apos;t make room for new record (len: 124) in capped ns local.oplog.rs&lt;br/&gt;
  Extent 0 (capExtent)&lt;br/&gt;
    magic: 41424344 extent-&amp;gt;ns: local.oplog.rs&lt;br/&gt;
    fr: null lr: 2:3771fd0 extent-&amp;gt;len: 192000256&lt;br/&gt;
 local.oplog.rs Assertion failure len * 5 &amp;gt; lastExtentSize db/namespace.cpp 423&lt;br/&gt;
0x100037d3e 0x10003d9c6 0x100097ed8 0x10009b95d 0x10009773c 0x1002ca34a 0x100287691 0x1000914a2 0x1001f4f0f 0x1001f79cb 0x1002baef0 0x1002bdd86 0x1003c5c1d 0x1007c6254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x0000000100037d3e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongod                              0x000000010003d9c6 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongod                              0x0000000100097ed8 _ZNK5mongo16NamespaceDetails13maybeComplainEPKci + 1736&lt;br/&gt;
 3   mongod                              0x000000010009b95d _ZN5mongo16NamespaceDetails11cappedAllocEPKci + 2637&lt;br/&gt;
 4   mongod                              0x000000010009773c _ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE + 492&lt;br/&gt;
 5   mongod                              0x00000001002ca34a _ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci + 394&lt;br/&gt;
 6   mongod                              0x0000000100287691 _ZN5mongoL8_logOpRSEPKcS1_S1_RKNS_7BSONObjEPS2_Pb + 1761&lt;br/&gt;
 7   mongod                              0x00000001000914a2 _ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pb + 66&lt;br/&gt;
 8   mongod                              0x00000001001f4f0f _ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE + 4367&lt;br/&gt;
 9   mongod                              0x00000001001f79cb _ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE + 315&lt;br/&gt;
 10  mongod                              0x00000001002baef0 _ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE + 1120&lt;br/&gt;
 11  mongod                              0x00000001002bdd86 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 5734&lt;br/&gt;
 12  mongod                              0x00000001003c5c1d _ZN5mongo10connThreadEPNS_13MessagingPortE + 653&lt;br/&gt;
 13  libboost_thread-mt.dylib            0x00000001007c6254 thread_proxy + 132&lt;br/&gt;
 14  libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 15  libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn10&amp;#93;&lt;/span&gt; update test.col  query: {} exception 0 assertion db/namespace.cpp:423 660ms&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn9&amp;#93;&lt;/span&gt; couldn&apos;t make room for new record (len: 124) in capped ns local.oplog.rs&lt;br/&gt;
  Extent 0 (capExtent)&lt;br/&gt;
    magic: 41424344 extent-&amp;gt;ns: local.oplog.rs&lt;br/&gt;
    fr: null lr: 2:3771fd0 extent-&amp;gt;len: 192000256&lt;br/&gt;
 local.oplog.rs Assertion failure len * 5 &amp;gt; lastExtentSize db/namespace.cpp 423&lt;br/&gt;
0x100037d3e 0x10003d9c6 0x100097ed8 0x10009b95d 0x10009773c 0x1002ca34a 0x100287691 0x1000914a2 0x1001f4f0f 0x1001f79cb 0x1002baef0 0x1002bdd86 0x1003c5c1d 0x1007c6254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x0000000100037d3e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongod                              0x000000010003d9c6 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongod                              0x0000000100097ed8 _ZNK5mongo16NamespaceDetails13maybeComplainEPKci + 1736&lt;br/&gt;
 3   mongod                              0x000000010009b95d _ZN5mongo16NamespaceDetails11cappedAllocEPKci + 2637&lt;br/&gt;
 4   mongod                              0x000000010009773c _ZN5mongo16NamespaceDetails5allocEPKciRNS_7DiskLocE + 492&lt;br/&gt;
 5   mongod                              0x00000001002ca34a _ZN5mongo11DataFileMgr17fast_oplog_insertEPNS_16NamespaceDetailsEPKci + 394&lt;br/&gt;
 6   mongod                              0x0000000100287691 _ZN5mongoL8_logOpRSEPKcS1_S1_RKNS_7BSONObjEPS2_Pb + 1761&lt;br/&gt;
 7   mongod                              0x00000001000914a2 _ZN5mongo5logOpEPKcS1_RKNS_7BSONObjEPS2_Pb + 66&lt;br/&gt;
 8   mongod                              0x00000001001f4f0f _ZN5mongo14_updateObjectsEbPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugEPNS_11RemoveSaverE + 4367&lt;br/&gt;
 9   mongod                              0x00000001001f79cb _ZN5mongo13updateObjectsEPKcRKNS_7BSONObjES2_bbbRNS_7OpDebugE + 315&lt;br/&gt;
 10  mongod                              0x00000001002baef0 _ZN5mongo14receivedUpdateERNS_7MessageERNS_5CurOpE + 1120&lt;br/&gt;
 11  mongod                              0x00000001002bdd86 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 5734&lt;br/&gt;
 12  mongod                              0x00000001003c5c1d _ZN5mongo10connThreadEPNS_13MessagingPortE + 653&lt;br/&gt;
 13  libboost_thread-mt.dylib            0x00000001007c6254 thread_proxy + 132&lt;br/&gt;
 14  libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 15  libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn9&amp;#93;&lt;/span&gt; update test.col  query: {} exception 0 assertion db/namespace.cpp:423 2312ms&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:63236 #19&lt;br/&gt;
Mon Jan 31 14:17:55 &lt;span class=&quot;error&quot;&gt;&amp;#91;ReplSetHealthPollTask&amp;#93;&lt;/span&gt; replSet member 127.0.0.1:27019 RECOVERING&lt;br/&gt;
Mon Jan 31 14:17:57 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn19&amp;#93;&lt;/span&gt; query local.oplog.rs reslen:12863 nscanned:103 { ts: &lt;/p&gt;
{ $gte: new Date(5568430794390648212) }
&lt;p&gt; }  nreturned:101 1656ms&lt;br/&gt;
Mon Jan 31 14:18:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn19&amp;#93;&lt;/span&gt; Assertion: 10334:Invalid BSONObj size: 50599229 (0x3D150403) first element: : ?type=122&lt;br/&gt;
0x10003e25d 0x100008d1d 0x1002ce8b8 0x1001d6380 0x1002b682d 0x1002bda27 0x1003c5c1d 0x1007c6254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x000000010003e25d _ZN5mongo11msgassertedEiPKc + 381&lt;br/&gt;
 1   mongod                              0x0000000100008d1d _ZNK5mongo7BSONObj14_assertInvalidEv + 1309&lt;br/&gt;
 2   mongod                              0x00000001002ce8b8 _ZN5mongo11BasicCursor7currentEv + 104&lt;br/&gt;
 3   mongod                              0x00000001001d6380 _ZN5mongo14processGetMoreEPKcixRNS_5CurOpEiRb + 1968&lt;br/&gt;
 4   mongod                              0x00000001002b682d _ZN5mongo15receivedGetMoreERNS_10DbResponseERNS_7MessageERNS_5CurOpE + 589&lt;br/&gt;
 5   mongod                              0x00000001002bda27 _ZN5mongo16assembleResponseERNS_7MessageERNS_10DbResponseERKNS_8SockAddrE + 4871&lt;br/&gt;
 6   mongod                              0x00000001003c5c1d _ZN5mongo10connThreadEPNS_13MessagingPortE + 653&lt;br/&gt;
 7   libboost_thread-mt.dylib            0x00000001007c6254 thread_proxy + 132&lt;br/&gt;
 8   libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 9   libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Jan 31 14:18:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn19&amp;#93;&lt;/span&gt; getmore local.oplog.rs cid:4746124974211895091 getMore: { query: {}, orderby: &lt;/p&gt;
{ $natural: -1 }
&lt;p&gt; }  exception 10334 Invalid BSONObj size: 50599229 (0x3D150403) first element: : ?type=122 bytes:20 nreturned:0 193ms&lt;br/&gt;
Mon Jan 31 14:18:11 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn19&amp;#93;&lt;/span&gt; end connection 127.0.0.1:63236&lt;br/&gt;
Mon Jan 31 14:18:12 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:63246 #20&lt;br/&gt;
Mon Jan 31 14:18:13 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn20&amp;#93;&lt;/span&gt; query local.oplog.rs reslen:12863 nscanned:103 { ts: &lt;/p&gt;
{ $gte: new Date(5568430794390648212) }
&lt;p&gt; }  nreturned:101 1525ms&lt;br/&gt;
Mon Jan 31 14:18:23 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn20&amp;#93;&lt;/span&gt; end connection 127.0.0.1:63246&lt;br/&gt;
Mon Jan 31 14:18:24 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:63253 #21&lt;br/&gt;
Mon Jan 31 14:18:26 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn21&amp;#93;&lt;/span&gt; query local.oplog.rs reslen:12863 nscanned:103 { ts: &lt;/p&gt;
{ $gte: new Date(5568430794390648212) }
&lt;p&gt; }  nreturned:101 1525ms&lt;br/&gt;
Mon Jan 31 14:18:36 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn21&amp;#93;&lt;/span&gt; end connection 127.0.0.1:63253&lt;br/&gt;
Mon Jan 31 14:18:37 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:63260 #22&lt;br/&gt;
Mon Jan 31 14:18:38 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn22&amp;#93;&lt;/span&gt; query local.oplog.rs reslen:12863 nscanned:103 { ts: &lt;/p&gt;
{ $gte: new Date(5568430794390648212) }
&lt;p&gt; }  nreturned:101 1529ms&lt;br/&gt;
Mon Jan 31 14:18:48 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn22&amp;#93;&lt;/span&gt; end connection 127.0.0.1:63260&lt;br/&gt;
Mon Jan 31 14:18:49 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; connection accepted from 127.0.0.1:63267 #23&lt;br/&gt;
Mon Jan 31 14:18:50 &lt;span class=&quot;error&quot;&gt;&amp;#91;dur&amp;#93;&lt;/span&gt; lsn set 234203&lt;br/&gt;
Mon Jan 31 14:18:51 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn23&amp;#93;&lt;/span&gt; query local.oplog.rs reslen:12863 nscanned:103 { ts: &lt;/p&gt;
{ $gte: new Date(5568430794390648212) }
&lt;p&gt; }  nreturned:101 1670ms&lt;/p&gt;</description>
                <environment>db version v1.8.0-rc0-pre-&lt;br/&gt;
git version: 9a6f2dcf91c2025f7549c875e78e4a203b26adcf</environment>
        <key id="14368">SERVER-2445</key>
            <summary>Assertion failure in replica set + durability while killing and restarting servers</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="5">Cannot Reproduce</resolution>
                                        <assignee username="-1">Unassigned</assignee>
                                    <reporter username="tonyh">Tony Hannan</reporter>
                        <labels>
                    </labels>
                <created>Mon, 31 Jan 2011 19:37:21 +0000</created>
                <updated>Sat, 3 Sep 2011 14:40:46 +0000</updated>
                            <resolved>Sat, 3 Sep 2011 14:40:46 +0000</resolved>
                                    <version>1.7.5</version>
                                                    <component>Replication</component>
                    <component>Storage</component>
                                        <votes>0</votes>
                                    <watches>0</watches>
                                                                                                                <comments>
                            <comment id="52381" author="eliot" created="Sat, 3 Sep 2011 14:40:46 +0000"  >&lt;p&gt;THis was probably one of the numerous journalling issues fixed&lt;/p&gt;</comment>
                            <comment id="23306" author="tonyh" created="Mon, 7 Feb 2011 22:08:58 +0000"  >&lt;p&gt;My C++ test (from original description above) induces the following server errors occasionally. I&apos;m unable to induce these from my Javascript test.&lt;/p&gt;

&lt;p&gt;Error 1 - can&apos;t restart after kill:&lt;br/&gt;
Mon Feb  7 13:41:37 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; recover cleaning up&lt;br/&gt;
Mon Feb  7 13:41:37 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; removeJournalFiles&lt;br/&gt;
Mon Feb  7 13:41:37 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; recover done&lt;br/&gt;
Mon Feb  7 13:41:38 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; preallocateIsFaster=true 3.02&lt;br/&gt;
Mon Feb  7 13:41:38 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; preallocating a journal file /db/db1/journal/prealloc.0&lt;br/&gt;
Mon Feb  7 13:42:01 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; preallocating a journal file /db/db1/journal/prealloc.1&lt;br/&gt;
Mon Feb  7 13:42:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; MongoDB starting : pid=2296 port=27018 dbpath=/db/db1 64-bit &lt;br/&gt;
Mon Feb  7 13:42:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; db version v1.8.0-rc0-pre-, pdfile version 4.5&lt;br/&gt;
Mon Feb  7 13:42:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; git version: b0834a3e4314353e6b4a9890343119d6810e8272&lt;br/&gt;
Mon Feb  7 13:42:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; sys info: Darwin dmmb.local 10.5.0 Darwin Kernel Version 10.5.0: Fri Nov  5 23:20:39 PDT 2010; root:xnu-1504.9.17~1/RELEASE_I386 i386 BOOST_LIB_VERSION=1_44&lt;/p&gt;
&lt;ul&gt;
	&lt;li&gt;
	&lt;ul&gt;
		&lt;li&gt;
		&lt;ul&gt;
			&lt;li&gt;
			&lt;ul&gt;
				&lt;li&gt;
				&lt;ul&gt;
					&lt;li&gt;
					&lt;ul&gt;
						&lt;li&gt;
						&lt;ul&gt;
							&lt;li&gt;
							&lt;ul&gt;
								&lt;li&gt;
								&lt;ul&gt;
									&lt;li&gt;
									&lt;ul&gt;
										&lt;li&gt;
										&lt;ul&gt;
											&lt;li&gt;
											&lt;ul&gt;
												&lt;li&gt;
												&lt;ul&gt;
													&lt;li&gt;
													&lt;ul&gt;
														&lt;li&gt;&lt;p&gt;old lock file: /db/db1/mongod.lock.  probably means unclean shutdown&lt;br/&gt;
but there are no journal files to recover.&lt;br/&gt;
see: &lt;a href=&quot;http://dochub.mongodb.org/core/repair&quot; class=&quot;external-link&quot; target=&quot;_blank&quot; rel=&quot;nofollow noopener&quot;&gt;http://dochub.mongodb.org/core/repair&lt;/a&gt; for more information&lt;br/&gt;
*************&lt;br/&gt;
Mon Feb  7 13:42:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;initandlisten&amp;#93;&lt;/span&gt; exception in initAndListen std::exception: old lock file, terminating&lt;br/&gt;
...&lt;/p&gt;&lt;/li&gt;
													&lt;/ul&gt;
													&lt;/li&gt;
												&lt;/ul&gt;
												&lt;/li&gt;
											&lt;/ul&gt;
											&lt;/li&gt;
										&lt;/ul&gt;
										&lt;/li&gt;
									&lt;/ul&gt;
									&lt;/li&gt;
								&lt;/ul&gt;
								&lt;/li&gt;
							&lt;/ul&gt;
							&lt;/li&gt;
						&lt;/ul&gt;
						&lt;/li&gt;
					&lt;/ul&gt;
					&lt;/li&gt;
				&lt;/ul&gt;
				&lt;/li&gt;
			&lt;/ul&gt;
			&lt;/li&gt;
		&lt;/ul&gt;
		&lt;/li&gt;
	&lt;/ul&gt;
	&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Error 2 - assertion failure:&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;ReplSetHealthPollTask&amp;#93;&lt;/span&gt; replSet info 127.0.0.1:27019 is up&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;ReplSetHealthPollTask&amp;#93;&lt;/span&gt; replSet member 127.0.0.1:27019 PRIMARY&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;ReplSetHealthPollTask&amp;#93;&lt;/span&gt; replSet info 127.0.0.1:27020 is up&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;ReplSetHealthPollTask&amp;#93;&lt;/span&gt; replSet member 127.0.0.1:27020 ARBITER&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; building new index on &lt;/p&gt;
{ _id: 1 }
&lt;p&gt; for local.me&lt;br/&gt;
Mon Feb  7 16:32:31 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; done for 0 records 0secs&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet our last op time written: Feb  7 16:32:09:8d0&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replset source&apos;s GTE: Feb  7 16:32:29:1&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 0&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet ROLLBACK&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 1&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 2 FindCommonPoint&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet info rollback our last optime:   Feb  7 16:32:09:8d0&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet info rollback their last optime: Feb  7 16:32:32:ad4&lt;br/&gt;
Mon Feb  7 16:32:32 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet info rollback diff in end of log times: -23 seconds&lt;br/&gt;
Mon Feb  7 16:32:33 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback found matching events at Feb  7 16:30:19:689&lt;br/&gt;
Mon Feb  7 16:32:33 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback findcommonpoint scanned : 691768&lt;br/&gt;
Mon Feb  7 16:32:33 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet replSet rollback 3 fixup&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 3.5&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 4 n:10000&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet minvalid=Feb  7 16:32:39 4d5064f7:1190&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; building new index on &lt;/p&gt;
{ _id: 1 }
&lt;p&gt; for local.replset.minvalid&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; done for 0 records 0secs&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 4.6&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 4.7&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 5 d:0 u:10000&lt;br/&gt;
Mon Feb  7 16:32:40 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 6&lt;br/&gt;
Mon Feb  7 16:32:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet syncThread: 13623 DR102 too much data written uncommitted&lt;br/&gt;
Mon Feb  7 16:32:41 &lt;span class=&quot;error&quot;&gt;&amp;#91;dur&amp;#93;&lt;/span&gt; lsn set 11485&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet our last op time written: Feb  7 16:32:09:8d0&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replset source&apos;s GTE: Feb  7 16:32:29:1&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 0&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 1&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 2 FindCommonPoint&lt;br/&gt;
Mon Feb  7 16:32:53 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet rollback 2 error findcommonpoint waiting a while before trying again&lt;br/&gt;
Mon Feb  7 16:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; DBClientCursor::init call() failed&lt;br/&gt;
Mon Feb  7 16:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn5&amp;#93;&lt;/span&gt; end connection 127.0.0.1:58367&lt;br/&gt;
Mon Feb  7 16:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt;   Assertion failure r.haveCursor() db/repl/rs_sync.cpp 285&lt;br/&gt;
0x10003853e 0x10003e176 0x10028667e 0x10028772e 0x1002878db 0x100287efb 0x100c44254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
 0   mongod                              0x000000010003853e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongod                              0x000000010003e176 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongod                              0x000000010028667e _ZN5mongo11ReplSetImpl8syncTailEv + 2558&lt;br/&gt;
 3   mongod                              0x000000010028772e _ZN5mongo11ReplSetImpl11_syncThreadEv + 318&lt;br/&gt;
 4   mongod                              0x00000001002878db _ZN5mongo11ReplSetImpl10syncThreadEv + 139&lt;br/&gt;
 5   mongod                              0x0000000100287efb _ZN5mongo15startSyncThreadEv + 731&lt;br/&gt;
 6   libboost_thread-mt.dylib            0x0000000100c44254 thread_proxy + 132&lt;br/&gt;
 7   libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
 8   libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
Mon Feb  7 16:33:06 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet syncThread: 0 assertion db/repl/rs_sync.cpp:285&lt;/p&gt;</comment>
                            <comment id="23214" author="tonyh" created="Mon, 7 Feb 2011 04:08:02 +0000"  >&lt;p&gt;Ok, jstests/multiClient/rsDurKillRestart1.js is now thread-safe. It gives following assertion error some of the time:&lt;/p&gt;

&lt;p&gt;m31000| Sun Feb  6 22:42:43 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt;   Assertion failure r.haveCursor() db/repl/rs_sync.cpp 285&lt;br/&gt;
m31000| 0x100038a1e 0x10003e656 0x10027a9fe 0x10027baae 0x10027bc5b 0x10027c27b 0x1007c8254 0x7fff887cb536 0x7fff887cb3e9 &lt;br/&gt;
m31000|  0   mongod                              0x0000000100038a1e _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
m31000|  1   mongod                              0x000000010003e656 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
m31000|  2   mongod                              0x000000010027a9fe _ZN5mongo11ReplSetImpl8syncTailEv + 2558&lt;br/&gt;
m31000|  3   mongod                              0x000000010027baae _ZN5mongo11ReplSetImpl11_syncThreadEv + 318&lt;br/&gt;
m31000|  4   mongod                              0x000000010027bc5b _ZN5mongo11ReplSetImpl10syncThreadEv + 139&lt;br/&gt;
m31000|  5   mongod                              0x000000010027c27b _ZN5mongo15startSyncThreadEv + 731&lt;br/&gt;
m31000|  6   libboost_thread-mt.dylib            0x00000001007c8254 thread_proxy + 132&lt;br/&gt;
m31000|  7   libSystem.B.dylib                   0x00007fff887cb536 _pthread_start + 331&lt;br/&gt;
m31000|  8   libSystem.B.dylib                   0x00007fff887cb3e9 thread_start + 13&lt;br/&gt;
m31000| Sun Feb  6 22:42:43 &lt;span class=&quot;error&quot;&gt;&amp;#91;replica set sync&amp;#93;&lt;/span&gt; replSet syncThread: 0 assertion db/repl/rs_sync.cpp:285&lt;/p&gt;</comment>
                            <comment id="23110" author="tonyh" created="Fri, 4 Feb 2011 15:21:00 +0000"  >&lt;p&gt;Ah, sorry, of course. I was used to connection object being thread-safe like in the other drivers. I will edit.&lt;/p&gt;</comment>
                            <comment id="23104" author="eliot" created="Fri, 4 Feb 2011 04:58:17 +0000"  >&lt;p&gt;I don&apos;t think your test is thread safe.&lt;/p&gt;

&lt;p&gt;Your&apos;e re-using the same connection to a server from multiple threads.&lt;/p&gt;
</comment>
                            <comment id="23074" author="tonyh" created="Thu, 3 Feb 2011 22:33:20 +0000"  >&lt;p&gt;Made same test case in javascript: jstests/multiClient/rsDurKillRestart1.js&lt;/p&gt;

&lt;p&gt;1. Build mongo shell with V8 so multiple client threads can run&lt;br/&gt;
2. $ mongo --nodb&lt;br/&gt;
3. &amp;gt; load(&apos;jstests/multiClient/rsDurKillRestart1.js&apos;)&lt;br/&gt;
4. &amp;gt; go(100)&lt;/p&gt;

&lt;p&gt;Get following 2 client errors in log. Cannot reproduce server error described above in this javascript version, only in above C++ version. Second error below does not happen as regularly as first.&lt;/p&gt;

&lt;p&gt;Error 1:&lt;/p&gt;

&lt;p&gt;update 1 round 59&lt;br/&gt;
Thu Feb  3 16:25:43 ERROR: MessagingPort::call() wrong id got:304d5c30 expect:304d5c2e&lt;br/&gt;
  toSend op: 2004&lt;br/&gt;
  response msgid:930392286&lt;br/&gt;
  response len:  64&lt;br/&gt;
  response op:  1&lt;br/&gt;
  farEnd: 127.0.0.1:31001&lt;br/&gt;
Thu Feb  3 16:25:43   Assertion failure false util/message.cpp 512&lt;br/&gt;
0x1000111be 0x100013e16 0x10007bba1 0x10007c685 0x10008d483 0x1000ab16d 0x10008e46d 0x1000a0c5b 0x1000d777a 0x1000dc9d6 0x10585abbf &lt;br/&gt;
 0   mongo                               0x00000001000111be _ZN5mongo12sayDbContextEPKc + 174&lt;br/&gt;
 1   mongo                               0x0000000100013e16 _ZN5mongo8assertedEPKcS1_j + 342&lt;br/&gt;
 2   mongo                               0x000000010007bba1 &lt;em&gt;ZN5mongo13MessagingPort4recvERKNS_7MessageERS1&lt;/em&gt; + 705&lt;br/&gt;
 3   mongo                               0x000000010007c685 &lt;em&gt;ZN5mongo13MessagingPort4callERNS_7MessageES2&lt;/em&gt; + 53&lt;br/&gt;
 4   mongo                               0x000000010008d483 _ZN5mongo18DBClientConnection4callERNS_7MessageES2_b + 51&lt;br/&gt;
 5   mongo                               0x00000001000ab16d _ZN5mongo14DBClientCursor4initEv + 301&lt;br/&gt;
 6   mongo                               0x000000010008e46d _ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 1085&lt;br/&gt;
 7   mongo                               0x00000001000a0c5b _ZN5mongo18DBClientConnection5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 139&lt;br/&gt;
 8   mongo                               0x00000001000d777a _ZN5mongo9mongoFindERKN2v89ArgumentsE + 858&lt;br/&gt;
 9   mongo                               0x00000001000dc9d6 &lt;em&gt;ZN5mongo10v8CallbackIXadL_ZNS_9mongoFindERKN2v89ArgumentsEEEEENS1_6HandleINS1_5ValueEEES4&lt;/em&gt; + 118&lt;br/&gt;
 10  ???                                 0x000000010585abbf 0x0 + 4387613631&lt;br/&gt;
query failed (down primary), retry soon: socket error on query&lt;/p&gt;

&lt;p&gt;Error 2:&lt;/p&gt;

&lt;p&gt;Thu Feb  3 16:25:53 trying reconnect to 127.0.0.1:31001&lt;br/&gt;
Thu Feb  3 16:25:53 MessagingPort recv() errno:9 Bad file descriptor 127.0.0.1:31001&lt;br/&gt;
Thu Feb  3 16:25:53 mongo got signal 11 (Segmentation fault), stack trace: &lt;/p&gt;

&lt;p&gt;Thu Feb  3 16:25:53 reconnect 127.0.0.1:31001 ok&lt;br/&gt;
Thu Feb  3 16:25:53 0x100020699 0x7fff887f267a 0x3abb0 0x10007b937 0x10007c685 0x10008d483 0x1000ab16d 0x10008e46d 0x1000a0c5b 0x1000d777a 0x1000dc9d6 0x10585abbf &lt;br/&gt;
 0   mongo                               0x0000000100020699 _Z12quitAbruptlyi + 1017&lt;br/&gt;
 1   libSystem.B.dylib                   0x00007fff887f267a _sigtramp + 26&lt;br/&gt;
 2   ???                                 0x000000000003abb0 0x0 + 240560&lt;br/&gt;
 3   mongo                               0x000000010007b937 &lt;em&gt;ZN5mongo13MessagingPort4recvERKNS_7MessageERS1&lt;/em&gt; + 87&lt;br/&gt;
 4   mongo                               0x000000010007c685 &lt;em&gt;ZN5mongo13MessagingPort4callERNS_7MessageES2&lt;/em&gt; + 53&lt;br/&gt;
 5   mongo                               0x000000010008d483 _ZN5mongo18DBClientConnection4callERNS_7MessageES2_b + 51&lt;br/&gt;
 6   mongo                               0x00000001000ab16d _ZN5mongo14DBClientCursor4initEv + 301&lt;br/&gt;
 7   mongo                               0x000000010008e46d _ZN5mongo12DBClientBase5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 1085&lt;br/&gt;
 8   mongo                               0x00000001000a0c5b _ZN5mongo18DBClientConnection5queryERKSsNS_5QueryEiiPKNS_7BSONObjEii + 139&lt;br/&gt;
 9   mongo                               0x00000001000d777a _ZN5mongo9mongoFindERKN2v89ArgumentsE + 858&lt;br/&gt;
 10  mongo                               0x00000001000dc9d6 &lt;em&gt;ZN5mongo10v8CallbackIXadL_ZNS_9mongoFindERKN2v89ArgumentsEEEEENS1_6HandleINS1_5ValueEEES4&lt;/em&gt; + 118&lt;br/&gt;
 11  ???                                 0x000000010585abbf 0x0 + 4387613631&lt;/p&gt;</comment>
                            <comment id="22888" author="eliot" created="Mon, 31 Jan 2011 21:17:40 +0000"  >&lt;p&gt;Can you do this with our normal test harness.&lt;/p&gt;</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>7.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 31 Jan 2011 21:17:40 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        12 years, 24 weeks, 4 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ian@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            12 years, 24 weeks, 4 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10021"><![CDATA[OS X]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>eliot</customfieldvalue>
            <customfieldvalue>tonyh</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrp7jj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hricwv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>20744</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrljdj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>