<!-- 
RSS generated by JIRA (9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66) at Thu Feb 08 03:22:09 UTC 2024

It is possible to restrict the fields that are returned in this document by specifying the 'field' parameter in your request.
For example, to request only the issue key and summary append 'field=key&field=summary' to the URL of your request.
-->
<rss version="0.92" >
<channel>
    <title>MongoDB Jira</title>
    <link>https://jira.mongodb.org</link>
    <description>This file is an XML representation of an issue</description>
    <language>en-us</language>    <build-info>
        <version>9.7.1</version>
        <build-number>970001</build-number>
        <build-date>13-04-2023</build-date>
    </build-info>


<item>
            <title>[SERVER-10056] Shard not returning some data when querying from MongoS</title>
                <link>https://jira.mongodb.org/browse/SERVER-10056</link>
                <project id="10000" key="SERVER">Core Server</project>
                    <description>&lt;p&gt;Setup: 30 shards, no replication, AWS Hi1.4x Large with SSD high IOPs&lt;br/&gt;
Database: sharded ouput of a routine map reduce operation (merged) &lt;br/&gt;
Problem: Some of the records of this sharded collection will only return when I&apos;m logged into the particular shard.  In MongoS, I get no results, except when I do a count(), then I get an expected return.  Example:&lt;/p&gt;

&lt;p&gt;I run the following query on MongoS -&lt;/p&gt;

&lt;p&gt;  var d = new Date(2013,5,26);&lt;br/&gt;
  db.hours.find({&quot;_id.date&quot; : {$gt : d}})&lt;/p&gt;

&lt;p&gt;I expect several records but nothing returns.  It just goes back to a blank command line. &lt;br/&gt;
 However, when I run a count on the same query, I get the expected result -&lt;/p&gt;

&lt;p&gt;  mongos&amp;gt; db.hours.find({&quot;date&quot; : {$gt : d}}).count()&lt;br/&gt;
  11318035&lt;/p&gt;

&lt;p&gt;When I log directly into the MongoD shard that has this date range of records and run the query there I get a cursor of documents back, as expected.  It appears I can only access these records by logging into the particular shard directly.  This is the case for all records after the 25th of June.&lt;/p&gt;

&lt;p&gt;The map reduce query that creates and updates this sharded collection is as follows:&lt;/p&gt;

&lt;p&gt;//MAP REDUCE COMMAND&lt;br/&gt;
        db.sessions.mapReduce (&lt;br/&gt;
               mapFxn,&lt;br/&gt;
               reduceFxn,&lt;br/&gt;
               { out : &lt;/p&gt;
{ merge : &quot;hour&quot;,
                        db : &quot;summary&quot;,
                        sharded : true,
                        nonAtomic : true
                        }
&lt;p&gt;,&lt;br/&gt;
                   query: { &quot;date&quot; : {$gte : lastRunTime, $lte : transactionTime}}&lt;/p&gt;

&lt;p&gt;                           }&lt;br/&gt;
           );&lt;/p&gt;


&lt;p&gt;-----------------------------------------&lt;br/&gt;
Looking at my sharding status -&lt;/p&gt;

&lt;p&gt;	summary.hour&lt;br/&gt;
			shard key: &lt;/p&gt;
{ &quot;_id&quot; : 1 }
&lt;p&gt;			chunks:&lt;br/&gt;
				shard0001	3&lt;br/&gt;
				shard0009	3&lt;br/&gt;
				shard0002	3&lt;br/&gt;
				shard0010	3&lt;br/&gt;
				shard0003	3&lt;br/&gt;
				shard0011	3&lt;br/&gt;
				shard0004	3&lt;br/&gt;
				shard0005	3&lt;br/&gt;
				shard0014	3&lt;br/&gt;
				shard0013	3&lt;br/&gt;
				shard0008	3&lt;br/&gt;
				shard0012	3&lt;br/&gt;
				shard0015	3&lt;br/&gt;
				shard0016	2&lt;br/&gt;
				shard0017	2&lt;br/&gt;
				shard0018	2&lt;br/&gt;
				shard0019	2&lt;br/&gt;
				shard0020	2&lt;br/&gt;
				shard0021	2&lt;br/&gt;
				shard0022	2&lt;br/&gt;
				shard0023	2&lt;br/&gt;
				shard0024	2&lt;br/&gt;
				shard0025	2&lt;br/&gt;
				shard0026	2&lt;br/&gt;
				shard0027	2&lt;br/&gt;
				shard0028	2&lt;br/&gt;
				shard0029	2&lt;br/&gt;
				shard0030	2&lt;br/&gt;
				shard0031	2&lt;br/&gt;
				shard0006	3&lt;br/&gt;
				shard0007	3&lt;br/&gt;
				shard0000	3&lt;br/&gt;
			too many chunks to print, use verbose if you want to force print&lt;/p&gt;



&lt;p&gt; Shard0000 is the primary shard and also the shard that  contains these records.  Here is a sample of the verbose listing of chunks for that collection.  The date key does not seem to be updating past the 25th of June, even though there are records all the way through the 27th -&lt;/p&gt;

&lt;p&gt;on : shard0000 &lt;/p&gt;
{ &quot;t&quot; : 77, &quot;i&quot; : 2 }
&lt;p&gt; &lt;/p&gt;

&lt;p&gt;{&lt;/p&gt;

&lt;p&gt;&quot;_id&quot; : &lt;/p&gt;
{

&quot;date&quot; : ISODate(&quot;2013-06-18T02:00:00Z&quot;),

&quot;ci&quot; : &quot;4352&quot;,

&quot;site&quot; : &quot;foo.com&quot;,

&quot;page&quot; : &quot;home&quot;,

&quot;Origin_Domain&quot; : &quot;bar.com&quot;,

&quot;pl&quot; : &quot;pub&quot;,

&quot;v&quot; : &quot;1.6&quot;,


}

}
&lt;p&gt; --&amp;gt;&amp;gt; {&lt;/p&gt;

&lt;p&gt;&quot;_id&quot; : &lt;/p&gt;
{

&quot;date&quot; : ISODate(&quot;2013-06-25T11:00:00Z&quot;),

&quot;ci&quot; : &quot;4543&quot;,

&quot;site&quot; : &quot;foo.com&quot;,

&quot;page&quot; : &quot;home&quot;,

&quot;Origin_Domain&quot; : &quot;unavailable&quot;,

&quot;pl&quot; : &quot;pub&quot;,

&quot;v&quot; : &quot;1.6&quot;

}

&lt;p&gt;} on : shard0000 &lt;/p&gt;
{ &quot;t&quot; : 77, &quot;i&quot; : 3 }
&lt;p&gt; &lt;/p&gt;

&lt;p&gt;{&lt;/p&gt;

&lt;p&gt;&quot;_id&quot; : &lt;/p&gt;
{

&quot;date&quot; : ISODate(&quot;2013-06-25T11:00:00Z&quot;),

&quot;ci&quot; : &quot;3453&quot;,

&quot;site&quot; : &quot;bar.com&quot;,

&quot;page&quot; : &quot;home&quot;,

&quot;Origin_Domain&quot; : &quot;unavailable&quot;,

&quot;pl&quot; : &quot;pub&quot;,

&quot;v&quot; : &quot;1.6&quot;

}

&lt;p&gt;} --&amp;gt;&amp;gt; { &quot;_id&quot; : &lt;/p&gt;
{ &quot;$maxKey&quot; : 1 }
&lt;p&gt; } on : shard0012 &lt;/p&gt;
{ &quot;t&quot; : 76, &quot;i&quot; : 0 }
&lt;p&gt; &lt;/p&gt;

{  &quot;_id&quot; : &quot;summmary&quot;,  &quot;partitioned&quot; : false,  &quot;primary&quot; : &quot;shard0021&quot; }



&lt;p&gt;========================&lt;/p&gt;</description>
                <environment>linux, AWS</environment>
        <key id="80714">SERVER-10056</key>
            <summary>Shard not returning some data when querying from MongoS</summary>
                <type id="1" iconUrl="https://jira.mongodb.org/secure/viewavatar?size=xsmall&amp;avatarId=14703&amp;avatarType=issuetype">Bug</type>
                                            <priority id="3" iconUrl="https://jira.mongodb.org/images/icons/priorities/major.svg">Major - P3</priority>
                        <status id="6" iconUrl="https://jira.mongodb.org/images/icons/statuses/closed.png" description="The issue is considered finished, the resolution is correct. Issues which are closed can be reopened.">Closed</status>
                    <statusCategory id="3" key="done" colorName="success"/>
                                    <resolution id="9">Done</resolution>
                                        <assignee username="david.hows">David Hows</assignee>
                                    <reporter username="tamer">Tamer Hassan</reporter>
                        <labels>
                    </labels>
                <created>Fri, 28 Jun 2013 16:53:40 +0000</created>
                <updated>Wed, 10 Dec 2014 23:18:15 +0000</updated>
                            <resolved>Mon, 12 Aug 2013 03:10:41 +0000</resolved>
                                    <version>2.4.1</version>
                    <version>2.4.4</version>
                                                    <component>Sharding</component>
                                        <votes>0</votes>
                                    <watches>6</watches>
                                                                                                                <comments>
                            <comment id="381824" author="david.hows" created="Tue, 16 Jul 2013 02:12:28 +0000"  >&lt;p&gt;Hi Tamer,&lt;/p&gt;

&lt;p&gt;Sorry for the delay in getting back to you.&lt;/p&gt;

&lt;p&gt;Has your rebuild fixed this issue?&lt;/p&gt;

&lt;p&gt;If not, then it looks like the next place to examine would be the MongoD&apos;s - as none of the shards seem to be returning a cursor to begin retrieving documents.&lt;/p&gt;

&lt;p&gt;Here is my reproduction i get results that look like the following, which are basically the same, except one of the shards returns a non-empty cursor, which is where the results come from. My next question would be, why is nothing returned by the MongoD?&lt;/p&gt;

&lt;p/&gt;
&lt;div id=&quot;syntaxplugin&quot; class=&quot;syntaxplugin&quot; style=&quot;border: 1px dashed #bbb; border-radius: 5px !important; overflow: auto; max-height: 30em;&quot;&gt;
&lt;table cellspacing=&quot;0&quot; cellpadding=&quot;0&quot; border=&quot;0&quot; width=&quot;100%&quot; style=&quot;font-size: 1em; line-height: 1.4em !important; font-weight: normal; font-style: normal; color: black;&quot;&gt;
		&lt;tbody &gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;  margin-top: 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.980 [conn4] pcursorcreating pcursor over QSpec { ns: &quot;test.test2&quot;, n2skip: 0, n2return: 0, options: 0, query: {}, fields: {} } and CInfo { v_ns: &quot;&quot;, filter: {} }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.980 [conn4] pcursorinitializing over 3 shards required by [test.test2 @ 19|1||51de4f46ac52fae2143a86ef]&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.980 [conn4] pcursorinitializing on shard shard0000:Pixl.local:30000, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.982 [conn4] pcursorinitialized query (lazily) on shard shard0000:Pixl.local:30000, current connection state is { state: { conn: &quot;Pixl.local:30000&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.982 [conn4] pcursorinitializing on shard shard0001:Pixl.local:30001, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorinitialized query (lazily) on shard shard0001:Pixl.local:30001, current connection state is { state: { conn: &quot;Pixl.local:30001&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorinitializing on shard shard0002:Pixl.local:30002, current connection state is { state: {}, retryNext: false, init: false, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorinitialized query (lazily) on shard shard0002:Pixl.local:30002, current connection state is { state: { conn: &quot;Pixl.local:30002&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorfinishing over 3 shards&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorfinishing on shard shard0000:Pixl.local:30000, current connection state is { state: { conn: &quot;Pixl.local:30000&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.983 [conn4] pcursorfinished on shard shard0000:Pixl.local:30000, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: { _id: ObjectId(&apos;51de4f5506cd8150cda334b3&apos;), loc: &quot;MWN&quot;, ser_num: 2403.0 }, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.984 [conn4] pcursorfinishing on shard shard0001:Pixl.local:30001, current connection state is { state: { conn: &quot;Pixl.local:30001&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.984 [conn4] pcursorfinished on shard shard0001:Pixl.local:30001, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.984 [conn4] pcursorfinishing on shard shard0002:Pixl.local:30002, current connection state is { state: { conn: &quot;Pixl.local:30002&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: false, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
				&lt;tr id=&quot;syntaxplugin_code_and_gutter&quot;&gt;
						&lt;td  style=&quot; line-height: 1.4em !important; padding: 0em; vertical-align: top;&quot;&gt;
					&lt;pre style=&quot;font-size: 1em; margin: 0 10px;   margin-bottom: 10px;  width: auto; padding: 0;&quot;&gt;&lt;span style=&quot;color: black; font-family: &apos;Consolas&apos;, &apos;Bitstream Vera Sans Mono&apos;, &apos;Courier New&apos;, Courier, monospace !important;&quot;&gt;Tue Jul 16 12:03:15.984 [conn4] pcursorfinished on shard shard0002:Pixl.local:30002, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;test.test2 @ 19|1||51de4f46ac52fae2143a86ef&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;/span&gt;&lt;/pre&gt;
			&lt;/td&gt;
		&lt;/tr&gt;
			&lt;/tbody&gt;
&lt;/table&gt;
&lt;/div&gt;
&lt;p/&gt;

&lt;p&gt;Please let me know where you are with this issue and we can continue from there.&lt;/p&gt;

&lt;p&gt;Thanks,&lt;br/&gt;
David&lt;/p&gt;</comment>
                            <comment id="378123" author="tamer" created="Wed, 10 Jul 2013 16:29:18 +0000"  >&lt;p&gt;a close look at the query in some of the log records from above -&lt;/p&gt;

&lt;p&gt;Wed Jul 10 11:04:44.016 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; creating pcursor over QSpec { ns: &quot;summary.$cmd&quot;, n2skip: 0, n2return: 1, options: 0, query: { count: &quot;hour&quot;, query: { _id.date: &lt;/p&gt;
{ $gte: new Date(1373241600000) }
&lt;p&gt; } }, fields: {} } and CInfo { v_ns: &quot;summary.hour&quot;, filter: { _id.date: &lt;/p&gt;
{ $gte: new Date(1373241600000) }
&lt;p&gt; } }&lt;br/&gt;
Wed Jul 10 11:04:44.016 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initializing over 8 shards required by &lt;span class=&quot;error&quot;&gt;&amp;#91;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&amp;#93;&lt;/span&gt;&lt;br/&gt;
Wed Jul 10 11:04:44.016 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11000&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.016 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11001&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11002&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11003&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11004&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11005&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11006&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized command (lazily) on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11010&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11000&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 77.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11001&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11002&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11003&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.017 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11004&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.018 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.018 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11005&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.254 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 1276334.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.254 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11006&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.254 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.254 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11010&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:44.254 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: { conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &lt;/p&gt;
{ n: 0.0, ok: 1.0 }
&lt;p&gt;, count: 0, done: false }, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:45.653 &lt;span class=&quot;error&quot;&gt;&amp;#91;Balancer&amp;#93;&lt;/span&gt; collection : summary.hour&lt;br/&gt;
Wed Jul 10 11:04:46.246 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; Request::process begin ns: summary.hour msg id: 39 op: 2004 attempt: 0&lt;br/&gt;
Wed Jul 10 11:04:46.246 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; shard query: summary.hour  { _id.date: &lt;/p&gt;
{ $gte: new Date(1373241600000) }
&lt;p&gt; }&lt;br/&gt;
Wed Jul 10 11:04:46.246 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; creating pcursor over QSpec { ns: &quot;summary.hour&quot;, n2skip: 0, n2return: 0, options: 0, query: { _id.date: &lt;/p&gt;
{ $gte: new Date(1373241600000) }
&lt;p&gt; }, fields: {} } and CInfo { v_ns: &quot;&quot;, filter: {} }&lt;br/&gt;
Wed Jul 10 11:04:46.246 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initializing over 8 shards required by &lt;span class=&quot;error&quot;&gt;&amp;#91;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&amp;#93;&lt;/span&gt;&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11000&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11001&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11002&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11003&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11004&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11005&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.247 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11006&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; initialized query (lazily) on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11010&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11000&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0000:queen0.ozoki.com:11000, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11001&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0001:queen0.ozoki.com:11001, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11002&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0002:queen0.ozoki.com:11002, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11003&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0003:queen0.ozoki.com:11003, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11004&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0004:queen0.ozoki.com:11004, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:46.248 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11005&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.480 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0005:queen0.ozoki.com:11005, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.480 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11006&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.480 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0006:queen0.ozoki.com:11006, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.480 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finishing on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: &lt;/p&gt;
{ conn: &quot;queen0.ozoki.com:11010&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: false, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.481 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; &lt;span class=&quot;error&quot;&gt;&amp;#91;pcursor&amp;#93;&lt;/span&gt; finished on shard shard0010:queen0.ozoki.com:11010, current connection state is { state: &lt;/p&gt;
{ conn: &quot;(done)&quot;, vinfo: &quot;summary.hour @ 8|1||51d464ff93fab1f02c087a5d&quot;, cursor: &quot;(empty)&quot;, count: 0, done: false }
&lt;p&gt;, retryNext: false, init: true, finish: true, errored: false }&lt;br/&gt;
Wed Jul 10 11:04:50.481 &lt;span class=&quot;error&quot;&gt;&amp;#91;conn36&amp;#93;&lt;/span&gt; Request::process end ns: summary.hour msg id: 39 op: 2004 attempt: 0 4234ms&lt;/p&gt;
</comment>
                            <comment id="377927" author="tamer" created="Wed, 10 Jul 2013 11:23:13 +0000"  >&lt;p&gt;David, I&apos;m going to have to tear down and rebuild this db soon due to our inability to provide reporting on our dashboard after 7/4 (due to this bug).  Is there any other info you&apos;d like me to give you before I do that?&lt;/p&gt;</comment>
                            <comment id="377919" author="tamer" created="Wed, 10 Jul 2013 11:13:10 +0000"  >&lt;p&gt;Also, looking at stats -&lt;/p&gt;

&lt;p&gt;mongos&amp;gt; db.hour.stats()&lt;br/&gt;
{&lt;br/&gt;
	&quot;sharded&quot; : true,&lt;br/&gt;
	&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
	&quot;count&quot; : 3316196,&lt;br/&gt;
	&quot;numExtents&quot; : 53,&lt;br/&gt;
	&quot;size&quot; : 1176244000,&lt;br/&gt;
	&quot;storageSize&quot; : 1662603264,&lt;br/&gt;
	&quot;totalIndexSize&quot; : 1677568032,&lt;br/&gt;
	&quot;indexSizes&quot; : &lt;/p&gt;
{
		&quot;_id.date_1__id.ci_1&quot; : 185137344,
		&quot;_id_&quot; : 1492430688
	}
&lt;p&gt;,&lt;br/&gt;
	&quot;avgObjSize&quot; : 354.696767018596,&lt;br/&gt;
	&quot;nindexes&quot; : 2,&lt;br/&gt;
	&quot;nchunks&quot; : 8,&lt;br/&gt;
	&quot;shards&quot; : {&lt;br/&gt;
		&quot;shard0000&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 37235,&lt;br/&gt;
			&quot;size&quot; : 11347744,&lt;br/&gt;
			&quot;avgObjSize&quot; : 304.76014502484225,&lt;br/&gt;
			&quot;storageSize&quot; : 37797888,&lt;br/&gt;
			&quot;numExtents&quot; : 8,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 15290368,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 13065248,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 10661504,
				&quot;_id.date_1__id.ci_1&quot; : 2403744
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0001&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 0,&lt;br/&gt;
			&quot;size&quot; : 0,&lt;br/&gt;
			&quot;storageSize&quot; : 8192,&lt;br/&gt;
			&quot;numExtents&quot; : 1,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8192,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 16352,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 8176,
				&quot;_id.date_1__id.ci_1&quot; : 8176
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0002&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 24672,&lt;br/&gt;
			&quot;size&quot; : 7474128,&lt;br/&gt;
			&quot;avgObjSize&quot; : 302.9396887159533,&lt;br/&gt;
			&quot;storageSize&quot; : 11182080,&lt;br/&gt;
			&quot;numExtents&quot; : 6,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8388608,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 11250176,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 10031952,
				&quot;_id.date_1__id.ci_1&quot; : 1218224
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0003&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 31068,&lt;br/&gt;
			&quot;size&quot; : 7704608,&lt;br/&gt;
			&quot;avgObjSize&quot; : 247.9917600103,&lt;br/&gt;
			&quot;storageSize&quot; : 11182080,&lt;br/&gt;
			&quot;numExtents&quot; : 6,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8388608,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 8920016,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 6949600,
				&quot;_id.date_1__id.ci_1&quot; : 1970416
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0004&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 30174,&lt;br/&gt;
			&quot;size&quot; : 7701200,&lt;br/&gt;
			&quot;avgObjSize&quot; : 255.22635381454234,&lt;br/&gt;
			&quot;storageSize&quot; : 11182080,&lt;br/&gt;
			&quot;numExtents&quot; : 6,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8388608,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 8069712,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 6990480,
				&quot;_id.date_1__id.ci_1&quot; : 1079232
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0005&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 3163919,&lt;br/&gt;
			&quot;size&quot; : 1134186320,&lt;br/&gt;
			&quot;avgObjSize&quot; : 358.4751442751853,&lt;br/&gt;
			&quot;storageSize&quot; : 1580060672,&lt;br/&gt;
			&quot;numExtents&quot; : 19,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 415145984,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 1628013296,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 1450610448,
				&quot;_id.date_1__id.ci_1&quot; : 177402848
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0006&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 29127,&lt;br/&gt;
			&quot;size&quot; : 7829808,&lt;br/&gt;
			&quot;avgObjSize&quot; : 268.816149963951,&lt;br/&gt;
			&quot;storageSize&quot; : 11182080,&lt;br/&gt;
			&quot;numExtents&quot; : 6,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8388608,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 8216880,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 7170352,
				&quot;_id.date_1__id.ci_1&quot; : 1046528
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		},&lt;br/&gt;
		&quot;shard0010&quot; : {&lt;br/&gt;
			&quot;ns&quot; : &quot;summary.hour&quot;,&lt;br/&gt;
			&quot;count&quot; : 1,&lt;br/&gt;
			&quot;size&quot; : 192,&lt;br/&gt;
			&quot;avgObjSize&quot; : 192,&lt;br/&gt;
			&quot;storageSize&quot; : 8192,&lt;br/&gt;
			&quot;numExtents&quot; : 1,&lt;br/&gt;
			&quot;nindexes&quot; : 2,&lt;br/&gt;
			&quot;lastExtentSize&quot; : 8192,&lt;br/&gt;
			&quot;paddingFactor&quot; : 1,&lt;br/&gt;
			&quot;systemFlags&quot; : 1,&lt;br/&gt;
			&quot;userFlags&quot; : 0,&lt;br/&gt;
			&quot;totalIndexSize&quot; : 16352,&lt;br/&gt;
			&quot;indexSizes&quot; : &lt;/p&gt;
{
				&quot;_id_&quot; : 8176,
				&quot;_id.date_1__id.ci_1&quot; : 8176
			}
&lt;p&gt;,&lt;br/&gt;
			&quot;ok&quot; : 1&lt;br/&gt;
		}&lt;br/&gt;
	},&lt;br/&gt;
	&quot;ok&quot; : 1&lt;br/&gt;
}&lt;/p&gt;</comment>
                            <comment id="377720" author="david.hows" created="Wed, 10 Jul 2013 02:12:43 +0000"  >&lt;p&gt;Hi Tamer,&lt;/p&gt;

&lt;p&gt;Can you attach the logs for the MongoS in question? &lt;/p&gt;

&lt;p&gt;That it spends a few seconds running the query only to return nothing is a little strange, I&apos;d like to see what that MongoS reports.&lt;/p&gt;

&lt;p&gt;I&apos;m going to see if I can reproduce this locally in the meantime.&lt;/p&gt;

&lt;p&gt;Thanks for the updated details.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
David&lt;/p&gt;</comment>
                            <comment id="377713" author="tamer" created="Wed, 10 Jul 2013 01:59:58 +0000"  >&lt;p&gt;summary.hour&lt;br/&gt;
			shard key: &lt;/p&gt;
{ &quot;_id&quot; : 1 }
&lt;p&gt;			chunks:&lt;br/&gt;
				shard0001	1&lt;br/&gt;
				shard0003	1&lt;br/&gt;
				shard0004	1&lt;br/&gt;
				shard0006	1&lt;br/&gt;
				shard0000	1&lt;br/&gt;
				shard0002	1&lt;br/&gt;
				shard0005	1&lt;br/&gt;
				shard0010	1&lt;/p&gt;</comment>
                            <comment id="377712" author="tamer" created="Wed, 10 Jul 2013 01:57:31 +0000"  >&lt;p&gt;If I log directly into a shard and run the same find() query I get expected documents returned.  &lt;/p&gt;</comment>
                            <comment id="377707" author="tamer" created="Wed, 10 Jul 2013 01:42:43 +0000"  >&lt;p&gt;Sorry for the discrepancies here.  I was doing a write up, not a direct cut and paste of the queries, so these discrepancies are typos.  The collection name is &quot;hour&quot;.  The date field is under &quot;_id.date&quot; as the result of the map/reduce.  More details to follow.&lt;/p&gt;</comment>
                            <comment id="377702" author="tamer" created="Wed, 10 Jul 2013 01:27:32 +0000"  >&lt;p&gt;Thank you!&lt;/p&gt;




</comment>
                            <comment id="377659" author="david.hows" created="Wed, 10 Jul 2013 00:21:23 +0000"  >&lt;p&gt;Hi Tamer,&lt;/p&gt;

&lt;p&gt;Not a problem and congratulations!&lt;/p&gt;

&lt;p&gt;Il leave this open waiting for you.&lt;/p&gt;

&lt;p&gt;Regards,&lt;br/&gt;
David&lt;/p&gt;</comment>
                            <comment id="377653" author="tamer" created="Wed, 10 Jul 2013 00:10:21 +0000"  >&lt;p&gt;David, please leave this open.  I was offline for the past week (had a baby) and am syncing back up now. &lt;/p&gt;

&lt;p&gt;I dropped the database and rebuilt from scratch and the issue recurred.  I&apos;ll reply to your specific requests later this evening.&lt;/p&gt;






</comment>
                            <comment id="371391" author="david.hows" created="Mon, 1 Jul 2013 06:15:35 +0000"  >&lt;p&gt;Hi Tamer,&lt;/p&gt;

&lt;p&gt;I&apos;ve just looked over what you posted and I&apos;m a little confused. From the output of your sh.status() and the map reduce it appears that you have the &quot;hour&quot; collection sharded. But when you run the two queries above they are on the &quot;hours&quot; collection (note the &quot;s&quot; on the end).&lt;/p&gt;

&lt;p&gt;I&apos;ve also noticed that you run the find against &quot;_id.date&quot; and the count against &quot;date&quot;. Was this deliberate? If not, does the difference in syntax explain the discrepancy?&lt;/p&gt;

&lt;p&gt;You also say that you do get data back from the MongoD when you run these queries? Can you confirm which of the two above queries you run to get these results? &lt;/p&gt;

&lt;p&gt;Would it be possible for you to do the following:&lt;/p&gt;
&lt;ul class=&quot;alternate&quot; type=&quot;square&quot;&gt;
	&lt;li&gt;Confirm if the collection is &quot;hour&quot; or &quot;hours&quot;. If the collection is &quot;hours&quot; can you attach the sharding details for that collection as only &quot;hour&quot; is posted above&lt;/li&gt;
	&lt;li&gt;Confirm the correct query syntax &quot;_id.date&quot; or &quot;date&quot;?&lt;/li&gt;
	&lt;li&gt;Run the correct query on the correct collection via the MongoS and attach the output&lt;/li&gt;
	&lt;li&gt;Run the correct query on the correct collection directly via the MongoD and attach the output&lt;/li&gt;
	&lt;li&gt;Can you provide a sample document from the correct collection mentioned above&lt;/li&gt;
&lt;/ul&gt;


&lt;p&gt;Thanks,&lt;br/&gt;
David&lt;/p&gt;</comment>
                            <comment id="371268" author="tamer" created="Sun, 30 Jun 2013 23:18:24 +0000"  >&lt;p&gt;Hi Ian,&lt;br/&gt;
 Any thoughts on this?  &lt;/p&gt;

&lt;p&gt;Thanks for taking a look.&lt;/p&gt;

&lt;p&gt;Tamer&lt;/p&gt;





</comment>
                    </comments>
                    <attachments>
                    </attachments>
                <subtasks>
                    </subtasks>
                <customfields>
                                                <customfield id="customfield_10050" key="com.atlassian.jira.toolkit:comments">
                        <customfieldname># Replies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>13.0</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                <customfield id="customfield_10055" key="com.atlassian.jira.ext.charting:firstresponsedate">
                        <customfieldname>Date of 1st Reply</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>Mon, 1 Jul 2013 06:15:35 +0000</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10052" key="com.atlassian.jira.toolkit:dayslastcommented">
                        <customfieldname>Days since reply</customfieldname>
                        <customfieldvalues>
                                        10 years, 31 weeks, 2 days ago
    
                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_18254" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Dependencies</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue><![CDATA[]]></customfieldvalue>


                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_15850" key="com.atlassian.jira.plugins.jira-development-integration-plugin:devsummary">
                        <customfieldname>Development</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_10057" key="com.atlassian.jira.toolkit:lastusercommented">
                        <customfieldname>Last comment by Customer</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>true</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_10056" key="com.atlassian.jira.toolkit:lastupdaterorcommenter">
                        <customfieldname>Last commenter</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>ramon.fernandez@mongodb.com</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_11151" key="com.atlassian.jira.toolkit:LastCommentDate">
                        <customfieldname>Last public comment date</customfieldname>
                        <customfieldvalues>
                            10 years, 31 weeks, 2 days ago
                        </customfieldvalues>
                    </customfield>
                                                                                                                        <customfield id="customfield_10000" key="com.atlassian.jira.plugin.system.customfieldtypes:radiobuttons">
                        <customfieldname>Old_Backport</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10000"><![CDATA[No]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10032" key="com.atlassian.jira.plugin.system.customfieldtypes:select">
                        <customfieldname>Operating System</customfieldname>
                        <customfieldvalues>
                                <customfieldvalue key="10020"><![CDATA[Linux]]></customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_10051" key="com.atlassian.jira.toolkit:participants">
                        <customfieldname>Participants</customfieldname>
                        <customfieldvalues>
                                        <customfieldvalue>david.hows</customfieldvalue>
            <customfieldvalue>tamer</customfieldvalue>
    
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                        <customfield id="customfield_14254" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Product Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|hrmosv:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                <customfield id="customfield_12550" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>Rank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>2|hrrerj:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                                                <customfield id="customfield_10558" key="com.pyxis.greenhopper.jira:gh-global-rank">
                        <customfieldname>Rank (Obsolete)</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>73776</customfieldvalue>
                        </customfieldvalues>
                    </customfield>
                                                                                            <customfield id="customfield_23361" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Requested By</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            <customfield id="customfield_10053" key="com.atlassian.jira.ext.charting:timeinstatus">
                        <customfieldname>Time In Status</customfieldname>
                        <customfieldvalues>
                            
                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                                                                                                        <customfield id="customfield_22870" key="com.onresolve.jira.groovy.groovyrunner:scripted-field">
                        <customfieldname>Triagers</customfieldname>
                        <customfieldvalues>
                                

                        </customfieldvalues>
                    </customfield>
                                                                                                                                                                                                                                                                                                                                                                                    <customfield id="customfield_14350" key="com.pyxis.greenhopper.jira:gh-lexo-rank">
                        <customfieldname>serverRank</customfieldname>
                        <customfieldvalues>
                            <customfieldvalue>1|ht014f:</customfieldvalue>

                        </customfieldvalues>
                    </customfield>
                                    </customfields>
    </item>
</channel>
</rss>