Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-10056

Shard not returning some data when querying from MongoS

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • None
    • 2.4.1, 2.4.4
    • Sharding
    • None
    • linux, AWS
    • Linux

    Description

      Setup: 30 shards, no replication, AWS Hi1.4x Large with SSD high IOPs
      Database: sharded ouput of a routine map reduce operation (merged)
      Problem: Some of the records of this sharded collection will only return when I'm logged into the particular shard. In MongoS, I get no results, except when I do a count(), then I get an expected return. Example:

      I run the following query on MongoS -

      var d = new Date(2013,5,26);
      db.hours.find({"_id.date" : {$gt : d}})

      I expect several records but nothing returns. It just goes back to a blank command line.
      However, when I run a count on the same query, I get the expected result -

      mongos> db.hours.find({"date" : {$gt : d}}).count()
      11318035

      When I log directly into the MongoD shard that has this date range of records and run the query there I get a cursor of documents back, as expected. It appears I can only access these records by logging into the particular shard directly. This is the case for all records after the 25th of June.

      The map reduce query that creates and updates this sharded collection is as follows:

      //MAP REDUCE COMMAND
      db.sessions.mapReduce (
      mapFxn,
      reduceFxn,
      { out :

      { merge : "hour", db : "summary", sharded : true, nonAtomic : true }

      ,
      query: { "date" : {$gte : lastRunTime, $lte : transactionTime}}

      }
      );

      -----------------------------------------
      Looking at my sharding status -

      summary.hour
      shard key:

      { "_id" : 1 }

      chunks:
      shard0001 3
      shard0009 3
      shard0002 3
      shard0010 3
      shard0003 3
      shard0011 3
      shard0004 3
      shard0005 3
      shard0014 3
      shard0013 3
      shard0008 3
      shard0012 3
      shard0015 3
      shard0016 2
      shard0017 2
      shard0018 2
      shard0019 2
      shard0020 2
      shard0021 2
      shard0022 2
      shard0023 2
      shard0024 2
      shard0025 2
      shard0026 2
      shard0027 2
      shard0028 2
      shard0029 2
      shard0030 2
      shard0031 2
      shard0006 3
      shard0007 3
      shard0000 3
      too many chunks to print, use verbose if you want to force print

      Shard0000 is the primary shard and also the shard that contains these records. Here is a sample of the verbose listing of chunks for that collection. The date key does not seem to be updating past the 25th of June, even though there are records all the way through the 27th -

      on : shard0000

      { "t" : 77, "i" : 2 }

      {

      "_id" :

      { "date" : ISODate("2013-06-18T02:00:00Z"), "ci" : "4352", "site" : "foo.com", "page" : "home", "Origin_Domain" : "bar.com", "pl" : "pub", "v" : "1.6", } }

      -->> {

      "_id" :

      { "date" : ISODate("2013-06-25T11:00:00Z"), "ci" : "4543", "site" : "foo.com", "page" : "home", "Origin_Domain" : "unavailable", "pl" : "pub", "v" : "1.6" }

      } on : shard0000

      { "t" : 77, "i" : 3 }

      {

      "_id" :

      { "date" : ISODate("2013-06-25T11:00:00Z"), "ci" : "3453", "site" : "bar.com", "page" : "home", "Origin_Domain" : "unavailable", "pl" : "pub", "v" : "1.6" }

      } -->> { "_id" :

      { "$maxKey" : 1 }

      } on : shard0012

      { "t" : 76, "i" : 0 }

      { "_id" : "summmary", "partitioned" : false, "primary" : "shard0021" }

      ========================

      Attachments

        Activity

          People

            david.hows David Hows
            tamer Tamer Hassan
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: