Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-3266

mongos consistently locks up distributing parallel updates to multiple shards -- cluster unusable

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker - P1 Blocker - P1
    • 1.8.3
    • 1.8.1, 1.8.2, 1.8.3
    • Concurrency, Sharding, Stability
    • None
    • centos, two machines, four shards each, 30 simultaneous updateres
    • Linux

    Description

      a data set of 120m docs in one collection is being updated to add data to each document.

      every few million records processed (mixed reads and writes, many fewer writes), mongos becomes unresponsive. this number has some down from 20m gradually until it now locks after only a few million.

      queries against each of the shard and config mongod instance show them to be responsive to requests

      queries against mongos hang indefinitely as does db.stats().

      gdb stack trace (attached) shows many threads in mongo::ChunkManager::getShardsForQuery waiting to obtain mongo::rwlock::rwlock

      Attachments

        Activity

          People

            greg_10gen Greg Studer
            rgiar Rob Giardina
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: