Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-42827

Allow sessions collection to return OK for creating indexes if at least one shard returns OK and others return CannotImplicitlyCreateCollection

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.6.13, 4.2.0-rc8, 4.0.12
    • Fix Version/s: 4.3.1, 4.2.6
    • Component/s: Sharding
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v4.2
    • Sprint:
      Sharding 2019-09-23
    • Case:

      Description

      Issue Status as of April 13, 2020

      ISSUE DESCRIPTION AND IMPACT

      This bug prevents config server replica set primaries from creating new sessions, resulting in a loss of availability of the node after an internal limit of 1 million sessions are created. Unfortunately, this limit is eventually reached independent of the number of active sessions at any given point.

      The impact occurs because the bug prevents the node from refreshing its in-memory logical session cache from the persisted config.system.sessions collection. This failure to refresh prevents the lastUsed TTL index on config.system.sessions from removing session records, and makes reaching 1 million in-memory sessions likely.

      Ultimately, the underlying cause is that TTL index creation failure on shards that do not contain config.sessions chunks ends up halting the synchronization process that refreshes the in-memory logical session cache.

      DIAGNOSIS AND AFFECTED VERSIONS

      MongoDB versions 4.2.0 to 4.2.5 are impacted by this bug. Signs that the issue is occurring include:

      • Sharding commands fail
      • Chunk migrations fail
      • Inability to access the config server primary

      REMEDIATION AND WORKAROUNDS

      To remediate loss of availability, kill and re-start the config server primary and allow replica set failover to reset the session cache on the new primary.

      Setting maxSessions to a higher number than the default of 1 million can delay the onset of this issue.

      FIX VERSIONS

      4.2.6

      Misha Tyulenev, LGTY?

      original description

      The cluster's createIndexes command allows for cannotImplicitlyCreateCollection to be returned as a valid error as long as one shard succeeds to create the index. This is because shards that don't have the collection locally won't be able to create it. We should change the session collection's createIndexes call to match this behavior.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              janna.golden Janna Golden
              Reporter:
              blake.oler Blake Oler
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: