Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-12650

Manual restoration procedure for sharded clusters should mention disableLogicalSessionCacheRefresh

    XMLWordPrintableJSON

Details

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major - P3 Major - P3
    • Server_Docs_20231030
    • 3.6.11
    • Server
    • None
    • 0.5

    Description

      Description

      Starting with MongoDB v3.6 collections are assigned an immutable unique identifier (UUID). The collection UUID remains the same across all members of a replica set and shards in a sharded cluster. That functionality comes into effect only after featureCompatibilityVersion is set to v3.6. UUID is necessary for Logical Sessions and transactions (introduced in v4.0).

      What it also means is that starting with v3.6 an oplog operation can fail if the UUID of the actual collection on the server does not match the UUID of the operation. That could be interpreted as inconsistency and therefore results into an abnormal termination of the `mongod` process (the crash you have experienced).

      It creates a problem with manual restoration of sharded clusters. Specifically, in step 1 we say that the member of a shard needs to be started in standalone for certain manipulations with the metadata. The problem is that:

      • If the user is restoring from an OM/CM snapshot
      • if it takes more than 5 minutes for to keep the member in standalone
        Then the config.systems collection will get created automatically on that member with UUID that will NOT match the rest of the cluster. Later it can cause crashes (see HELP-6628) and possibly some other undefined behaviour.

      The solution is to make sure that when the node is started in standalone for the restore procedure, it is started with the disableLogicalSessionCacheRefresh parameter (undocumented yet) enabled.

      Scope of changes

      Impact to Other Docs

      MVP (Work and Date)

      Resources (Scope or Design Docs, Invision, etc.)

      Attachments

        Activity

          People

            ravind.kumar Ravind Kumar (Inactive)
            dmitry.ryabtsev@mongodb.com Dmitry Ryabtsev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:
              4 years, 39 weeks, 6 days ago