Uploaded image for project: 'MongoDB Database Tools'
  1. MongoDB Database Tools
  2. TOOLS-3679

Investigate changes in SERVER-82221: listCollections and listIndexes should include commit-pending namespaces

    • Type: Icon: Investigation Investigation
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Tools and Replicator
    • 10

      Original Downstream Change Summary

      As of SERVER-82221, listCollections may return results in non-sorted order. The listCollections documentation page does not state that the output is sorted, and its sample output is unsorted. That being said, the implementation provided incidentally sorted output until SERVER-82221, which introduced a rare case where the bottom entries may not be sorted. This downstream notification is to ensure that our internal usages do not rely on any implicit ordering of the output.

      Description of Linked Ticket

      listCollections and listIndexes do not include commit-pending metadata. This can race with initial sync: the clone phase may fail to detect a collection or index that is pending commit (specifically, the storage transaction has committed, but the catalog changes are yet to be published to the in-mem catalog), while the beginApplyingTimestamp may point to that catalog change because the storage transaction has committed.
       
      We have observed a failure where an initial syncing node did not clone a collection that was implicitly created via index creation (two oplog entries, one for collection creation, one for index creation). The beginApplyingTimestamp was set to the optime of the index creation, and listCollections failed to find the collection because it had yet to be published in the in-mem catalog.
       
      This issue is unlikely to be observed in the field - it requires a careful combination of implicit collection creation + slow commit handlers + initial sync pointing the beginApplyingTimestamp to the creation optime. It has also probably existed since the dawn of time (I've reproduced on 6.0 onwards).
       
      listCollections and listIndexes always read at latest; one solution would be to include commit-pending entries to their output.

            Assignee:
            Unassigned Unassigned
            Reporter:
            backlog-server-pm Backlog - Core Eng Program Management Team
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: