Investigate eligible, non-oplog collections absent in initializeMetadata()

ExportXMLWordPrintableJSON

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Storage Execution
    • ALL
    • Hide

      Run the following commands:

      $ bazel build install-dist-test
$ buildscripts/resmoke.py run --suite=no_passthrough jstests/noPassthrough/cluster_scalability_misc/catalog_shard.js --additionalFeatureFlags=featureFlagReplicatedFastCount

      Then, grep for the log ID 12580001.

      I also ran an Evergreen patch with the LOGV2_ERROR/WARNING replaced with a LOGV2_FATAL, thus causes any tasks that encounter this log to fail.

       

      Show
      Run the following commands: $ bazel build install-dist-test $ buildscripts/resmoke.py run --suite=no_passthrough jstests/noPassthrough/cluster_scalability_misc/catalog_shard.js --additionalFeatureFlags=featureFlagReplicatedFastCount Then, grep for the log ID 12580001. I also ran an Evergreen patch with the LOGV2_ERROR/WARNING replaced with a LOGV2_FATAL, thus causes any tasks that encounter this log to fail.  
    • Storage Execution 2026-06-08, Storage Execution 2026-06-22, Storage Execution 2026-07-06, Storage Execution 2026-07-20, Storage Execution 2026-08-03, Storage Execution 2026-08-17
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      In SERVER-125800, I added the following log to initializeMetadata():

      LOGV2_WARNING(
    12580001,
    "Replicated fast count eligible namespace found in the collection "
    "catalog but not tracked in the metadata store or the oplog",
    "nss"_attr = coll->ns().toStringForErrorMsg(),
    "uuid"_attr = coll->uuid());

      We expect this log to not be triggered. In general, we expect that the accumulator map in initializeMetadata() contains entries for every replicated fast count eligible collection in the catalog except the oplog. On ASC, the oplog has a different UUID on every node, so we won't have the old oplog UUID in the catalog.

      However, in practice, there are sometimes five collections absent from accumulator:

      1 [js_test:catalog_shard] c20042| {"t":{"$date":"2026-05-08T11:02:23.081-04:00"},"s":"E",  "c":"STORAGE",  "id":12580001,"svc":"S", "ctx":"initandlisten","msg":"Replicated fast count eligible namespace found in the collection catalog but not tracked in the              metadata store or the oplog","attr":{"nss":"admin.system.keys","uuid":{"uuid":{"$uuid":"60b7aa30-b63d-47f9-a39a-86b455440914"}}}}
      2 [js_test:catalog_shard] c20042| {"t":{"$date":"2026-05-08T11:02:23.081-04:00"},"s":"E",  "c":"STORAGE",  "id":12580001,"svc":"S", "ctx":"initandlisten","msg":"Replicated fast count eligible namespace found in the collection catalog but not tracked in the              metadata store or the oplog","attr":{"nss":"config.sampledQueries","uuid":{"uuid":{"$uuid":"44cfda13-dad8-4ef9-b79c-5fc8bf7c238f"}}}}
      3 [js_test:catalog_shard] c20042| {"t":{"$date":"2026-05-08T11:02:23.081-04:00"},"s":"E",  "c":"STORAGE",  "id":12580001,"svc":"S", "ctx":"initandlisten","msg":"Replicated fast count eligible namespace found in the collection catalog but not tracked in the              metadata store or the oplog","attr":{"nss":"config.reshardingOperations","uuid":{"uuid":{"$uuid":"a2dd6189-7481-4d34-8d86-6bcede80571a"}}}}
      4 [js_test:catalog_shard] c20042| {"t":{"$date":"2026-05-08T11:02:23.081-04:00"},"s":"E",  "c":"STORAGE",  "id":12580001,"svc":"S", "ctx":"initandlisten","msg":"Replicated fast count eligible namespace found in the collection catalog but not tracked in the              metadata store or the oplog","attr":{"nss":"config.sampledQueriesDiff","uuid":{"uuid":{"$uuid":"a3cbf51d-2721-4fad-baa6-9a3e738f4555"}}}}
      5 [js_test:catalog_shard] c20042| {"t":{"$date":"2026-05-08T11:02:23.081-04:00"},"s":"E",  "c":"STORAGE",  "id":12580001,"svc":"S", "ctx":"initandlisten","msg":"Replicated fast count eligible namespace found in the collection catalog but not tracked in the              metadata store or the oplog","attr":{"nss":"config.analyzeShardKeySplitPoints","uuid":{"uuid":{"$uuid":"e1808b48-6643-488c-b86a-fe3fddf90a77"}}}}

      When an entry is not in accumulator, it means that (1) the fast count store does not contain an entry for that UUID and (2) the oplog (since the valid-as-of timestamp in the timestamp store) does not contain an entry for that UUID.

      Investigate why the UUIDs for the above collections are not found during initialization. Is this expected? Should we convert the log to debug or remove it?

            Assignee:
            Unassigned
            Reporter:
            Cedric Sirianni
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: