Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-46640

The TTLMonitor can remove the cached index information from the TTLCollectionCache for a newly created index before the implicitly created collection is registered in the CollectionCatalog

XMLWordPrintableJSON

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 4.4.0-rc0, 4.7.0
    • Affects Version/s: 4.4.0-rc0, 4.5.1
    • Component/s: Storage
    • None
    • Fully Compatible
    • ALL
    • v4.4
    • Hide

      Remove these lines in ttl.cpp: https://github.com/mongodb/mongo/blob/3e30e2f0705cbb167d28cfe6f5344c354f2e9442/src/mongo/db/ttl.cpp#L94-L97

      Run `resmoke --suites=no_passthrough jstests/noPassthrough/ttl_partial_index.js`

      It should fail on the first execution, if not, just run it again with --opt=on.

      Show
      Remove these lines in ttl.cpp: https://github.com/mongodb/mongo/blob/3e30e2f0705cbb167d28cfe6f5344c354f2e9442/src/mongo/db/ttl.cpp#L94-L97 Run `resmoke --suites=no_passthrough jstests/noPassthrough/ttl_partial_index.js` It should fail on the first execution, if not, just run it again with --opt=on.
    • Execution Team 2020-03-23
    • 13

      The TTLMonitor uses the CollectionCatalog to look-up the collection namespace given a UUID here (prior to obtaining any collection locks). If the collection with the given UUID does not exist in the CollectionCatalog, the cached information gets removed from the TTLCollectionCache for that index. This will prevent the TTLMonitor from ever running on that index, and as a consequence, the TTLMonitor won't remove any expired documents in the collection for the given index unless it is recreated, which would cause it to be registered in the TTLCollectionCache again.

       

      Based on these logs, when creating a new index that requires the collection to be created implicitly, the index will be registered in the TTLCollectionCache before the collection is registered with the CollectionCatalog. If the TTLMonitor were to run between these two events, it would think that the collection for that index was dropped, and so it will deregister that index from the TTLCollectionCache.

       

       

      {"t":{"$date":"2020-03-05T09:08:09.268-0500"},"s":"I", "c":"STORAGE", "id":20320,"ctx":"conn1","msg":"createCollection: {nss} with {generatedUUID_generated_provided} UUID: {optionsWithUUID_uuid_get} and options: {options}","attr":{"nss":"test.ttl_partial_index","generatedUUID_generated_provided":"generated","optionsWithUUID_uuid_get":{"uuid":{"$uuid":"4a58143c-819c-43bd-890e-368d0389dc4d"}},"options":{}}}
{"t":{"$date":"2020-03-05T09:08:09.268-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
...
{"t":{"$date":"2020-03-05T09:08:09.298-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
{"t":{"$date":"2020-03-05T09:08:09.319-0500"},"s":"I", "c":"INDEX",   "id":20345,"ctx":"conn1","msg":"index build: done building index {indexName} on ns {nss}","attr":{"indexName":"_id_","nss":"test.ttl_partial_index"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
...
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"conn1","msg":"{}","attr":{"message":"+++: Registering in TTLCollectionCache { v: 2, key: { x: 1.0 }, name: \"x_1\", expireAfterSeconds: 0.0, partialFilterExpression: { z: { $exists: true } } }"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 1"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: UUID 4a58143c-819c-43bd-890e-368d0389dc4d indexName x_1"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: deregistering collection from ttlCollectionCache"}}
{"t":{"$date":"2020-03-05T09:08:09.322-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
...
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":20345,"ctx":"conn1","msg":"index build: done building index {indexName} on ns {nss}","attr":{"indexName":"x_1","nss":"test.ttl_partial_index"}}
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
...
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"STORAGE", "id":0,"ctx":"conn1","msg":"{}","attr":{"message":"+++: registering collection 4a58143c-819c-43bd-890e-368d0389dc4d test.ttl_partial_index"}}
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}
...
{"t":{"$date":"2020-03-05T09:08:09.352-0500"},"s":"I", "c":"INDEX",   "id":0,"ctx":"TTLMonitor","msg":"{}","attr":{"message":"+++: length of ttlInfos: 0"}}

            Assignee:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Reporter:
            gregory.wlodarek@mongodb.com Gregory Wlodarek
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: