-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
ALL
-
None
-
3
-
None
-
None
-
None
-
None
-
None
-
None
Hi,
We are experiencing extremely slow TTL index deletions in our mongo 3 node setup. The current deletion speed is around 1000 documents per second, with a 60-second sleep interval. Initially, we observed high I/O utilization, so we increased the IOPS from 6K to 16K to improve performance. However, even after increasing IOPS, the deletion rate remains the same at 1000 docs/sec, and there is no improvement in batch deletion speed.
OS: Amazon Linux 2 (aarch64)
MongoDB Version: 7.0.14
{“t”:{“$date”:“2025-03-07T18:53:54.121+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1480,“durationMillis”:1000}}
{“t”:{“$date”:“2025-03-07T18:54:00.142+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1540,“durationMillis”:1006}}
{“t”:{“$date”:“2025-03-07T18:54:08.177+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1550,“durationMillis”:1004}}
{“t”:{“$date”:“2025-03-07T18:54:12.189+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1270,“durationMillis”:1004}}
{“t”:{“$date”:“2025-03-07T18:54:17.212+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1560,“durationMillis”:1005}}
{“t”:{“$date”:“2025-03-07T18:54:23.225+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1570,“durationMillis”:1001}}
{“t”:{“$date”:“2025-03-07T18:54:25.234+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1580,“durationMillis”:1002}}
{“t”:{“$date”:“2025-03-07T18:54:27.246+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1520,“durationMillis”:1003}}
{“t”:{“$date”:“2025-03-07T18:54:29.254+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1560,“durationMillis”:1005}}
{“t”:{“$date”:“2025-03-07T18:54:35.292+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1070,“durationMillis”:1007}}
{“t”:{“$date”:“2025-03-07T18:54:50.337+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1540,“durationMillis”:1002}}
{“t”:{“$date”:“2025-03-07T18:55:03.417+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1570,“durationMillis”:1002}}
{“t”:{“$date”:“2025-03-07T18:55:06.424+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1530,“durationMillis”:1001}}
{“t”:{“$date”:“2025-03-07T18:55:23.489+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1330,“durationMillis”:1001}}
{“t”:{“$date”:“2025-03-07T18:55:42.551+00:00”},“s”:“I”, “c”:“INDEX”, “id”:5479200, “ctx”:“TTLMonitor”,“msg”:“Deleted expired documents using index”,“attr”:{“namespace”:“prod.worker”,“index”:“expiresAt_1”,“numDeleted”:1490,“durationMillis”:1007}}
Is this expected behavior for TTL index deletion in MongoDB? Are there any ways to optimize or speed up the TTL deletion process? Additionally, we would like to understand why increasing IOPS did not have any effect on the deletion rate. Kindly help on this.
- is related to
-
SERVER-63040 Batch TTL deletions
-
- Closed
-