[SERVER-60959] Insert to a Time-Series getting error collection ErrorCodes::TimeseriesBucketCleared Created: 25/Oct/21  Updated: 29/Oct/23  Resolved: 15/Dec/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 5.0.3
Fix Version/s: 5.3.0, 5.0.6, 5.2.0-rc5

Type: Bug Priority: Major - P3
Reporter: hung hoang Assignee: Dan Larkin-York
Resolution: Fixed Votes: 0
Labels: Bug
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Duplicate
is duplicated by SERVER-60960 Insert to a Time-Series getting error... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v5.2, v5.0
Steps To Reproduce:

Client side:

  • Using golang driver
  • Setup 6 concurrent connections to insert continuously to a Time-Series collections using InsertMany
  • Each batch contains about 100k documents

Server side:

  • Mongo version 5.0.3
  • Replica-set: 1 primary, 1 secondary
Sprint: Execution Team 2021-12-27
Participants:

 Description   

My service has 6 threads and concurrently write to a Time-Series collection using InsertMany. My mongo server is a replica set with one primary and one secondary. After few successful insertions, in the mongo db server log I'm getting this error then the server is shutdown. And I did use any transaction in my code.

{{

{"t":\{"$date":"2021-10-25T16:24:03.520+07:00"}

,"s":"F", "c":"-", "id":23081, "ctx":"conn66","msg":"Invariant failure","attr":{"expr":"batchStatus == ErrorCodes::TimeseriesBucketCleared || batchStatus.isA<ErrorCategory::Interruption>()","msg":"Unexpected error when aborting time-series batch: WriteConflict: WriteConflict error: this operation conflicted with another operation. Please retry your operation or multi-document transaction.","file":"src/mongo/db/timeseries/bucket_catalog.cpp","line":381}}}}

What is happening to my mongo server and how can I solve this? I try to google it but did have any luck.



 Comments   
Comment by Githook User [ 07/Jan/22 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-60959 Remove assumptions about error type when aborting a time-series insert batch
Branch: v5.0
https://github.com/mongodb/mongo/commit/85f2edddcb347e5ebcc1189d46cf4574340e98bc

Comment by Githook User [ 06/Jan/22 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-60959 Remove assumptions about error type when aborting a time-series insert batch
Branch: v5.2
https://github.com/mongodb/mongo/commit/746e60c44001d9222a73a4bfd591d8fee8d45879

Comment by Githook User [ 15/Dec/21 ]

Author:

{'name': 'Dan Larkin-York', 'email': 'dan.larkin-york@mongodb.com', 'username': 'dhly-etc'}

Message: SERVER-60959 Remove assumptions about error type when aborting a time-series insert batch
Branch: master
https://github.com/mongodb/mongo/commit/3172c17c2ae36a4e0e78528fcfed316a4f9bbfb7

Comment by Edwin Zhou [ 16/Nov/21 ]

Hi hunghoang@chotot.vn,

We still need additional information to diagnose the problem. If this is still an issue for you, would you please attach logs from 2021-10-25 or which cover a repeat incident?

Best,
Edwin

Comment by Edwin Zhou [ 03/Nov/21 ]

Hi hunghoang@chotot.vn,

Thanks for your report. I attempted to reproduce this issue with 25 concurrent connections inserting 100k documents into a time-series collection using insertMany() but was not able to crash the server.

Would you please archive (tar or zip) the mongod.log files and the $dbpath/diagnostic.data directory (the contents are described here) and upload them to this support uploader location?

Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time.

Best,
Edwin

Generated at Thu Feb 08 05:51:11 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.