[SERVER-38357] mongod crashed when building index and updating data at the same time Created: 03/Dec/18 Updated: 28/Feb/19 Resolved: 28/Feb/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Jack | Assignee: | Danny Hatcher (Inactive) |
| Resolution: | Incomplete | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Operating System: | ALL |
| Participants: |
| Description |
|
I build a replicaset of mongodb 3.4.14 with one primary and two secondary in three VMs. I run the build index command with {backgroup:true} and was updating data on primary at the same time. then the both secondary mongod process crashed. the backtrace are as follows: |
| Comments |
| Comment by Danny Hatcher (Inactive) [ 28/Feb/19 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hello Jack, We haven’t heard back from you for some time, so I’m going to mark this ticket as resolved. If this is still an issue for you, please provide additional information and we will reopen the ticket. Thanks, Danny | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 01/Feb/19 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hello Jack, We still need additional information to diagnose the problem. If this is still an issue for you, would you please respond to the questions posed in my last comment? Thanks, Danny | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 07/Dec/18 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hello Jack, I have spun up a replica set on 3.4.14 and run a series of applyOps commands against one namespace while building a background index on the other. However, I have not been able to reproduce your error. 1. Has this happened multiple times or was it a one-time occurrence? Thank you, Danny | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jack [ 04/Dec/18 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hello Daniel, thanks for your attention. Actually I made a mistake, I thought "there are many update op in this oplog" when I saw the backtrace info involved in so many 'update' message. While after analysed the local.oplog.rs collection in Secondary today, I found out the operations are composed all of the insert operations. So here is the scene: Firstly, I run a build index command on db :
and the corresponding log in mongod.log is:
Then I use applyOps import in some data continuously, the data sample is as follows:
As you can see, those two actions are taken on the same database but the different collections. After a while, the Secondaries crashed and start rebuild index on sun_test.streamLine collection when I restart the process. After confirming the index build action is done, I run the applyOps again with nothing wrong happened and all things turn back to normal. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Danny Hatcher (Inactive) [ 03/Dec/18 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Hello Jack,
Can you please provide example commands for both the index build and your import? Thank you, Danny | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Comment by Jack [ 03/Dec/18 ] | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
"I run the build index command with {backgroup:true} and was updating data on primary at the same time. " sorry for that the description is not that accurate. The accurate scene is "I run the build index command with {backgroup:true} and was using applyOps command importing in some data(a bson format oplog) , and there are many update op in this oplog "
|