[SERVER-54296] Invariant failure | aborting after invariant Created: 04/Feb/21 Updated: 29/Oct/23 Resolved: 15/Mar/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework, Stability |
| Affects Version/s: | 4.4.3 |
| Fix Version/s: | 4.9.0, 4.4.5 |
| Type: | Task | Priority: | Critical - P2 |
| Reporter: | Abdul Moiz Baig | Assignee: | Arun Banala |
| Resolution: | Fixed | Votes: | 3 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||
| Issue Links: |
|
||||
| Backwards Compatibility: | Fully Compatible | ||||
| Backport Requested: |
v4.4
|
||||
| Sprint: | Query Execution 2021-02-22, Query Execution 2021-03-08, Query Execution 2021-03-22 | ||||
| Participants: | |||||
| Description |
|
I have recently upgraded my MongoDB server from 4.2.1 to 4.4.3 to utilize some new features. Since then I am consistently facing the issue of mongod service failure. I have been searching for any solution available for a couple of days but unfortunately couldn't find one. Below are my logs which took out right after my mongod service got failed.
Can you please explain why this service is getting failed or being killed? As my server was working perfectly fine when I was on 4.2.1 |
| Comments |
| Comment by Abdul Moiz Baig [ 17/Mar/21 ] |
|
Arun, Thanks for keeping me posted. |
| Comment by Arun Banala [ 15/Mar/21 ] |
|
abdulmoiz.baig.work@gmail.com We have merged a commit which fixes the reported issue. This fix should be released as part of an upcoming release of 4.4. You can keep an eye on our release page. Thank you for bringing this bug to our notice! |
| Comment by Githook User [ 12/Mar/21 ] |
|
Author: {'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}Message: (cherry picked from commit abf0352882f93e760537d612401eb9fb6e5a030e) |
| Comment by Githook User [ 11/Mar/21 ] |
|
Author: {'name': 'Arun Banala', 'email': 'arun.banala@mongodb.com', 'username': 'banarun'}Message: |
| Comment by Abdul Moiz Baig [ 09/Mar/21 ] |
|
Arun, Thanks. Please keep me posted so I can retry 4.4 again in my environment. |
| Comment by Arun Banala [ 08/Mar/21 ] |
|
abdulmoiz.baig.work@gmail.com We have identified a bug that could be potentially causing this issue. I'm working on a fix for this. I will post an update when I merge the fix into our master branch. |
| Comment by Abdul Moiz Baig [ 02/Mar/21 ] |
|
Arun, Did you find anything? What's causing this issue? Regards, Moiz |
| Comment by Arun Banala [ 19/Feb/21 ] |
|
Thank you for the report, I am looking into this issue. kghazanfar4@gmail.com Are you also using $graphLookup lookup stage? Also is the crash happening consistently? I've been looking at the $graphLookup code, and it looks like we haven't changed the logic since 4.2. I will continue to investigate this issue, and provide an update soon. |
| Comment by Ghazanfar Khan [ 19/Feb/21 ] |
|
I am also facing the same issue after upgrading to 4.4. |
| Comment by Abdul Moiz Baig [ 10/Feb/21 ] |
|
Hello Team, Can you please tell me how we are moving forward with this and what issue was diagnosed? Thanks, Moiz |
| Comment by Edwin Zhou [ 08/Feb/21 ] |
|
Hi abdulmoiz.baig.work@gmail.com, Thank you for providing diagnostic data and keeping us updated with new information! I will be assigning this to the appropriate team for further evaluation. Best, |
| Comment by Abdul Moiz Baig [ 08/Feb/21 ] |
|
Hi Edwin, Today we faced the same issue again in the morning. So we decided to downgrade from 4.4.3 to 4.4.1 but It went down again. So we are facing this issue on 4.4.1 as well which is probably the major release of 4.4. I have uploaded my today's logs on the support uploader(Zip File name: mongod_08_feb_2
Now we are downgrading back to 4.2 release which was working perfectly fine for us. Let us know if you need anything else. Kind Regards, A.Moiz |
| Comment by Abdul Moiz Baig [ 04/Feb/21 ] |
|
Hi Edwin, I have uploaded log files for the days I have faced this issue along with the complete diagnostic.data directory on the support uploader as you requested. The screenshot is attached for reference.
Thanks for a quick response. Really looking forward to hearing back from you about the findings. Don't hesitate to ask If you need anything else to investigate the issue. Kind regards,
|
| Comment by Edwin Zhou [ 04/Feb/21 ] |
|
Hi abdulmoiz.baig.work@gmail.com, Would you please archive (tar or zip) the mongod.log files and the $dbpath/diagnostic.data directory (the contents are described here) and upload them to this support uploader location? Files uploaded to this portal are visible only to MongoDB employees and are routinely deleted after some time. Kind regards, |
| Comment by Abdul Moiz Baig [ 04/Feb/21 ] |
|
Added two screenshots to show my OS details on the server and the specs as well. |