[SERVER-84924] Ensure that 7.0.0 release does not have any applicable critical or high Coverity issues Created: 31/Jul/23  Updated: 12/Jan/24  Resolved: 26/Sep/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Critical - P2
Reporter: Judah Schvimer Assignee: Kyle Suarez
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File coverity-high-impact-outstanding.png     PNG File coverity-outstanding-issues-all.png     PNG File coverity-outstanding-issues-major.png    
Issue Links:
Depends
Related
Sprint: QE 2023-09-04, QE 2023-09-18, QE 2023-10-02
Participants:

 Description   

We should include a screenshot that includes the date of the check, similar to WRITING-14730, for documentation.



 Comments   
Comment by Salman Baset [ 22/Sep/23 ]

kyle.suarez@mongodb.com I have reviewed the revised report and it looks good to me. Thank you so much for completing this. This ticket can be closed.

Comment by Brooke Miller [ 21/Sep/23 ]

salman.baset@mongodb.com can this be considered resolved and done?

kyle.suarez@mongodb.com If we don't hear back from Salman, I think this can be closed. The other doc requested (Third party dependencies validation WRITING-14730) didn't go through this review process.

Comment by Kyle Suarez [ 05/Sep/23 ]

salman.baset@mongodb.com, any request for further changes?

Comment by Brooke Miller [ 18/Aug/23 ]

TY. I assumed that salman.baset@mongodb.com will copy the entirety of the "Revised Report" sheet. I agree with removing the "Internal Notes" column from that sheet.

Comment by Kyle Suarez [ 18/Aug/23 ]

Thanks brooke.miller@mongodb.com! I made some minor adjustments:

  • Coverity scans the enterprise modules so I also added "MongoDB Server Enterprise" to the Products line
  • Slight tweak to the description of "No Fix Needed" to include shadowed lambda variables as another example of a "cosmetic issue"

One question I have for you: is this the exact report we will give to IBM? Or are we going to just copy-paste the final values into a new report? I have a preference for the latter; maybe we need to also remove the internal notes?

Comment by Brooke Miller [ 18/Aug/23 ]

Thanks, kyle.suarez@mongodb.com! I modified the intro of the Revised Report tab to specify the product this applies to (MongoDB Server Community) and the tool used (Coverity) and updated the format of the legend just to align exactly with the details in the http://go/ssdlc-policy.

This LGTM.

salman.baset@mongodb.com can you please review and comment with an LGTM by next Wednesday so that Kyle can close this out? Thanks!

Comment by Kyle Suarez [ 16/Aug/23 ]

I went in and analyzed the 30 remaining issues:

  • 28 of them are no longer issues because the offending code in question is no longer in mainline. (I suspect that these Coverity issues were actually duplicates – there's no way currently in Coverity to mark something as a duplicate so they get set to "Pending".) I've marked them all as "No Fix Needed".
  • One of them (CID 134149) was actually complete. I tracked down the WiredTiger ticket (WT-10190) and realized it was missing the coverity label and did not have the CID in the title; I updated the ticket and added it to the "Jira Tickets Fixed" tab.
  • The last issue (CID 133975) is an Impact=Low redundant test issue that is still open (WT-9993). This ticket was also missing the coverity label which I added in and also linked the ticket in the final report.

Sending this back to review.

Comment by Kyle Suarez [ 15/Aug/23 ]

Moving this back in progress – I will investigate the 30 issues with a missing Jira ticket.

Comment by Brooke Miller [ 15/Aug/23 ]

Hey kyle.suarez@mongodb.com, after our review meeting today I added a new tab ("Revised Report"). A few updates:

  • Updated Coverity issues to the applicable status according to our revised status categories (Fix Committed, No Fix Needed, False Positive, and Fix Pending)
  • Included the relevant Jira ticket per Coverity issue
  • Verified that tickets with "Fixed Committed" status indeed had a Jira ticket for the issue which had Fixed, Done or Duplicate Resolutions. If Duplicate, I verified that the linked Duplicate ticket was marked as Fixed, Done.

However, I was unable to find Jira tickets for 30 Coverity issues previously marked as "Fix Committed". (29 of these issues are either High or Medium Impact). If no Jira tickets were generated for them, I assume we may need to generate Jira tickets to verify these issues indeed had a fix committed for 7.0.0, or to determine no fix is needed. Do you think that's feasible, or do you have an alternative suggestion?

Comment by Kyle Suarez [ 14/Aug/23 ]

Legend is complete and I also did another sweep through the sheet. Moving back to review.

Comment by Kyle Suarez [ 11/Aug/23 ]

I just noticed the comment by Salman and Brooke that

We also need to provide explanation of the classification status in this document and how that maps to Action status of "No action"/

I'll move this back into Open and I'll get this done next week before the meeting.

Comment by Kyle Suarez [ 11/Aug/23 ]

brooke.miller@mongodb.com, I misspoke, it's actually just 5 issues.

Are those... Unresolved Issues attributed to a High or Medium Impact? (You mentioned Minor but I didn't follow whether that was related to the Jira Priority or Coverity's Low Impact.)

All 5 issues are "P4 - Minor" in Jira. In Coverity, four of the five issues are Impact = "Low", so those are fine. The fifth and final issue (SERVER-70518) is a Medium, but I did some code inspection and actually realized the issue is fixed, so I commented as such in the ticket. (Coverity's own system actually labeled this issue as "fixed" because it correctly determined the old problematic code no longer exists.)

I'd recommend including them for now so that we can have visibility. If they are High / Medium impact, I'd like to understand whether those issues are actively in progress and whether they will be fixed for 7.0.0?

I added all five issues to the very bottom of the sheet. The four low-impact unresolved tickets say "Fix Pending", and the one medium-impact ticket I said "Done" since the code no longer has the defect.

Can you possibly include the instructions explaining what filters you used to generate this report from Coverity so that we don't lose insight into this?

I am happy to make a basic guide on how I generated this spreadsheet. (Probably best to track this with another WRITING ticket?) But if we sincerely think that we'll be moving off Coverity onto a different system, then I'd prefer not to make it super detailed if that work will eventually be obviated.

Also, re: the Jira tickets: how were you able to realize which Coverity Issues mapped to the relevant Jira issue to update the correct Coverity issue ID on the first tab? Just based on the CID being included in the Jira Issue Summary?

Yes, each Jira ticket title has the CID, and then I search for the CID in both the existing exported table as well as looking it up in the Coverity system itself to see what it has to say about that particular defect.

Comment by Brooke Miller [ 11/Aug/23 ]

Thanks kyle.suarez@mongodb.com! Are those 8 Unresolved Issues attributed to a High or Medium Impact? (You mentioned Minor but I didn't follow whether that was related to the Jira Priority or Coverity's Low Impact.)

I'd recommend including them for now so that we can have visibility. If they are High / Medium impact, I'd like to understand whether those issues are actively in progress and whether they will be fixed for 7.0.0?

Also, sorry for this manual effort on your behalf. Can you possibly include the instructions explaining what filters you used to generate this report from Coverity so that we don't lose insight into this? Also, re: the Jira tickets: how were you able to realize which Coverity Issues mapped to the relevant Jira issue to update the correct Coverity issue ID on the first tab? Just based on the CID being included in the Jira Issue Summary?

Comment by Kyle Suarez [ 11/Aug/23 ]

OK, updates to the sheet:

  • I hid the columns "Owner" and "Severity" because the values are always "Unassigned" and "Unspecified" anyway. (We can choose to fully delete them instead of just hiding them if we feel they're not adding any value.)
  • There is a new column "Classification / Action" which merges together those original two columns so that there's always a meaningful string.
  • I added a conditional validation rule that would flag any ticket with a final "Classification / Action" of either "Unspecified" or "Undecided", then went in to add an appropriate action.
  • Regarding the "Pending" ones: it turns out that all of the Coverity tickets listed on the "All Issues" sheet are only including ones that are resolved. That means it doesn't include the 8 pending tickets that are unresolved. Therefore I went in and replaced all the "Pending" ones such that the action is "Done".
    • Given that the 8 remaining issues are not represented on the sheet, would you like me to add them and say they are still "Fix Pending"?
Comment by Kyle Suarez [ 11/Aug/23 ]

brooke.miller@mongodb.com and salman.baset@mongodb.com:

What do you think about consolidating the Classification and Action columns? e.g. False Positive classification and Action of "Undecided" seems confusing? Otherwise, we should update the Action column to make "Undecided" more useful.

Sounds good to me and makes sense. I can write a formula to merge the two columns – my proposal is to pick Classification first, and fall back to Action if Classification=Undecided.

For Impacts other than Low: if the classification is not "Fix Required", then the Action should be "No action" instead of "Undecided"

Agreed.

Similarly, we noticed quite a few "Pending" statuses for High / Medium impacted issues. Do those investigations need to be complete still, or is that information outdated?

Sadly that information is outdated. Basically, when I export a Medium/High+ ticket from Coverity to Jira, the classification is generally "Pending". But when the Jira ticket is resolved, the system does not automatically go back and then update the Coverity status to "Done". Essentially, anything that wasn't a false positive or intentional can be considered "Done" except for the 8 outstanding minor tickets. I can go back and essentially mark them all as fixed but then change the 8 outstanding ones to "Pending" or "Awaiting Fix" or something of the sort.

I'll have all these changes made by our meeting and we can discuss / review further.

Comment by Brooke Miller [ 11/Aug/23 ]

Hey kyle.suarez@mongodb.com, I talked with Salman about this and we had a few takeaways.

We updated the http://go/ssdlc-policy description to be more specific:
A report that contains the list of all False positive issues and open Low issues, and validation that there are no unresolved Critical, High or Medium issues. For each False Positive and open Low issue, the report contains the name of the issue, date the issue was reported by the tool, and the severity identified by the tool

To align with that:

  • What do you think about consolidating the Classification and Action columns? e.g. False Positive classification and Action of "Undecided" seems confusing? Otherwise, we should update the Action column to make "Undecided" more useful. We wouldn't expect to see any "Undecided" High / Medium impacted issues as the expectation is that all High and Medium impacted issues are resolved.
  • For Impacts other than Low: if the classification is not "Fix Required", then the Action should be "No action" instead of "Undecided"
  • Similarly, we noticed quite a few "Pending" statuses for High / Medium impacted issues. Do those investigations need to be complete still, or is that information outdated?

I'll find time next week with you, Salman and Judah (optionally) to get closure on this (thanks for the ping!).

Comment by Kyle Suarez [ 11/Aug/23 ]

salman.baset@mongodb.com, brooke.miller@mongodb.com and judah.schvimer@mongodb.com: following up on this after my vacation, is there anything else y'all need for the reports and/or the spreadsheets or is this sufficient for closeout?

Comment by Kyle Suarez [ 02/Aug/23 ]

Adding a link to the Coverity 7.0 Report.

Comment by Kyle Suarez [ 02/Aug/23 ]

salman.baset@mongodb.com, correct, the only issues that are outstanding in Jira are low-priority ones, and all of the high-priority ones have been fixed.

Comment by Salman Baset [ 02/Aug/23 ]

kyle.suarez@mongodb.com thank you so much for sharing the screenshots. So if I understand correctly, for 7.0.0, Coverity is only reporting low issues that are not yet fixed, right?

It would be helpful to generate a Coverity CSV list similar to this one for 6.0
https://docs.google.com/spreadsheets/d/1OgEcsKpAingUJ8GQprVqDg0N2Eu4B2rMmcAdT6jG5C0/edit#gid=1778483548

Comment by Kyle Suarez [ 02/Aug/23 ]

salman.baset@mongodb.com, tagging you here to please review the screenshots. I figure I will also go ahead and generate the usual CSV we give to IBM and will share that with you when it is ready.

Comment by Kyle Suarez [ 02/Aug/23 ]

In the Coverity interface itself, here are the outstanding high impact issues. There is only one issue listed, and a fix was committed as part of SERVER-78538 / WT-11252.

Comment by Kyle Suarez [ 02/Aug/23 ]

This screenshot verifies that we have no outstanding issues in Coverity that are severity: Major or higher:

The next screenshot shows all of the remaining unresolved Coverity issues, all of which are Minor or below:

This snapshot was taken on Wednesday, August 2.

Generated at Thu Feb 08 06:56:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.