[SERVER-82200] burn_in_tags should include indirectly affected tests Created: 14/Oct/23 Updated: 29/Jan/24 Resolved: 15/Nov/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.3.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Tommaso Tocci | Assignee: | Mikhail Shchatko |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
||||||||||||
| Issue Links: |
|
||||||||||||
| Assigned Teams: |
Correctness
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Participants: | |||||||||||||
| Description |
|
Currently, burn_in_tags only run js tests that have been explicitly modified in the patch. On the other hand, if the patch affects a base FSM test, brun_in_tags will only run that base FSM and will not test all the FSM test derived from that one. We have a similar issue also with js library modifications, in fact when a patch impacts a js test library, burn_in_tags will not execute any tests that use/include that library. To improve js dependency discovery, we could potentially use the ES modules introduced in PM-3399 |
| Comments |
| Comment by Githook User [ 29/Jan/24 ] | ||||
|
Author: {'name': 'Steve Gross', 'email': 'steve.gross@mongodb.com', 'username': 'stevegrossmongodb'}Message: Revert " This reverts commit b9caeab9c76ea36e704571d1648e362caf4f7002. GitOrigin-RevId: 6fef8824d84b67e6fe05070492a1b7f313a5f432 | ||||
| Comment by Githook User [ 29/Jan/24 ] | ||||
|
Author: {'name': 'Steve Gross', 'email': 'steve.gross@mongodb.com', 'username': 'stevegrossmongodb'}Message: Revert " This reverts commit ef1b0924376fd0a0d563bc5050f037a70a87acfe. GitOrigin-RevId: d20a53ae1f488f29598715139b6ea6df48d43085 | ||||
| Comment by Githook User [ 14/Nov/23 ] | ||||
|
Author: {'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}Message: | ||||
| Comment by Githook User [ 13/Nov/23 ] | ||||
|
Author: {'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}Message: | ||||
| Comment by Alex Neben [ 01/Nov/23 ] | ||||
|
You are right that in a perfect world changing one file that is included everywhere shouldn't fail burn in. However, this is a major step in the right direction. In most cases this will be better. In a few (such as the case you presented) it will be exactly the same because burn in already wasn't running. Now it will just be red. | ||||
| Comment by Mikhail Shchatko [ 31/Oct/23 ] | ||||
|
alex.neben@mongodb.com but in this case the user is changing just one file and there is no way to split it into several patches or something to overcome the issue. Does it make sense to tolerate burn-in failure in such case? | ||||
| Comment by Alex Neben [ 30/Oct/23 ] | ||||
|
I don't think we should assume generate.tasks will ever be able to support so much. For something this prolific i think it could be ok to fail burn in. We basically tell the user. Hey this touches so many files we cannot possibly ensure this is safe. | ||||
| Comment by Mikhail Shchatko [ 30/Oct/23 ] | ||||
|
While investigating this I've got from_jstest_files_to_their_dependants_map.json For example I have this patch, where I made a change to jstests/libs/uuid_util.js file, which generated a huge task generation config (26.7MB and 686185 lines of json) with hundreds of tasks (Generated Burn In Task Config - Execution 0 config file is attached to the task here and attached to this ticket generated-burn-in-config-653f6f0e0ae606a657a5dffa.tgz
I think it blocks the solution to be merged until we make sure that generate.tasks command is able to consume such large configuration json and generate such large amount of tasks. I opened a draft PR to come back to it later: https://github.com/10gen/mongo/pull/16499 |