[SERVER-30634] Git detects all files as having changed on Windows causing burn_in_tests task to time out in Evergreen Created: 14/Aug/17  Updated: 30/Oct/23  Resolved: 16/Nov/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.4.11, 3.6.0-rc5, 3.7.1

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Yves Duhem
Resolution: Fixed Votes: 0
Labels: disabled-test
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Gantt Dependency
Related
is related to SERVER-30565 burn_in_tests.py shouldn't cause comp... Closed
Backwards Compatibility: Fully Compatible
Operating System: Windows
Backport Requested:
v3.6, v3.4
Sprint: TIG 2017-10-23, TIG 2017-11-13, TIG 2017-12-04
Participants:

 Description   

See this Evergreen task timeout as an example.

My initial suspicion was that core.autocrlf wasn't being set to true when we cloned repositories in Evergreen on Windows, so I tried out the following patch in an Evergreen patch build. Had I tested it out locally, I would have immediately found out that specifying the --name-only to git diff causes it to ignore the --ignore-space-at-eol option and still display the names of files whose only changes are LF -> CRLF. (git diff --ignore-space-at-eol correctly generates no output though.)

diff --git a/buildscripts/burn_in_tests.py b/buildscripts/burn_in_tests.py
index a78c905f97..4251a11c07 100644
--- a/buildscripts/burn_in_tests.py
+++ b/buildscripts/burn_in_tests.py
@@ -175,7 +175,13 @@ def find_changed_tests(branch_name, base_commit, max_revisions, buildvariant, ch
               "the --maxRevisions option."
         return changed_tests
 
-    changed_files = callo(["git", "diff", "--name-only", base_commit]).splitlines()
+    changed_files = callo(["git",
+                           "diff",
+                           "--name-only",
+                           # We specify --ignore-space-at-eol to avoid considering a file changed on
+                           # Windows simply because core.autocrlf isn't set to true.
+                           "--ignore-space-at-eol",
+                           base_commit]).splitlines()
     # New files ("untracked" in git terminology) won't show up in the git diff results.
     untracked_files = callo(["git", "status", "--porcelain"]).splitlines()

Further investigation is still needed, but I'm filing this ticket so that we can temporarily disable the burn_in_tests task on the "Windows 2008R2 DEBUG" builder rather than reverting the changes from SERVER-30565. The changes from SERVER-30565 are still desirable to avoid causing the compile task to fail when the Evergreen API server isn't available.



 Comments   
Comment by Githook User [ 17/Jan/18 ]

Author:

{'name': 'Yves Duhem', 'email': 'yves.duhem@mongodb.com', 'username': 'syev'}

Message: SERVER-30565 Move burn_in_tests.py invocation out of compile task

(cherry picked from commit 7969a3e83d918a9a3ccaae65c6509b1abad4b34e)

Also includes the following related fix:
SERVER-30634 Fix burn_in_tests test detection on Windows
(cherry picked from commit 6b815203b752f4a1e1f8f810b7dda5e0a331b4ea)
Branch: v3.4
https://github.com/mongodb/mongo/commit/0006869bf571ef76636a3cc75b033de4e6f168c6

Comment by Githook User [ 16/Nov/17 ]

Author:

{'name': 'Yves Duhem', 'username': 'syev', 'email': 'yves.duhem@mongodb.com'}

Message: SERVER-30634 Fix burn_in_tests test detection on Windows

(cherry picked from commit 6b815203b752f4a1e1f8f810b7dda5e0a331b4ea)
Branch: v3.6
https://github.com/mongodb/mongo/commit/b0ef90bc3495561fe1cba07b1a8d7f8e20b3a564

Comment by Yves Duhem [ 16/Nov/17 ]

After extracting the artifacts in the repository directory, git 1.9.5 on windows listed all the files as changed (using 'git diff --name-only') even though 'git diff' displayed the correct changes.
We checked that the files were not modified by the extraction (EOL or anything else).
We tested changing file metadata (file permissions, Windows file attributes, etc.) to try to reproduce the issue but could not outside Evergreen task runs.

We worked around this git issue by cloning the repository in a separate directory and only copying the dbtest executable in the cloned directory.

Comment by Githook User [ 16/Nov/17 ]

Author:

{'name': 'Yves Duhem', 'username': 'syev', 'email': 'yves.duhem@mongodb.com'}

Message: SERVER-30634 Fix burn_in_tests test detection on Windows
Branch: master
https://github.com/mongodb/mongo/commit/6b815203b752f4a1e1f8f810b7dda5e0a331b4ea

Comment by Jonathan Abrahams [ 18/Oct/17 ]

yves.duhem Please ensure that you add "set up virtualenv" into the burn_in task.

Comment by Githook User [ 14/Aug/17 ]

Author:

{'name': 'Max Hirschhorn', 'username': 'visemet', 'email': 'max.hirschhorn@mongodb.com'}

Message: SERVER-30634 Temporarily disable burn_in_tests on Windows 2008R2 DEBUG.
Branch: master
https://github.com/mongodb/mongo/commit/3c0a1869857bc91a5fb0d2eee0a1652209df09e3

Generated at Thu Feb 08 04:24:30 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.