[SERVER-34438] Consolidate requirements.txt Created: 12/Apr/18 Updated: 27/Oct/23 Resolved: 28/Sep/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | 3.7.3 |
| Fix Version/s: | None |
| Type: | Task | Priority: | Trivial - P5 |
| Reporter: | Robert Guo (Inactive) | Assignee: | DO NOT USE - Backlog - Test Infrastructure Group (TIG) |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Participants: | |||||||||
| Description |
|
There are currently a number of requirements.txt files used by various Python components of the server codebase. Since the server is logically a single project that is typically run under the same virtualenv, we should have a single place to specify requirements to avoid various issues like conflicting dependencies (even with proper dependency resolution) and missing dependencies. |
| Comments |
| Comment by Robert Guo (Inactive) [ 28/Sep/18 ] |
|
Definitely! I will close this ticket. |
| Comment by Benjamin Caimano (Inactive) [ 28/Sep/18 ] |
|
robert.guo, do you think |
| Comment by Robert Guo (Inactive) [ 12/Apr/18 ] |
|
I don't think there are missing dependencies in there. pytests/requirements.txt was what I had in mind when saying "hunt down all the requirement files", it is currently independent of buildscripts. |
| Comment by Max Hirschhorn [ 12/Apr/18 ] |
I think that's what buildscripts/requirements.txt should be doing already. If there are dependencies which are missing then we should probably address that. The boto3 package was an interesting case of we didn't want local users of resmoke.py to need to bother installing it since archival is only enabled in Evergreen when we have AWS credentials to use. https://www.python.org/dev/peps/pep-0508/#extras may be a better way of expressing the behavior we want. |
| Comment by Robert Guo (Inactive) [ 12/Apr/18 ] |
|
max.hirschhorn what do you think of adding a top-level requirements.txt file to include all the other requirements. This would allow us to catch conflicting dependencies across multiple sub-projects and not force users to hunt down all the requirement files just to run pylinters.py lint. (The pylinters issue is temporarily worsened by a subtle dependency of boto3 in pytests/requirements by buildscripts/ because of boto3 not being in all the necessary requirements files due to external factors.) On an unrelated note, I had misinterpreted 1a in the dependency resolution issue: https://github.com/pypa/pip/issues/988
This isn't the case for my local testing. So fixing the "missing dependencies" benefit in this ticket description is incorrect |
| Comment by Max Hirschhorn [ 12/Apr/18 ] |
|
I think individual Python packages should declare their own requirements. This is why -r resmokelib/requirements.txt is used to reference resmoke.py's dependencies in buildscripts/requirements.txt. I'd be happy to change it to requirements/resmoke.txt, etc. but I'd still like them in separate files. |