[SERVER-76751] Add poetry as a way to pin python deps in mongo master Created: 02/May/23 Updated: 17/Jan/24 Resolved: 05/Sep/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Alex Neben | Assignee: | Alex Neben |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Assigned Teams: |
Server Development Platform
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 35 | ||||||||||||||||||||||||
| Description |
|
https://github.com/pantsbuild/pex
We are consistently having problems like BF-29350 that are due to unpinned python dependencies. We should use poetry lock file to pin all our python dependencies. |
| Comments |
| Comment by Alex Neben [ 15/Sep/23 ] |
|
There is no plans to backport this as the lift required would be enormous |
| Comment by Simon Eismann [ 15/Sep/23 ] |
|
alex.neben@mongodb.com FYI: The switch to poetry ended up causing some intermittent failures in sys-perf as it parallelizes the package installation based on the number of cores of the underlying instance (BF-29897). We have fixed this on sys-perf by limiting the number of workers used by poetry, but I wanted to raise this in case this issue crops up on other waterfalls as well. On a side note: Are there plans to backport this? Keeping the compile identical across all branches seems preferable. |
| Comment by Githook User [ 05/Sep/23 ] |
|
Author: {'name': 'Alex Neben', 'email': 'alex.neben@mongodb.com', 'username': 'IamXander'}Message: |
| Comment by Alex Neben [ 19/Jun/23 ] |
|
Looks like poetry might have a bug when trying to export all pip dependencies. When trying to run "poetry export --output="requirements.txt" --format="requirements.txt" --without-hashes" it gets stuck on pip libraries that have multiple versions and are separated by markers. |
| Comment by Alex Neben [ 19/Jun/23 ] |
|
See https://github.com/10gen/mongo/pull/new/IamXander/pex_rewrite for full example. Making a smaller version to get pinning before we take a bigger bite if this problem. |