-
Type:
Bug
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines
-
17.726
-
None
-
None
WT-17626 changed dist/s_clang_format to obtain clang-format by creating a Python virtual environment and pip-installing the clang-format wheel from PyPI:
venv="$TOP_DIR/.venv"
download_clang_format()
This introduces a hard dependency on ensurepip (the Debian/Ubuntu python3.10-venv package). On a stock Ubuntu box where that package isn't installed, plain python3 -m venv creates a pip-less venv and the next line fails.
Steps to reproduce:
- On a system without python3-venv/ensurepip, with a clang-format other than 12.0.1 on PATH (or none).
- Run dist/s_all (or dist/s_fast, or any generator such as ./api_config_gen.py).
Actual result:
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt install python3.10-venv
...
./s_clang_format: line 14: /home/.../.venv/bin/pip: No such file or directory
Command '['./s_clang_format', '/home/.../src/include/config.h.MOD']' returned non-zero exit status 1.
Impact:
- Regression: the previous prebuilt-binary path worked on these machines without python3-venv.
- Repository is polluted with changes as a result of the s_all failuer:
(use "git restore <file>..." to discard changes in working directory) modified: ../src/checkpoint/checkpoint.h modified: ../src/checkpoint/checkpoint_private.h modified: ../src/evict/evict.h modified: ../src/evict/evict_private.h modified: ../src/include/extern.h modified: ../src/include/extern_darwin.h modified: ../src/include/extern_linux.h modified: ../src/include/extern_posix.h modified: ../src/include/extern_win.h modified: ../src/live_restore/live_restore.h modified: ../src/live_restore/live_restore_private.h modified: ../src/log/log.h modified: ../src/log/log_auto.c modified: ../src/log/log_private.h modified: ../src/reconcile/reconcile.h modified: ../src/reconcile/reconcile_private.h modified: ../test/format/format_config.h modified: ../test/format/format_config_def.c
- Not isolated to the dedicated clang-format step. Every generator that formats its output via dist.py:format_srcfile() shells out to s_clang_format (api_config_gen.py, stat.py, log.py, prototypes.py, api_err.py, type_to_str.py), so they all hit this. Because format_srcfile catches the error and only prints it, generators can "succeed" while emitting unformatted C.
- Clobber risk: the venv is built at $TOP_DIR/.venv (the repo root), where many developers keep their own virtualenv. Running s_all does python3 -m venv over it and pip-installs into it; on an affected box it leaves a broken, pip-less .venv in the tree.
Suggested fix:
- Don't hard-depend on ensurepip/python3-venv. Use a tool that bundles pip (virtualenv/uv), or python3 -m venv --without-pip + bootstrap pip, or fall back to user pip/pipx; at minimum print an actionable error and abort the s_fast
- Build the venv in a dedicated, gitignored location (e.g. dist/.clang-format-venv or a cache dir) instead of $TOP_DIR/.venv to avoid clobbering developers' environments.
- is related to
-
WT-17626 s_clang_format: simplify binary download step
-
- Closed
-