Local run of bazel lint fails on too many files

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Fixed
    • Priority: Major - P3
    • 9.0.0-rc0
    • Affects Version/s: None
    • Component/s: Bazel
    • None
    • Fully Compatible
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Summary

      Developers are unable to run bazel run //:format locally if there are too many untracked files. There should be an easy way to exclude directories and scope down to the most relevant files.

      Steps to Reproduce

      1. Perform actions (like mdb data files, fuzzer runs, etc etc) that generate directories in the root of the source tree with lots of files.
      2. bazel run //:format

      Expected Result

      A simple error message with actionable steps. Not a Python backtrace.

      Actual Result

      $ bazel run //:format 
INFO: running wrapper hook...
INFO: auto_header build generated: 628.8 ms
INFO: Invocation ID: 66751a9f-d04c-40d7-b960-5e89267f430b
INFO: Streaming build results to: https://sodalite.cluster.engflow.com/invocation/66751a9f-d04c-40d7-b960-5e89267f430b
INFO: Analyzed target //:format (3 packages loaded, 32 targets configured).
INFO: Found 1 target...
Target //bazel/format:rules_lint_format_wrapper up-to-date:
  bazel-bin/bazel/format/rules_lint_format_wrapper
INFO: Elapsed time: 1.197s, Critical Path: 0.57s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/bazel/format/rules_lint_format_wrapper --prettier prettier_/prettier --rules-lint-format bazel/format/rules_lint_format.bash --rules-lint-format-check bazel/format/rules_lint_format.check.bash
INFO: Streaming build results to: https://sodalite.cluster.engflow.com/invocation/66751a9f-d04c-40d7-b960-5e89267f430b
Running prettier
Traceback (most recent call last):
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/execroot/_main/bazel-out/aarch64-fastbuild/bin/bazel/format/rules_lint_format_wrapper.runfiles/_main/bazel/format/rules_lint_format_wrapper.py", line 238, in <module>
    exit(main())
         ~~~~^^
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/execroot/_main/bazel-out/aarch64-fastbuild/bin/bazel/format/rules_lint_format_wrapper.runfiles/_main/bazel/format/rules_lint_format_wrapper.py", line 229, in main
    if run_prettier(prettier_path, args.check, files_to_format)
       ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/execroot/_main/bazel-out/aarch64-fastbuild/bin/bazel/format/rules_lint_format_wrapper.runfiles/_main/bazel/format/rules_lint_format_wrapper.py", line 127, in run_prettier
    subprocess.run(command, check=True)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/external/_main~setup_mongo_python_toolchains~py_linux_arm64/dist/lib/python3.13/subprocess.py", line 554, in run
    with Popen(*popenargs, **kwargs) as process:
         ~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/external/_main~setup_mongo_python_toolchains~py_linux_arm64/dist/lib/python3.13/subprocess.py", line 1039, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
    ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        pass_fds, cwd, env,
                        ^^^^^^^^^^^^^^^^^^^
    ...<5 lines>...
                        gid, gids, uid, umask,
                        ^^^^^^^^^^^^^^^^^^^^^^
                        start_new_session, process_group)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/external/_main~setup_mongo_python_toolchains~py_linux_arm64/dist/lib/python3.13/subprocess.py", line 1991, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: '/home/ubuntu/.cache/bazel/_bazel_ubuntu/438e730df43f16e18dd90600e6cbe8fb/execroot/_main/bazel-out/aarch64-fastbuild/bin/prettier_/prettier'

      Impact

      Unable to unblock required bazel lint errors on PRs, grinds development to a halt.

       

      Proposed Solution

      Prevent

      Automatically exclude certain types of directories known to not contain source files to the bazel formatter. Or rather, scope to specific source directories like src/ etc.

      Detect

      This Python script should detect if there are a large number of files being picked up for the linter, and print out a simple message explaining the issue and steps to remediate. For me, I stuffed all these directories in a directory called "cruft" with a simple .gitignore file inside consisting of "*".

            Assignee:
            Zack Winter
            Reporter:
            Matt Kneiser
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: