Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-49352

4.0 does not build with python 3.8

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 4.0.19
    • Fix Version/s: 3.6.20, 4.0.21
    • Component/s: None
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v3.6
    • Sprint:
      Dev Platform 2020-08-10

      Description

      When I run, in 4.0.19 tree,

        pip install --user -r buildscripts/requirements.txt
      

      I receive the following error:

          Running setup.py install for typed-ast ... error
          ERROR: Command errored out with exit status 1:
           command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-xvnr8otx/typed-ast/setup.py'"'"'; __file__='"'"'/tmp/pip-install-xvnr8otx/typed-ast/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-9z5dkcdx/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/build/.local/include/python3.8/typed-ast
               cwd: /tmp/pip-install-xvnr8otx/typed-ast/
          Complete output (23 lines):
          running install
          running build
          running build_py
          creating build
          creating build/lib.linux-x86_64-3.8
          creating build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/conversions.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/ast3.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/__init__.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/ast27.py -> build/lib.linux-x86_64-3.8/typed_ast
          running build_ext
          building '_ast27' extension
          creating build/temp.linux-x86_64-3.8
          creating build/temp.linux-x86_64-3.8/ast27
          creating build/temp.linux-x86_64-3.8/ast27/Parser
          creating build/temp.linux-x86_64-3.8/ast27/Python
          creating build/temp.linux-x86_64-3.8/ast27/Custom
          x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iast27/Include -I/usr/include/python3.8 -c ast27/Parser/acceler.c -o build/temp.linux-x86_64-3.8/ast27/Parser/acceler.o
          ast27/Parser/acceler.c:13:10: fatal error: pgenheaders.h: No such file or directory
             13 | #include "pgenheaders.h"
                |          ^~~~~~~~~~~~~~~
          compilation terminated.
          error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
      
      

      4.0 build transcript: https://gist.github.com/p-mongo/2804e19d6862d08f4192faa45ae1e159

      I also get this error if I run the same command in 4.4.0-rc12 tree:

          ERROR: Command errored out with exit status 1:
           command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-262v9yio/typed-ast/setup.py'"'"'; __file__='"'"'/tmp/pip-install-262v9yio/typed-ast/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-8z8zcirp/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/build/.local/include/python3.8/typed-ast
               cwd: /tmp/pip-install-262v9yio/typed-ast/
          Complete output (23 lines):
          running install
          running build
          running build_py
          creating build
          creating build/lib.linux-x86_64-3.8
          creating build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/conversions.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/ast3.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/__init__.py -> build/lib.linux-x86_64-3.8/typed_ast
          copying typed_ast/ast27.py -> build/lib.linux-x86_64-3.8/typed_ast
          running build_ext
          building '_ast27' extension
          creating build/temp.linux-x86_64-3.8
          creating build/temp.linux-x86_64-3.8/ast27
          creating build/temp.linux-x86_64-3.8/ast27/Parser
          creating build/temp.linux-x86_64-3.8/ast27/Python
          creating build/temp.linux-x86_64-3.8/ast27/Custom
          x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Iast27/Include -I/usr/include/python3.8 -c ast27/Parser/acceler.c -o build/temp.linux-x86_64-3.8/ast27/Parser/acceler.o
          ast27/Parser/acceler.c:13:10: fatal error: pgenheaders.h: No such file or directory
             13 | #include "pgenheaders.h"
                |          ^~~~~~~~~~~~~~~
          compilation terminated.
          error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
          ----------------------------------------
      

      4.4 transcript: https://gist.github.com/p-mongo/f6748100d02e999d4315fd3609153809

      The issue appears to be described here: https://github.com/python/typed_ast/issues/126

      The solution appears to be updating to a newer version of typed_ast which appears to not be a direct dependency listed in build requirements.txt.

      On 4.4 however, the instructions I have call for running instead the following which does work:

      pip3 install --user -r etc/pip/compile-requirements.txt
      

      So the issue seems to affect servers like 4.0 that do not have the etc/pip/compile-requirements.txt file, and whoever uses the buildscripts/requirements.txt file in newer servers which apparently isn't needed for building by users.

        Attachments

          Activity

            People

            Assignee:
            ryan.egesdahl Ryan Egesdahl
            Reporter:
            oleg.pudeyev Oleg Pudeyev
            Participants:
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: