[CDRIVER-4767] Please provide a usable archive Created: 06/Nov/23  Updated: 05/Jan/24

Status: Investigating
Project: C Driver
Component/s: Build
Affects Version/s: 1.25.0
Fix Version/s: 1.25.3, 1.25.4, 1.25.5

Type: Bug Priority: Unknown
Reporter: Remi Collet Assignee: Rishabh Bisht
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to CDRIVER-4809 Add a default VERSION_CURRENT file to... Closed
is related to CDRIVER-4763 Reapply `BUILD_VERSION` CMake option Closed
is related to CDRIVER-4788 Provide alternate for python-sphinx-d... Closed
Tested
tested by CDRIVER-4794 Test building RPM with EPEL packages Backlog

 Description   

Until version 1.24.4 provided archive was usable.

Since version 1.25.0 changelog states:

The constructed source-distribution archive is no longer available as a
release artifact. Please instead use the per-release repository archive
attached to a GitHub release, or clone the repository at the desired release
tag.

 

But this one is NOT usable

-- Computing the current release version...
fatal: not a git repository (or any parent up to mount point /dev)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
Traceback (most recent call last):
  File "/dev/shm/extras/BUILD/mongo-c-driver-1.25.0/build/cmake/../calc_release_version.py", line 400, in <module>
    RELEASE_VER = previous(main()) if PREVIOUS else main()
                                                    ^^^^^^
  File "/dev/shm/extras/BUILD/mongo-c-driver-1.25.0/build/cmake/../calc_release_version.py", line 320, in main
    head_commit_short = check_output(['git', 'rev-parse', '--revs-only',
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/dev/shm/extras/BUILD/mongo-c-driver-1.25.0/build/cmake/../calc_release_version.py", line 136, in check_output
    raise subprocess.CalledProcessError(ret, args[0])
subprocess.CalledProcessError: Command 'git' returned non-zero exit status 128.
CMake Error at build/cmake/BuildVersion.cmake:43 (message):
  Computing the build version failed! [1]:Call Stack (most recent call first):
  build/cmake/BuildVersion.cmake:50 (compute_build_version)
  CMakeLists.txt:5 (include)

 

Adding the VERSION_CURRENT is a workaround, but build fails later

 

Configuration error:
There is a programmable error in your configuration file:Traceback (most recent call last):
  File "/usr/lib/python3.11/site-packages/sphinx/config.py", line 343, in eval_config_file
    exec(code, namespace)
  File "/dev/shm/extras/BUILD/mongo-c-driver-1.25.0/src/libmongoc/doc/conf.py", line 13, in <module>
    from sphinx_design.dropdown import DropdownDirective
ModuleNotFoundError: No module named 'sphinx_design'



 Comments   
Comment by Githook User [ 04/Jan/24 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 Support Python 3.6 in man page build (#1504)

  • remove use of `:=`

Assignment expressions were introduced in Python 3.8

  • use `typing.Union` instead of `|`

Writing union types as `X | Y` was introduced in Python 3.10.

  • do not use Union `|` operator

The Union `|` operator was added in Python 3.9. See: PEP 584

  • use `typing.List` and `typing.Dict`

Type hinting for `list`, `dict` was added in Python 3.9. See: PEP 585
Branch: r1.25
https://github.com/mongodb/mongo-c-driver/commit/4147d49bf46e3d247a6473c1bb0cbae982a9c18f

Comment by Githook User [ 04/Jan/24 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 Support Python 3.6 in man page build (#1504)

  • remove use of `:=`

Assignment expressions were introduced in Python 3.8

  • use `typing.Union` instead of `|`

Writing union types as `X | Y` was introduced in Python 3.10.

  • do not use Union `|` operator

The Union `|` operator was added in Python 3.9. See: PEP 584

  • use `typing.List` and `typing.Dict`

Type hinting for `list`, `dict` was added in Python 3.9. See: PEP 585
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/0eb1047a452e201bfa9b24c5a0b16679f8c51256

Comment by Kevin Albertson [ 04/Jan/24 ]

PR to restore Python 3.6 support for man page build: https://github.com/mongodb/mongo-c-driver/pull/1504

But these workarounds should not be necessary; your software should be able to build itself without me needing to tell it what version it is.

I agree. I filed CDRIVER-4809 to suggest adding a VERSION_CURRENT file in the source.

Comment by Ryan Schmidt [ 04/Jan/24 ]

This seems to have devolved into a discussion of the secondary issue (about sphinx) but the primary issue (a git error in a directory that is not a git repository) still prevents me from building 1.25.4 from your downloadable source tarball.

Remi mentioned a workaround of "adding the VERSION_CURRENT" and this worked for me, but for anyone else who wasn't sure what that meant, it means creating a file called VERSION_CURRENT at the root of the extracted source code directory. The contents of the file should be the version of the software you're building (e.g. 1.25.4). The better fix according to CDRIVER-3410 appears to be using the cmake flag -DBUILD_VERSION=1.25.4. But these workarounds should not be necessary; your software should be able to build itself without me needing to tell it what version it is.

Comment by Remi Collet [ 03/Jan/24 ]

Build of 1.25.4 is OK with RHEL+EPEL 9 but KO for 8

 

Configuration error:
There is a syntax error in your configuration file: future feature annotations is not defined (mongoc_common.py, line 1)
Did you change the syntax from 2.x to 3.x?
gmake[2]: *** [src/libbson/doc/CMakeFiles/bson-man.dir/build.make:566: src/libbson/doc/man/bson_aligned_alloc.3] Error 2
gmake[2]: Leaving directory '/builddir/build/BUILD/mongo-c-driver-1.25.4'
gmake[1]: *** [CMakeFiles/Makefile2:1034: src/libbson/doc/CMakeFiles/bson-man.dir/all] Error 2
gmake[1]: *** Waiting for unfinished jobs....Configuration error:
There is a syntax error in your configuration file: bad input (conf.py, line 112)
Did you change the syntax from 2.x to 3.x?
gmake[2]: *** [src/libmongoc/doc/CMakeFiles/mongoc-man.dir/build.make:1474: src/libmongoc/doc/man/mongoc_advanced_connections.3] Error 2
gmake[2]: Leaving directory '/builddir/build/BUILD/mongo-c-driver-1.25.4'
gmake[1]: *** [CMakeFiles/Makefile2:1166: src/libmongoc/doc/CMakeFiles/mongoc-man.dir/all] Error 2
gmake[2]: Leaving directory '/builddir/build/BUILD/mongo-c-driver-1.25.4'

Comment by Rishabh Bisht [ 03/Jan/24 ]

Hi remi
We released the version 1.25.4 that brings back support to build with Sphinx 1.7.6. Could you please give it a try and see if it fixes the EPEL build? Thanks!

Comment by Kevin Albertson [ 27/Dec/23 ]

Support for building man pages with Sphinx 1.7.6 was added back with https://github.com/mongodb/mongo-c-driver/pull/1492 and is planned for the 1.25.4 release.
Thank you for bringing to our attention remi. If you can verify the EPEL build is now fixed, that may help. If other issues remain, we may be able to address them before a release.

Comment by Githook User [ 27/Dec/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 support Sphinx 1.7.6 (#1492)

  • import older `DirectoryHTMLBuilder` if needed

The import path of `DirectoryHTMLBuilder` was changed from `sphinx.builders.html` to `sphinx.builders.dirhtml` in Sphinx v2.0.0

  • import `Directive`, not `SphinxDirective`

`SphinxDirective` was added in Sphinx v2.0.0

  • do not import Project

`Project` was added in Sphinx v2.0.0

  • update `needs_sphinx` to 1.7
  • error if `add_css_file` is unavailable for HTML builder.

But do not error otherwise. This is intended to help build man pages without HTML requirements.

`Sphinx.add_css_file` was added in Sphinx v3.5.0

  • use `builder-inited`

`config-inited` was added in Sphinx v1.8.0

  • do not use `external` role

`external` role was added in intersphinx extension in v4.4

  • do not use `noindexentry`

`noindexentry` option was added to C domain in Sphinx v3.2

  • remove `name` from `index` directive

`name` option was added in Sphinx v3.0

  • populate `man_pages` in `setup` instead of event handler

The `config-inited` event was added in Sphinx v1.8.0
The `ManualPageBuilder` constructor raises an exception for an empty `man_pages` value.

  • remove parallel build of man pages

To avoid an error observed on Sphinx 1.7.6:
```
AttributeError: Can't pickle local object 'BuildEnvironment._read_parallel.<locals>.merge'
```

  • remove unhelpful warning

An error is raised at runtime with a message suggesting to install `sphinx-design`. The warning was a source of confusion in CDRIVER-4767

  • update NEWS for 1.25.3
Comment by Githook User [ 27/Dec/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 support Sphinx 1.7.6 (#1492)

  • import older `DirectoryHTMLBuilder` if needed

The import path of `DirectoryHTMLBuilder` was changed from `sphinx.builders.html` to `sphinx.builders.dirhtml` in Sphinx v2.0.0

  • import `Directive`, not `SphinxDirective`

`SphinxDirective` was added in Sphinx v2.0.0

  • do not import Project

`Project` was added in Sphinx v2.0.0

  • update `needs_sphinx` to 1.7
  • error if `add_css_file` is unavailable for HTML builder.

But do not error otherwise. This is intended to help build man pages without HTML requirements.

`Sphinx.add_css_file` was added in Sphinx v3.5.0

  • use `builder-inited`

`config-inited` was added in Sphinx v1.8.0

  • do not use `external` role

`external` role was added in intersphinx extension in v4.4

  • do not use `noindexentry`

`noindexentry` option was added to C domain in Sphinx v3.2

  • remove `name` from `index` directive

`name` option was added in Sphinx v3.0

  • populate `man_pages` in `setup` instead of event handler

The `config-inited` event was added in Sphinx v1.8.0
The `ManualPageBuilder` constructor raises an exception for an empty `man_pages` value.

  • remove parallel build of man pages

To avoid an error observed on Sphinx 1.7.6:
```
AttributeError: Can't pickle local object 'BuildEnvironment._read_parallel.<locals>.merge'
```

  • remove unhelpful warning

An error is raised at runtime with a message suggesting to install `sphinx-design`. The warning was a source of confusion in CDRIVER-4767

  • update NEWS for 1.25.3
Comment by Kevin Albertson [ 18/Dec/23 ]

Here is a PR to add back support for Sphinx 1.7.6: https://github.com/mongodb/mongo-c-driver/pull/1492

Comment by Githook User [ 07/Dec/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 import annotations to support Python 3.9 (#1490)
Branch: r1.25
https://github.com/mongodb/mongo-c-driver/commit/d089f840f8b5c9be268d7b0fc9ad87581989934e

Comment by Githook User [ 07/Dec/23 ]

Author:

{'name': 'Kevin Albertson', 'email': 'kevin.albertson@mongodb.com', 'username': 'kevinAlbs'}

Message: CDRIVER-4767 import annotations to support Python 3.9 (#1490)
Branch: master
https://github.com/mongodb/mongo-c-driver/commit/3a624d53dfed8820019a8fb0e436f6151588f0f5

Comment by Remi Collet [ 07/Dec/23 ]

> To confirm: is 1.2.3 the version of Sphinx used

No, 1.7.6 on EL-8 and 3.4.3 on EL-9

See https://rpms.remirepo.net/rpmphp/zoom.php?rpm=python-sphinx

(not an official app, but indexing repository content from Fedora, EPEL, RHEL...)

Comment by Kevin Albertson [ 07/Dec/23 ]

Unable to import sphinx_design. Documentation cannot be built as HTML.

That is a non-fatal warning. Man pages can still be built.

This project needs at least Sphinx v5.0 and therefore cannot be built with this version.

What version of Sphinx used? https://packages.fedoraproject.org/pkgs/python3-sphinx/python3-sphinx/ lists 1.2.3 for EPEL 7. Building with Sphinx 1.2.3 locally results in a different error:

  File "conf.py", line 9, in <module>
ModuleNotFoundError: No module named 'sphinx.builders.dirhtml'

Sorry for the trouble. The C driver tests building the RPM package on RHEL 9.0. Filed CDRIVER-4794 to propose testing with EPEL packages (assuming that is possible).

Comment by Remi Collet [ 07/Dec/23 ]

Using patchfrom PR #1490

Build still fails

Sphinx version error:
This project needs at least Sphinx v5.0 and therefore cannot be built with this version.
Unable to import sphinx_design. Documentation cannot be built as HTML.

Comment by Kevin Albertson [ 06/Dec/23 ]

Error reproduces when building with Python 3.9.18. Here is a PR with a proposed fix: https://github.com/mongodb/mongo-c-driver/pull/1490

Comment by Remi Collet [ 06/Dec/23 ]

rishabh.bisht@mongodb.com don't know if python-sphinx-design issue is resolved, btw build still failing somewhere else, probably related to other changes (python or shpinx ?)

 

There is a programmable error in your configuration file:Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sphinx/config.py", line 326, in eval_config_file
    execfile_(filename, namespace)
  File "/usr/lib/python3.9/site-packages/sphinx/util/pycompat.py", line 88, in execfile_
    exec(code, _globals)
  File "/builddir/build/BUILD/mongo-c-driver-1.25.2/src/libbson/doc/conf.py", line 10, in <module>
    from mongoc_common import *
  File "/builddir/build/BUILD/mongo-c-driver-1.25.2/build/sphinx/mongoc_common.py", line 29, in <module>
    def _file_man_page_name(fpath: Path) -> str | None:
TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'

Comment by Rishabh Bisht [ 05/Dec/23 ]

Hi remi , CDRIVER-4788 (python-sphinx-design not required for man pages) is fixed with 1.25.2 release.

Could you please check if the provided fix is good enough to get latest version of C Driver (1.25.2) into EPEL? Thanks!

Comment by Rishabh Bisht [ 24/Nov/23 ]

Thanks for confirming. I have logged CDRIVER-4788 to see if we can have any workaround for this. 

Comment by Remi Collet [ 23/Nov/23 ]

> Is this because of the python-sphinx-design package missing in EPEL?

 

Mostly yes

Comment by Rishabh Bisht [ 23/Nov/23 ]

Hi remi 

BTW, probably going to keep old 1.24 in EPEL forever

Is this because of the python-sphinx-design package missing in EPEL?

Comment by Rishabh Bisht [ 09/Nov/23 ]

remi BUILD_VERSION issue should be fixed with CDRIVER-4763, part of the 1.25.1 release.

Looks like there is no action item for us here at the moment, but please let me know if I am missing something. Thank you!

PS: libutf8proc is shipped as a part of C driver so even if libutf8proc library is not present, it should not cause an issue.

Comment by Remi Collet [ 07/Nov/23 ]

Documentation is not mandatory, but as it was provided until 1.24.0, this could be considered as a breakage

So: https://bugzilla.redhat.com/show_bug.cgi?id=2248522

Dependency change (libicu => libutf8proc) may also be another problem

BTW, probably going to keep old 1.24 in EPEL forever

 

Comment by Kevin Albertson [ 07/Nov/23 ]

remi sorry for the trouble. Is building HTML documentation needed for the package? If no, configuring with ENABLE_HTML_DOCS=OFF may be a workaround.

Comment by Remi Collet [ 06/Nov/23 ]

Adding python3-sphinx-design allows to build it on Fedora.

As this dependency is missing on RHEL, it is not possible anymore to build this project there

Comment by PM Bot [ 06/Nov/23 ]

Hi remi, thank you for reporting this issue! The team will look into it and get back to you soon.

Generated at Wed Feb 07 21:21:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.