[CXX-491] Determine how ABI versioning / soname versions should work Created: 22/Jan/15  Updated: 30/Jan/23  Resolved: 10/Feb/15

Status: Closed
Project: C++ Driver
Component/s: None
Affects Version/s: None
Fix Version/s: 0.1.0

Type: Task Priority: Major - P3
Reporter: Tyler Brock Assignee: Adam Midvidy
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by CXX-428 Establish driver versioning scheme Closed
is duplicated by CXX-442 Implement and Document ABI strategy Closed
Related
related to CXX-1569 CXX ABI Blocked
Epic Link: PM-123

 Comments   
Comment by Githook User [ 12/Feb/15 ]

Author:

{u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}

Message: CXX-491 add VERSIONING.md
Branch: master
https://github.com/mongodb/mongo-cxx-driver/commit/3ecb65662d191dec7d0f37e9f6b26dc8bbd7b4d4

Comment by Githook User [ 12/Feb/15 ]

Author:

{u'username': u'amidvidy', u'name': u'Adam Midvidy', u'email': u'amidvidy@gmail.com'}

Message: CXX-491 allow parallel header installation
Branch: master
https://github.com/mongodb/mongo-cxx-driver/commit/9b8801d0038db733ab9d198eb9490bfd7d7574b4

Comment by Adam Midvidy [ 10/Feb/15 ]

added VERSIONING.md to repository.

Comment by Tyler Brock [ 09/Feb/15 ]

I don't have any further comments, looks good to me. Maybe acm does though?

Comment by Adam Midvidy [ 06/Feb/15 ]

inline namespaces and versioned include directories have been implemented in the build system. The work on sonames hasn't because we won't actually be using it yet. acm and tyler.brock@gmail.com, do you have any further comments on the document (https://docs.google.com/a/10gen.com/document/d/16hPoMBKcaCcvKV2-_4uEo9Nb_jNDIsHuz20iqG_28ds/edit) ?

Comment by Adam Midvidy [ 04/Feb/15 ]

reopening until this is actually implemented in CMake

Comment by Adam Midvidy [ 03/Feb/15 ]

The aforementioned situation of depending on functionality introduced in a minor version and failing to link to a version of the library with the same soname is inconvenient. However, solving this isn't really the goal of DSO versioning. The real issue we want to prevent is the case when the ABI changes but the soname doesn't (e.g. https://code.google.com/p/lz4/issues/detail?id=147) - causing potentially catastrophic effects. That we get some forwards compatibility (version that was compiled against 1.2 can link against 1.3) is a nice bonus.

Regarding not using the libtool conventions:

The reason (as far as I can tell) libtool uses this confusing current,revision,age scheme is that the actual DSO names are calculated differently on different platforms - i.e. a library with C=16,R=3,A=1 might be named libfoo.so.15.16.3 (with a soname of libfoo.so.15) on Linux but libfoo-15.dll on windows. As CMake (to my knowledge) doesn't calculate DSO names in this manner, it doesn't seem worthwhile to reverse engineer it in CMake script. See this long-winded libtool thread for details: (https://lists.gnu.org/archive/html/libtool/2009-08/msg00002.html)

The other value of using the libtool scheme would be to separate ABI and API versioning. If we still want to do this I think it would be simpler to just use a separate semantic version for ABI as well.

Also, we don't plan on stabilizing ABI for now - so what do we provide to users until then? I think it would be harmful to provide a soname without promising ABI stability (or constantly bumping the major version...). Then, presumably, when we do an initial release that promises a stable ABI, we just start providing a soname from that point forward.

Lastly we haven't discussed what the initial version of the library will be. Would it make sense to call it 0.X.X, and then bump to 1.0 when we start keeping the ABI stable?

Comment by Adam Midvidy [ 02/Feb/15 ]

See proposal here: https://docs.google.com/a/10gen.com/document/d/16hPoMBKcaCcvKV2-_4uEo9Nb_jNDIsHuz20iqG_28ds/edit?usp=sharing

I've opted to use semver for both API and ABI versioning. This simplifies our versioning scheme and gives us almost all the benefit that the libtool convention would have provided.

Earlier today, Drew and I discussed the issue of what would happen if an application depended on API functionality added in a minor release.
I.e. an application depends on functionality introduced in libmongocxx-1.2 that doesn't exist in libmongocxx-1.1. If a user who has version 1.1 installed is building the application from source they will simply fail to compile - but if the user is relying on a binary package, at runtime it will find libmongocxx.so.1 (1.2 and 1.1 have the same soname) then presumably fail to link. Is this acceptable behavior or is there a better solution?

Note that this problem exists with the libtool scheme as well - i.e. app with c=1, r=2, a=0 adds some functionality without breaking ABI, now c=2, r=2, a=1 and the soname is still the same (libxxx.so.1)

Comment by Mira Carey [ 29/Jan/15 ]

To add a little more fuel to the fire, I'll toss in what I know about version scripts for gnu ld and the solaris linker:

Basically, there's a nifty feature of ELF shared libraries which allows to you bind individual symbols to special ABI symbols. These symbols can inherit from each other and a program which links against one of these libraries will gain a dependency on the ABI symbol which most strictly contains the symbols it needs.

For an example:

VERS_1.0 {
	 global:
		a;
	local:
		*;
};
 
VERS_1.1 {
	global:
		b;
} VERS_1.0;
 
VERS_1.2 {
	global:
		c;
} VERS_1.1;

In the above:

  • A program with a dependency on "a", would pull the VERS_1.0 symbol
  • A program with a dependency on "b", would pull VERS_1.1
  • A program with a dependency on "c", would pull VERS_1.2

Note that this functionality is a superset of export functionality (it hides visibility and then some).

This functionality was first introduced in sun's linker from Solaris 2.5


The gnu linker expands on this features set by adding support for C++, wildcards and different code binding for the same name under different symbols.

C++ support:

{
	global:
		extern "C++" {
			foo*;
			baz::baz*;
			baz::s*;
		};
	local:
		*;
};

Note the use of extern "C++" to demangle names and the use of wildcards to capture the parametric polymorphism.

------

So basically, all you have to do is create a file with an enumerated ABI, then pass a linker flag with --version-script=path/to/version_script and the magic happens. It's kind of nifty having the ABI documented as well.

For some links:

Comment by Andrew Morrow (Inactive) [ 26/Jan/15 ]

Adam -

I think there is something not 100% correct in the above. Looking at example 7 (just because it is visually nearest), it shows the SONAME in the physical library file (meaning not the symlinks), as libmongocxx.so.3.4.0.

So, if I'm developing my 'foo' app, and I say -lmongocxx when linking, my foo app will get a DT_NEEDED entry saying libmongocxx.so.3.4.0. That seems wrong, given the names of the "soname symlink" files that you have, like libmongocxx.so.3. If that is the structure of the soname symlink, I'd expect the SONAME to be libmongocxx.so.3. Otherwise, what do the libmongocxx.so.x links do? They aren't picked up at link time (that is what the dev symlink like libmongocxx.so is for), and they won't get used at runtime (since the DT_NEEDED entry is to the physical library file, not the soname symlinks). Can you clarify?

I'd also like to understand a few other things:

  • Presumably, the project itself uses semver, so we will have LIBMONGOCXX_VERSION_MAJOR, LIBMONGOCXX_VERSION_MINOR, LIBMONGOCXX_VERSION_PATCH or similar. I presume, then, that we will also have LIBMONGOCXX_ABI_CURRENT, LIBMONGOCXX_ABI_REVISION, and LIBMONGOCXX_ABI_AGE macros?
  • What is the relationship between the LIBMONGOCXX_VERSION_* and LIBMONGOCXX_ABI_* macros? What is there pattern of variation?
  • We had planned to use an inline namespace. We had wanted the name of that inline namespace to be injected via a macro, so that it tracked some aspect of the ABI. Do we just use LIBMONGOCXX_ABI_CURRENT token pasted onto v? Or something else? What is the relationship between the inline namespace version and the ABI/VERSION macros?
  • Similar question, but this time re the installation path for the headers. I think there is concensus that we want people to name the library as the leading part of the include (so, #include <libmongocxx/driver/base/client.hpp>). We could either do this by installing headers to $PREFIX/include/libmongocxx, in which case there is no major ABI adornment, preventing side by side installation, but making it easier for users to include the library, as they will simply need to say -I$PREFIX (or nothing, if $PREFIX) is in the default system include path of the toolchain. Or we could install to $PREFIX/include/libmongocxx-XXX/libmongocxx), where we inject a major API (ABI?) tagged directory into the path. This would make it possible to have side by side installations of the headers with different APIs. This would make specifying the include path harder, but this may not matter if we are using pkgconfig. Thoughts?

Sorry for the long list of questions, but I'd like us to take the time to work through these and get it right, as it will save us and our users significant trouble down the road.

Comment by Adam Midvidy [ 23/Jan/15 ]

EDIT: I retract this comment, partially because it is incorrect, but also because I think a different scheme is better.

Original text:

I think we should do things the libtool way.

See: http://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html

ABI version is a triplet (Current, Revision, Age).

Before doing a release:

  • if ABI is completely unchanged we increment revision only
  • if there have been additions to the ABI but no changes or removals, then increment age, increment current, and set revision to 0
  • if there have been backwards incompatible changes to ABI (removals/changes) increment current, set age to 0, set revision to 0

then the SONAME for a version is calculated as libmongocxx.so.(Current-Age).Current.Revision

we provide a symlink for libmongocxx.so that always points to the newest version of the library
we also provide a symlink for each value of current-age that points to the most recent version of the library.

very verbose example:

1. First version of the library is (c=0, r=0, a=0),

  • soname is libmongocxx.so.0.0.0
  • libmongocxx.so is symlinked to libmongocxx.so.0.0.0
  • libmongocxx.so.0 is symlinked to libmongocxx.so.0.0.0

2. second version of the library, no ABI change (c=0, r=1, a=0)

  • soname is libmongocxx.so.0.0.1
  • libmongocxx.so is symlinked to libmongocxx.so.0.0.1
  • libmongocxx.so.0 is symlinked to libmongocxx.so.0.0.1

3. third version of the library with a non-breaking ABI change (c=1, r=0, a=1)

  • soname is libmongocxx.so.0.1.0
  • libmongocxx.so is symlinked to libmongocxx.so.0.1.0
  • libmongocxx.so.0 is symlinked to libmongocxx.0.1.0

4. fourth version of the library with a breaking ABI change (c=2, r=0, a=0)

  • soname is libmongocxx.so.2.2.0
  • libmongocxx.so is symlinked to libmongocxx.so.2.2.0
  • libmongocxx.so.0 is symlinked to libmongocxx.0.1.0
  • libmongocxx.so.2 is symlinked to libmongocxx.2.2.0

5. fifth version of the library with no abi change (c=2, r=1, a=0)

  • soname is libmongocxx.so.2.2.1
  • libmongocxx.so is symlinked to libmongocxx.so.2.2.1
  • libmongocxx.so.0 is symlinked to libmongocxx.0.1.0
  • libmongocxx.so.2 is symlinked to libmongocxx.2.2.1

6. sixth version of the library with breaking abi change (c=3, r=0, a=0)

  • soname is libmongocxx.so.3.3.0
  • libmongocxx.so is symlinked to libmongocxx.so.3.3.0
  • libmongocxx.so.0 is symlinked to libmongocxx.0.1.0
  • libmongocxx.so.2 is symlinked to libmongocxx.2.2.1
  • libmongocxx.so.3 is symlinked to libmongocxx.3.3.0

7. seventh version of the library with non-breaking abi change (c=4, r=0, a=1)

  • soname is libmongocxx.so.3.4.0
  • libmongocxx.so is symlinked to libmongocxx.so.3.4.0
  • libmongocxx.so.0 is symlinked to libmongocxx.0.1.0
  • libmongocxx.so.2 is symlinked to libmongocxx.2.2.1
  • libmongocxx.so.3 is symlinked to libmongocxx.3.4.0

..etc

Generated at Wed Feb 07 21:59:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.