Problem

As a result of WT-14536, the shared metadata table ID was made predefined. This introduced a conflict with the key provider table ID at the PALI level, as both now use the same ID.

The shared metadata ID is 2 because the first three bits of table IDs are reserved for namespaces and the special namespace sets these bits to 2, and the metadata ID within this namespace is 0.

The key provider ID is always set to 2 without using any namespace, which does not conform to the current table ID design. However, this is how it has been defined since the disagg great merge.

Another table ID that does not use namespaces, but luckily does not cause a conflict, is the turtle file ID, which is set to 1.

With the current ID assignment scheme, using KEK leads to very unpredictable and non-obvious failures (for example, https://jira.mongodb.org/browse/HELP-89290).

Possible solutions

The solution that conforms to the design would be to place both the turtle and key provider IDs into the special namespace. However, this would be a format-breaking change and would likely affect all existing disagg clusters.

Another option is to make the turtle ID an exception (since it does not currently cause conflicts) and change only the key provider ID. This would likely affect KEK-enabled clusters only.

We could also make both the key provider ID and the turtle ID exceptions and instead change the shared metadata ID (which was modified last week, shortly before the format-breaking changes merge cutoff). This approach would likely affect all disagg clusters as well.

It is worth noting that changing either the key provider ID or the turtle table ID will require updates in PALI, as these values are currently hardcoded there.