-
Type:
Task
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Storage Engines
-
None
-
None
To implement the publish API for the followers, we would have to make changes to how the metadata operation queue (conn->disaggregated_storage.shared_metadata_qh) works on the followers. The current assumption is that the queue is empty, but we could reuse it to keep pending metadata operations that we could replay during node step-up.
Specifically, this means:
- Enable the publish API for followers, if disabled.
- Enqueue metadata operations from table creation and table drop on followers by removing the check for leaders. Note that table creation would not be able to supply the metadata for the corresponding stable table, as a follower is not allowed to create it.
- When picking up a checkpoint, prune all operations from the metadata operation queue that have the schema_epoch set, and where the schema_epoch is included based on checkpoint_schema_epoch from the checkpoint's metadata.
- If practical, optionally (e.g., in the diagnostic mode, or when enabled via a special flag) perform checks that the pruned are indeed included in the checkpoint's shared metadata table (not the local metadata table). Otherwise split this work into a separate ticket.
- Modify __layered_create_missing_stable_tables (which is called during the step up) to create any missing stable tables based on the contents of the metadata operation queue and fill in the appropriate stable_value fields in the elements of the queue.