Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43117

Create oplog entry applier registry



    • Type: New Feature
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: Backlog
    • Component/s: Replication
    • Labels:


      This idea is originally from Siyuan Zhou as part of the Address tech debt in oplog application code path project.

      The logic to apply command oplog entries is all in oplog.cpp. The command-appliers are implemented as lambdas, which are defined inline in a single 400-line statement that initializes a map called kOpsMap. The logic that applies insert, update, and delete oplog entries is implemented separately in a single 400-line function.

      Having all the applier logic in oplog.cpp couples this file with many other subsystems, such as DDL and transactions. Thus the file must include headers from all over the codebase, and the file must be linked with 18 library dependencies.

      Solution: Make an OpApplier interface. Each subsystem implements OpApplier subclasses for the commands it can apply. Create an OpApplierRegistry (similar to CommandRegistry) where subsystems can register command appliers during startup. This will minimize oplog.cpp's includes and link dependencies. Remove unnecessary headers by guesswork and trial and error. If the Libdeps Linter project is ready, use it to remove link dependencies, otherwise we must remove link dependencies by guesswork too.

      Consider implementing insert, update, and delete ops with the same OpApplier interface as commands. Note that, for performance reasons, there is special logic for "grouped inserts" (in InsertGroup), which should remain special.

      Note from Andrew Morrow: "Avoid doing implicit self-registration of types by creating static instances of the class, like the Command registry does. Doing so creates libraries that don't satisfy symbol references, but silently break things if they drop out of the graph. It makes it much harder to manipulate the library dependency graph."

      In other words, let's not continue to use the sort of pattern we see in replication_info.cpp, for example:

      class CmdIsMaster : public CommandBase {
       } cmdIsMaster;

      That source file isn't required to satisfy any symbol references: If you forget to link to that file, you won't get a linker error about an undefined symbol. However, mongod won't work, because the singleton instance "cmdIsMaster" must be constructed in order to add itself to the command registry.




            backlog-server-repl Backlog - Replication Team
            jesse A. Jesse Jiryu Davis
            0 Vote for this issue
            7 Start watching this issue