Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-10074

Create version of the wt utility that works with Queryable Backup

    • Type: Icon: New Feature New Feature
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Storage Engines
    • StorEng - Refinement Pipeline

      Summary
      Top provide another option for examining data from customer systems, we should create a version of the wt utility that compiles with the queryable backup file system extension. This would let us run wt against a cloud backup of a MongoDB instance.

      Motivation

      Occasionally we have an issue where it would be useful to examine a customer database (with their agreement, of course). A challenge we have faced in the past is generating the correct version of wt with debugging enabled for appropriate the OS and MongoDB Version and then installing and running it on a replica in Atlas.

      WT-10068 addresses the problem of building these wt versions. This ticket provides another potential tool for examining user data in these circumstances. If the customer/user has backups in the cloud capable of supporting queryable backup, we could use that to access a recent snapshot of their data. 

      To make this work, we would need to build a version of wt that uses the blockstore file system extension created to support queryable backup. This is a WT_FILE_SYSTEM extension that provides read-only access to a backup snapshot stored (I believe) in S3. 

      To make this work, we would need a way to pass any configuration parameters to this extension at startup. We would also need to set up anything that normally would be provided by the server when running configurable backup – possibly network connections, authentication to AWS, etc.

      Acceptance Criteria (Definition of Done)

      • Design, implement, and test a version of wt that works via Queryable Backup
      • Have a build option to create it 
      • Decide on which (if any) older version we would want to support with this feature and create the appropriate backport tickets
      • Write documentation (and decide where to put it) about how to build and use this tool.

      Commentary

      I don't know that it is worth the effort to figure out how to do this. I expect that WT-10068 will address most of the issues we've had here. But I wanted to capture this idea for posterity.

            Assignee:
            backlog-server-storage-engines [DO NOT USE] Backlog - Storage Engines Team
            Reporter:
            keith.smith@mongodb.com Keith Smith
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: