Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-98194

Minimise recorded response data

    • Type: Icon: New Feature New Feature
    • Resolution: Unresolved
    • Priority: Icon: Blocker - P1 Blocker - P1
    • None
    • Affects Version/s: None
    • Component/s: None
    • Query Optimization

      Motivation for recording response body:

      Workload recordings will include getMore requests. getMore references the cursor data returned by an earlier query. When the replay client replays the original (e.g.,) find, the cursor data will be different.

      To support getMore the replay client will need to maintain a map of "cursor data in recording" to "regenerated cursor data during replay".

      This requires the recording to include the cursor data returned from a given query, so later getMore requests for the same cursor can be patched up before being replayed.

      Additionally, metrics derived from the response body (e.g., number of documents returned) are likely to be useful when validating replays.

      Current state:

      The entire query response is recorded

      Problem:

      This may be a significant amount of data, and currently we have no use for the vast majority of it.

      Goal:

      Reduce the recorded data by extracting the cursor data from the response body, and establish what other information should be retained from the response data of other commands.

      E.g., for updateMany

      > db.coll2.updateMany({}, [])
      { "acknowledged" : true, "matchedCount" : 9970, "modifiedCount" : 0 }
      

      This is a small response, and the information contained will be useful for validation.

      I'd suggest structuring this as "record the full response by default, but for special cases for x, y, z, record only specific values". E.g., we know that the size of a find/getMore response is very variable, depends on the customer data, and the specific values are not currently going to be used when validating the replay. Thus, for that particular command, we may chose to extract the cursor data, and "N documents returned".

            Assignee:
            Unassigned Unassigned
            Reporter:
            james.harrison@mongodb.com James Harrison
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: