Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-70048

Improve developer experience with golden tests

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Fixed
    • Icon: Major - P3 Major - P3
    • 6.3.0-rc0
    • None
    • None
    • Fully Compatible

    Description

      When a golden test fails because the actual output didn't match the expected output (as opposed to because of a crash during the test), the output isn't very clear/legible:

      It would greatly improve readability if, when this happens, the output dump that follows the error message was either omitted and replaced with a note to run 

      golden diff

       for more details or omitted and replaced with the output of 

      golden diff

       directly.

       

      An idea for an approach would be to change the error message that is logged here and/or here to simplify the messages.

      Also, in the case where a test fails because the actual output didn't match the expected output, the backtrace that is logged doesn't provide any additional information. One way to simplify this would be to create a custom AssertionFailureException class for golden tests that would replace the exception thrown here. This class would not print the stack trace in its constructor like TestAssertionFailure does.

      Attachments

        Activity

          People

            milena.ivanova@mongodb.com Milena Ivanova
            militsa.sotirova@mongodb.com Militsa Sotirova
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: