[SERVER-66135] lookup_pit_pre_and_post_image_in_transaction.js never succeeds in uploading logs to logkeeper Created: 03/May/22  Updated: 29/Oct/23  Resolved: 23/May/22

Status: Closed
Project: Core Server
Component/s: Change streams, Testing Infrastructure
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: Bug Priority: Major - P3
Reporter: Max Hirschhorn Assignee: Rishab Joshi (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-67073 Failures appear to be masked by setup... Closed
is related to SERVER-66062 make sure jstests/change_streams/ddl_... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: QE 2022-05-16, QE 2022-05-30
Participants:
Linked BF Score: 141

 Description   

Logkeeper has a 4MB line length limit but the jstests/change_streams/lookup_pit_pre_and_post_image_in_transaction.js test is consistently generating several 7MB log lines.

$ python buildscripts/resmoke.py run --suite=change_streams_mongos_passthrough jstests/change_streams/lookup_pit_pre_and_post_image_in_transaction.js > temp.log
$ awk '{print length}' temp.log | sort -n | uniq -c | tail -n 3
      1 3846
     16 7340097
     16 7340098

This causes tests which fail in the same execution task to become "setup failed" (lavender) rather than "failed" (red) in Evergreen. Setup failures are intentionally ignored by the Build Barons so this can lead to delays in the timeliness of identifying true failures. (Setup failures are ignored because logkeeper instability has been generally accepted and accommodated within the testing infrastructure, see SERVER-35472.)

[2022/05/02 20:52:55.994] [fallback] Encountered an HTTP error: 400 Client Error: Bad Request for url: https://logkeeper.mongodb.org/build/604b1fc3e6d8c5e6d9e2fd5da06b419f/test/6270449ebe07c46a236738a7/
[2022/05/02 20:52:58.104] [fallback] Encountered an HTTP error: 400 Client Error: Bad Request for url: https://logkeeper.mongodb.org/build/604b1fc3e6d8c5e6d9e2fd5da06b419f/test/6270449ebe07c46a236738a7/
[2022/05/02 21:01:48.569] [fallback] Failed to flush all log output (2205 messages) to logkeeper.

https://evergreen.mongodb.com/lobster/evergreen/task/mongodb_mongo_master_enterprise_rhel_80_64_bit_dynamic_all_feature_flags_required_change_streams_mongos_passthrough_561d65eebd69ee00c98e8f19592d9f1eb9798a7e_22_05_02_19_26_14/0/task#bookmarks=0%2C2144&f~=100~%5C%5Bfallback%5C%5D&l=1

The 7MB log lines in the test success case appears to be caused by how ChangeStreamTest#assertNextChangesEqual() uses JavaScript exceptions for control flow. assertChangeStreamEventEq() throwing an exception is caught by _convertExceptionToReturnStatus() but catching the exception thrown by an assert.xx() function still logs an assertion message.



 Comments   
Comment by Githook User [ 23/May/22 ]

Author:

{'name': 'Rishab Joshi', 'email': 'rishab.joshi@mongodb.com', 'username': 'rishvin'}

Message: SERVER-66135 Introduce function to transform a large change event for debugging.
Branch: master
https://github.com/mongodb/mongo/commit/10945bc699a008934c360b922cdd9ac5e6332af8

Generated at Thu Feb 08 06:04:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.