Loading...

Type: Bug
Resolution: Fixed
Priority: Major - P3
Fix Version/s: 9.0.0-rc0
Affects Version/s: None
Component/s: None
Labels:
None

Assigned Teams:

Query Integration
Backwards Compatibility:
Fully Compatible
Operating System:
ALL
Linked BF Score:
200
CAR Domain/s:
None

Aha! Reference:
None
Tracking Level:
None
Risk Status:
None
Exec Notes:
None
Goal Name(s):
None
Goal Link:
None

Host: macos-14-arm64-23.mac.devprod.mongodb.com
Project: mongodb-mongo-master
Commit: 39d1720f
Please refer to BF(G) Playbook for instructions on handling BF and BFG tickets as well as Auto-Resolution Rules

Failing Test Logs:

default_max_time_ms_metrics.js - Logs | History | GitHub

Task Logs:

task auth_4-macos

Task Log Assertion Extractor

Logs:

       ~~~~~~~~~~~~~~~~~~~~~~^
        bucket=Bucket,
        ^^^^^^^^^^^^^^
    ...<3 lines>...
        callback=Callback,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/transfer.py", line 406, in download_file
    future.result()
    ~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 264, in result
    raise self._exception
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/tasks.py", line 265, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/download.py", line 352, in _submit
    response = client.head_object(
        Bucket=transfer_future.meta.call_args.bucket,
        Key=transfer_future.meta.call_args.key,
        **transfer_future.meta.call_args.extra_args,
    )
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/s3_binary/download.py", line 164, in _download_and_verify
    download_from_s3_with_boto(s3_path, output_path)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/util/download_utils.py", line 77, in download_from_s3_with_boto
    s3_client.download_file(bucket_name, object_key, output_file)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/inject.py", line 192, in download_file
    return transfer.download_file(
           ~~~~~~~~~~~~~~~~~~~~~~^
        bucket=Bucket,
        ^^^^^^^^^^^^^^
    ...<3 lines>...
        callback=Callback,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/transfer.py", line 406, in download_file
    future.result()
    ~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 264, in result
    raise self._exception
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/tasks.py", line 265, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/download.py", line 352, in _submit
    response = client.head_object(
        Bucket=transfer_future.meta.call_args.bucket,
        Key=transfer_future.meta.call_args.key,
        **transfer_future.meta.call_args.extra_args,
    )
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/util/download_utils.py", line 68, in download_from_s3_with_boto
    s3_client.download_file(bucket_name, object_key, output_file)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/inject.py", line 192, in download_file
    return transfer.download_file(
           ~~~~~~~~~~~~~~~~~~~~~~^
        bucket=Bucket,
        ^^^^^^^^^^^^^^
    ...<3 lines>...
        callback=Callback,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/transfer.py", line 406, in download_file
    future.result()
    ~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 264, in result
    raise self._exception
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/tasks.py", line 265, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/download.py", line 352, in _submit
    response = client.head_object(
        Bucket=transfer_future.meta.call_args.bucket,
        Key=transfer_future.meta.call_args.key,
        **transfer_future.meta.call_args.extra_args,
    )
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/s3_binary/download.py", line 164, in _download_and_verify
    download_from_s3_with_boto(s3_path, output_path)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/util/download_utils.py", line 77, in download_from_s3_with_boto
    s3_client.download_file(bucket_name, object_key, output_file)
    ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/inject.py", line 192, in download_file
    return transfer.download_file(
           ~~~~~~~~~~~~~~~~~~~~~~^
        bucket=Bucket,
        ^^^^^^^^^^^^^^
    ...<3 lines>...
        callback=Callback,
        ^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/boto3/s3/transfer.py", line 406, in download_file
    future.result()
    ~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
           ~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/futures.py", line 264, in result
    raise self._exception
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/tasks.py", line 265, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/s3transfer/download.py", line 352, in _submit
    response = client.head_object(
        Bucket=transfer_future.meta.call_args.bucket,
        Key=transfer_future.meta.call_args.key,
        **transfer_future.meta.call_args.extra_args,
    )
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 569, in _api_call
    return self._make_api_call(operation_name, kwargs)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/botocore/client.py", line 1023, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/s3_binary/download.py", line 171, in _download_and_verify
    download_from_s3_with_requests(s3_path, output_path, raise_on_error=True)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ec2-user/data/mci/9114e781ec92c267ad1fd548f16d888c/src/buildscripts/util/download_utils.py", line 59, in download_from_s3_with_requests
    reader.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/data/mci/9114e781ec92c267ad1fd548f16d888c/venv/lib/python3.13/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://mciuploads.s3.amazonaws.com/mongodb-mongo-master/macos-debug-suggested/mongodb_mongo_master_39d1720f367a45af12011b1816029bcce041a04c/binaries/mongo-85756.zst

logs

Task Log Assertion Extractor

Logs:

ERROR: curl: (22) The requested URL returned error: 403

logs

Fatal Backtrace

Logs:

Assertion while executing command
command: find
db: default_max_time_ms_metrics
commandArgs: {'find': 'test', 'filter': {'$where': 'sleep(1000); return true;'}, 'maxTimeMS': 2000, 'lsid': {'id': {'$uuid': '9eecc522-ac10-4935-b4c2-9ada1db05371'}}, '$clusterTime': {'clusterTime': {'$timestamp': {'t': 1780518454, 'i': 25}}, 'signature': {'hash': {'$binary': {'base64': 'Rvz10TwotK+1YRHj5cJ0UnT2TKI=', 'subType': '0'}}, 'keyId': 7647268525559513096}}, '$traceCtx': {'traceparent': '00-0d09ea4d3bba66b0679fce50a50499e1-2a8b0e34acce68c9-01'}, '$db': 'default_max_time_ms_metrics'}
error: Interrupted: error while executing at wasm backtrace:
    0: 0x5178eb - <unknown>!<wasm function 35936>

Caused by:
    wasm trap: interrupt

logs

Fatal Backtrace

Logs:

Clearing all idle JS contexts due to out of memory

logs

Fatal Backtrace

Logs:

Slow query
type: command
isFromUserConnection: True
isFromPriorityPortConnection: False
ns: default_max_time_ms_metrics.test
collectionType: normal
appName: MongoDB Shell
queryShapeHash: 143F9EBBA5388D433380BA0025FBCD9CABC3BDFC7113D28D83961B4BD83DCAE2
command: {'find': 'test', 'filter': {'$where': 'sleep(1000); return true;'}, 'maxTimeMS': 2000, 'lsid': {'id': {'$uuid': '9eecc522-ac10-4935-b4c2-9ada1db05371'}}, '$clusterTime': {'clusterTime': {'$timestamp': {'t': 1780518454, 'i': 25}}, 'signature': {'hash': {'$binary': {'base64': 'Rvz10TwotK+1YRHj5cJ0UnT2TKI=', 'subType': '0'}}, 'keyId': 7647268525559513096}}, '$traceCtx': {'traceparent': '00-0d09ea4d3bba66b0679fce50a50499e1-2a8b0e34acce68c9-01'}, '$db': 'default_max_time_ms_metrics'}
opid: 59397
planningTimeMicros: 4958724
numYields: 0
ok: 0
errMsg: error while executing at wasm backtrace:
    0: 0x5178eb - <unknown>!<wasm function 35936>

Caused by:
    wasm trap: interrupt

errName: Interrupted
errCode: 11601
reslen: 340
locks: {'Global': {'acquireCount': {'r': 1}}}
remote: 10.0.30.101:64895
protocol: op_msg
numInterruptChecks: 6
priorityLowered: False
wasMarkedNonDeprioritizable: False
delinquencyInfo: {'totalDelinquentAcquisitions': 1, 'totalAcquisitionDelinquencyMillis': 5448, 'maxAcquisitionDelinquencyMillis': 5448}
queues: {'ingress': {'admissions': 1, 'totalTimeQueuedMicros': 0}, 'execution': {'admissions': 2, 'totalTimeQueuedMicros': 0}}
workingMillis: 5448
interruptLatencyNanos: 1983314042
deadline: {'$date': '2026-06-03T20:27:42.774Z'}
durationMillis: 5448

logs

Fatal Backtrace

Logs:

uncaught exception: Error: command did not fail with any of the following codes [ 50 ] {
"ok" : 0,
"errmsg" : "error while executing at wasm backtrace:\n    0: 0x5178eb - <unknown>!<wasm function 35936>\n\nCaused by:\n    wasm trap: interrupt\n",
"code" : 11601,
"codeName" : "Interrupted",
"$clusterTime" : {
"clusterTime" : Timestamp(1780518454, 25),
"signature" : {
"hash" : BinData(0,"Rvz10TwotK+1YRHj5cJ0UnT2TKI="),
"keyId" : NumberLong("7647268525559513096")
}
},
"operationTime" : Timestamp(1780518454, 25)
}. errmsg: error while executing at wasm backtrace:
0: 0x5178eb - <unknown>!<wasm function 35936>

Caused by:
wasm trap: interrupt
: { res: { ok: 0.0, errmsg: "error while executing at wasm backtrace:
0: 0x5178eb - <unknown>!<wasm function 35936>

Caused by:
wasm trap: interrupt
", code: 11601, codeName: "Interrupted", $clusterTime: { clusterTime: Timestamp(1780518454, 25), signature: { hash: BinData(0, 46FCF5D13C28B4AFB56111E3E5C2745274F64CA2), keyId: 7647268525559513096 } }, operationTime: Timestamp(1780518454, 25) }, expectedCode: [ 50.0 ] } :
_getErrorWithCode@src/mongo/shell/utils.js:32:13
doassert@src/mongo/shell/assert.js:47:14
_doassert@src/mongo/shell/assert.js:181:13
_assertCommandFailed@src/mongo/shell/assert.js:1328:26
assert.commandFailedWithCode@src/mongo/shell/assert.js:1536:12
assertCommandFailedWithMaxTimeMSExpired@jstests/auth/default_max_time_ms_metrics.js:68:16
runTests@jstests/auth/default_max_time_ms_metrics.js:82:44
@jstests/auth/default_max_time_ms_metrics.js:115:9

logs

Fatal Backtrace

Logs:

Error: command did not fail with any of the following codes [ 50 ] {
"ok" : 0,
"errmsg" : "error while executing at wasm backtrace:\n    0: 0x5178eb - <unknown>!<wasm function 35936>\n\nCaused by:\n    wasm trap: interrupt\n",
"code" : 11601,
"codeName" : "Interrupted",
"$clusterTime" : {
"clusterTime" : Timestamp(1780518454, 25),
"signature" : {
"hash" : BinData(0,"Rvz10TwotK+1YRHj5cJ0UnT2TKI="),
"keyId" : NumberLong("7647268525559513096")
}
},
"operationTime" : Timestamp(1780518454, 25)
}. errmsg: error while executing at wasm backtrace:
0: 0x5178eb - <unknown>!<wasm function 35936>

Caused by:
wasm trap: interrupt
failed to load: jstests/auth/default_max_time_ms_metrics.js
exiting with code -3

logs

Repro Artifacts:

Resmoke.py Invocation for Local Usage

fixes

SERVER-126787 WASM JS engine surfaces Interrupted error code instead of the real kill reason when a wasmtime epoch interrupt fires

Closed

Details

Description

Attachments

Issue Links

Activity

People

Dates

PagerDuty