Goal

Create the S3 bucket the langchain-mongodb-deepagents-vfs e2e tests need, grant access to both principals that must write to it, and get the real_e2e suites running in CI rather than silently skipping.

Background

tests/e2e_tests/test_real_e2e.py and tests/e2e_tests/test_watcher_e2e.py need a writable S3 bucket. No bucket has ever existed for this project, so these tests have been skipping — which is how the Python 3.14 dependency problems (INTPYTHON-1054, INTPYTHON-1065) reached main unnoticed.

The suites are not merely unrun, they are unrunnable: _require_env skips the session when S3_BUCKET_NAME is unset, and the s3_access preflight fails the session when the bucket is not writable.

Two AWS accounts are involved

This is the part most easily missed. The permission gap exists in two places, and fixing one does not fix the other.

Verbatim, from a local attempt:

An error occurred (AccessDenied) when calling the CreateBucket operation: User:
arn:aws:sts::857654397073:assumed-role/AWSReservedSSO_drivers-test-secrets-role_f4acc14f7c0898ab/casey.clements@mongodb.com
is not authorized to perform: s3:CreateBucket

and from an Evergreen run against an existing unrelated bucket:

An error occurred (AccessDenied) when calling the PutObject operation: User:
arn:aws:sts::557821124784:assumed-role/evergreen_task_hosts_instance_role_production/i-0ba8678f8ebc654e6
is not authorized to perform: s3:PutObject on resource:
"arn:aws:s3:::pymongo-voyageai/langchain_mongodb_deepagents_vfs_e2e_test/_preflight_..."

Both are implicit denies ("no identity-based policy allows"), not explicit Deny from an SCP, so both are grantable by policy.

If the bucket is created with access only for the SSO role, local runs will work and CI will still fail. Cross-account access for the Evergreen role is best granted by a bucket policy naming that role, rather than widening its identity policy — it is a shared EC2 instance role used by every Evergreen task host, so the blast radius of an identity-policy change is large.

What is needed

1. Bucket langchain-mongodb-deepagents-vfs in us-east-1.
2. Grant s3:PutObject, s3:GetObject, s3:DeleteObject, s3:ListBucket to:
   • the SSO role in 857654397073 (local development), and
   • evergreen_task_hosts_instance_role_production in 557821124784 (CI).
3. A Secrets Manager entry whose value is the bucket name, consumed by the existing fetch script and exported as S3_BUCKET_NAME. Note the convention: secret key underscored, value dashed.
4. Lifecycle rule expiring objects under langchain_mongodb_deepagents_vfs_e2e_test/ after ~7 days. The fixtures clean up after themselves, but a crashed run leaves orphans.
5. Wire the real_e2e suites into CI so they actually execute.

A request for the bucket has already been raised with Enterprise Security; this ticket tracks the whole task, including the CI-account grant and the test wiring.

Note on embeddings

The default EMBEDDING_PROVIDER is bedrock, and the SSO role has no bedrock:InvokeModel — verified directly, not merely ListFoundationModels. Runs must set EMBEDDING_PROVIDER=openai, or the backend starts, fails every key during initial sync, and aborts with Backend initialization reported errors. Whatever role runs these in CI needs either Bedrock invoke permission or an OpenAI/Azure key.

Interim: MinIO

Both suites have been verified against MinIO standing in for S3, with real Atlas and real embeddings. conftest.py deliberately exempts real_e2e from its AWS-env stripping, and botocore honours AWS_ENDPOINT_URL_S3 natively, so no code change is required.

Watcher timings confirm real exercise rather than vacuous passes: with a 10s poll interval the single-event test landed at ~10s and the two-phase tests at ~21s, i.e. one and two poll cycles.

This covers hybrid $rankFusion grep, Atlas Search and Vector Search index provisioning, ETag-diff change detection and watcher prefix isolation — but not anything genuinely AWS-specific: IAM behaviour, multipart ETag semantics, eventual consistency, real network timeouts. Setup is documented in MINIO_E2E_TESTING.md.

Acceptance criteria

• uv run pytest tests/e2e_tests/test_real_e2e.py tests/e2e_tests/test_watcher_e2e.py passes against the real bucket from a developer machine.
• The same suites pass in Evergreen CI, executing rather than skipping.
• A skipped or preflight-failed e2e run is visible in CI output rather than passing silently.
• Bucket has a lifecycle rule bounding test-object retention.