[ai-ml-pipeline-testing] Refactor VOYAGEAI_API_KEY to VOYAGE_API_KEY for consistency

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Unresolved
    • Priority: Minor - P4
    • None
    • Affects Version/s: None
    • Component/s: ABX
    • None
    • None
    • Python Drivers
    • None
    • None
    • None
    • None
    • None
    • None

      Goal

      Standardize on a single environment variable name for the Voyage AI API key across ai-ml-pipeline-testing. Today the repo carries both VOYAGEAI_API_KEY (from the secrets vault) and VOYAGE_API_KEY (what the voyageai client library reads by default), with .evergreen/provision-atlas.sh exporting the same value under both names. Collapse this to VOYAGE_API_KEY so there is one name to reason about.

      Current state (verified on branch auto-embedding-preview, working tree)

      Item Status
      .evergreen/provision-atlas.sh exports both VOYAGEAI_API_KEY and VOYAGE_API_KEY from the same secret (lines 26-27)
      .evergreen/setup-remote.sh exports only VOYAGEAI_API_KEY (line 70), so remote-Atlas tasks never get VOYAGE_API_KEY
      .evergreen/mongodb-community-search/start-services.sh derives VOYAGE_QUERY_API_KEY/VOYAGE_INDEXING_API_KEY from VOYAGEAI_API_KEY
      pymongo-voyageai/run.sh re-exports VOYAGEAI_API_KEY
      pymongo-voyageai-multimodal code/tests client.py and tests/test_client_integration.py read VOYAGEAI_API_KEY
      pymongo-voyageai-multimodal docs inconsistent: README.md and docs/index.rst use VOYAGE_API_KEY; docs/examples.rst uses VOYAGEAI_API_KEY
      Secrets vault key is stored as VOYAGEAI_API_KEY in drivers/ai-ml-pipeline-testing (fetched by .evergreen/fetch-secrets.sh)

      Prerequisites not in the repo docs

      1. The name originates in AWS Secrets Manager under drivers/ai-ml-pipeline-testing, consumed via drivers-evergreen-tools/.evergreen/secrets_handling/setup-secrets.sh. Renaming only in this repo will not work on its own: either the vault entry is added/renamed to VOYAGE_API_KEY, or the shell scripts keep a single translation line at the point the secrets are sourced.
      2. Downstream sibling projects (langchain-python, langchain-js, mastra, etc.) may consume VOYAGEAI_API_KEY from the generated env.sh. Grep each subproject before dropping the old export.

      Steps

      1. Add VOYAGE_API_KEY to the drivers/ai-ml-pipeline-testing vault entry (or map it once immediately after source secrets-export.sh).
      2. Update .evergreen/provision-atlas.sh to export only VOYAGE_API_KEY.
      3. Update .evergreen/setup-remote.sh to export VOYAGE_API_KEY.
      4. Update .evergreen/mongodb-community-search/start-services.sh to source VOYAGE_QUERY_API_KEY/VOYAGE_INDEXING_API_KEY from VOYAGE_API_KEY.
      5. Update pymongo-voyageai/run.sh.
      6. Update pymongo-voyageai-multimodal: pymongo_voyageai_multimodal/client.py, tests/test_client_integration.py, and docs/examples.rst (README and docs/index.rst already use the new name).
      7. Grep the repo for any remaining VOYAGEAI_API_KEY and remove the vault entry / compatibility shim once CI is green.
      8. Run the affected Evergreen tasks (local Atlas, remote Atlas, community-with-search) to confirm nothing silently loses the key.

      Acceptance criteria

      • grep -r VOYAGEAI_API_KEY over the repo returns no hits.
      • Local Atlas, remote Atlas, and community-with-search Evergreen variants all pass with the key supplied only as VOYAGE_API_KEY.
      • pymongo-voyageai-multimodal docs and code agree on one variable name.

      Decisions needed

      1. Vault rename vs. shim. Rename the secret in drivers/ai-ml-pipeline-testing to VOYAGE_API_KEY, or keep the vault name and translate once in fetch-secrets.sh/provision-atlas.sh? A rename is cleaner but coordinates with anything else reading that vault entry.
      2. Transition window. Drop VOYAGEAI_API_KEY immediately, or export both for one release cycle so downstream subprojects can catch up?

      Notes

      • VOYAGE_API_KEY is the name the voyageai Python client picks up automatically, which is why the dual export exists at all - standardizing on it removes the need for any explicit plumbing in most subprojects.
      • VOYAGEAI_S3_BUCKET is a separate variable and is out of scope here.

            Assignee:
            Unassigned
            Reporter:
            Casey Clements
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated: