Create small reproducible issue demonstrating slow search indexing

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Won't Do
    • Priority: Unknown
    • None
    • Affects Version/s: None
    • Component/s: AI/ML
    • None
    • None
    • Python Drivers
    • Hide

      1. What would you like to communicate to the user about this feature?
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
      3. Which versions of the driver/connector does this apply to?

      Show
      1. What would you like to communicate to the user about this feature? 2. Would you like the user to see examples of the syntax and/or executable code and its output? 3. Which versions of the driver/connector does this apply to?
    • None
    • None
    • None
    • None
    • None
    • None

      Context

      Our integration testing pipelines, to run green, have to add retries with significant sleeps. The upper bounds on the sleep required for these test appears to be higher than the mongot team expects. Beyond our own testing, our impression is that they will not give a good  developer experience.

      https://spruce.mongodb.com/commits/ai-ml-pipeline-testing?view=FAILED

      Definition of done

      The task here is to create a test that mimics the apparatus that we use everywhere (see for example, in langchain [here|https://github.com/langchain-ai/langchain/blob/master/libs/partners/mongodb/tests/utils.py|https://github.com/langchain-ai/langchain/blob/master/libs/partners/mongodb/tests/utils.py])]

      We do not want any external dependency, except perhaps openai, though it would be better not to share that key.. We CAN include programmatic index creation, for simplicity, although that isn't the cause of the problem.

      One need just add N documents with an indexed vector-search field, and then run a $vectorSearch on it until N documents are returned. 

      All the pieces can be quickly gathered from langchain-mongodb.

      Pitfalls

            Assignee:
            Casey Clements
            Reporter:
            Casey Clements
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: