-
Type: Task
-
Resolution: Fixed
-
Priority: Unknown
-
None
-
Affects Version/s: None
-
Component/s: AI/ML
-
None
Context
We have seen test failures that are suspicious
- chatgpt-retrieval-plugin. Results imply that the vector search is returning less that "limit" which is the number of documents requested. I have spoken to Trevor M on #vector-search-engineering who said this should not happen.
- I saw this locally once, when I ran in PyCharm upon first running after leaving the test for some time, but additional runs wouldn't reproduce. (Apologies if that is confusing.)
- llama-index. We get different answers to a query.
- I have seen this also happen just once locally, but not reproduce-ably.
- Could it be that the vector search is returning different results? I have to dig into the algorithm, but it does have random components.
Definition of done
What must be done to consider the task complete?
Pitfalls
What should the implementer watch out for? What are the risks?