-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
Let's add 5 (or more) tests to our generative ai accuracy tests. Let's aim to make these more complex generations and hopefully hit some of the negative feedback received from https://docs.google.com/spreadsheets/d/1vhzZorEFSLsP5UYJzI6Jc0Tv-Q5sWvE_ny9E0npig8k/edit#gid=1482967694
Accuracy tests: https://github.com/mongodb-js/compass/blob/main/packages/compass-generative-ai/scripts/ai-accuracy-tests.ts
Ideally these will serve as a way for future model evaluations to show their improvements.
Let's be sure we do one with nested documents and nested arrays.
- related to
-
COMPASS-7896 Add accuracy tests
- Closed