-
Type: Task
-
Resolution: Gone away
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: GAI
-
3
-
Iteration Lambeosaurus
-
Not Needed
See also: https://github.com/mongodb-js/compass/pull/4877#pullrequestreview-1636071114
In the AI frontend we are checking if msgBody.length exceeds AI_MAX_REQUEST_SIZE.
Currently there are 2 issues with this solution:
- This validation limit can go out of sync with the server limits
- This validation is a catch-all that includes not only the user prompt but also other information like schema and potentially sample documents. As a result, the error handling is not correct and the error for the user might be confusing.
The 2 cases where the user prompt is exceeding limit vs the other data are too big should be handled differently: when the user prompt itself is too long users can simply shorten their prompt, when schema and docs are too large we should strip a subset of the data and let the server handle the rest of the validation.
- related to
-
COMPASS-7239 Retry without schema on PROMPT_TOO_LONG and MESSAGE_TOO_LONG.
- Open