[COMPASS-7243] Revisit the validation for msgBody.length > AI_MAX_REQUEST_SIZE Created: 20/Sep/23  Updated: 22/Nov/23

Status: Ready for Work
Project: Compass
Component/s: GAI
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Maurizio Casimirri Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to COMPASS-7239 Retry without schema on PROMPT_TOO_LO... Needs Triage
Story Points: 3
Sprint: Iteration Lambeosaurus

 Description   

See also: https://github.com/mongodb-js/compass/pull/4877#pullrequestreview-1636071114

In the AI frontend we are checking if msgBody.length exceeds AI_MAX_REQUEST_SIZE.

Currently there are 2 issues with this solution:

  • This validation limit can go out of sync with the server limits
  • This validation is a catch-all that includes not only the user prompt but also other information like schema and potentially sample documents. As a result, the error handling is not correct and the error for the user might be confusing.

The 2 cases where the user prompt is exceeding limit vs the other data are too big should be handled differently: when the user prompt itself is too long users can simply shorten their prompt, when schema and docs are too large we should strip a subset of the data and let the server handle the rest of the validation.


Generated at Wed Feb 07 22:45:53 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.