-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
Currently if folks use the generative ai feature in Compass with a collection with documents with a lot of fields, we'll show an error:
"Sorry, your collections have too many fields to process. Please try
using this feature on a collection with smaller documents."
PROMPT_TOO_LONG error from mms
This is because we don't want to exceed the token limit of open ai. We recently changed from gpt-3.5-turbo to gpt-4o. Lets increase the character restriction now to a higher amount that matches what might be tokenized then: https://platform.openai.com/tokenizer
Context window tokens here: https://platform.openai.com/docs/models/gpt-4o
4o: 128,000 tokens
3.5-turbo: 16,385 tokens
Since we want the final validation to be on the server lets do 10x what the server limit is at. The server does some further changes to schema and sample documents so we should let it be the decider. The frontend should only prevent uploading massive schemas and documents in the request.
Let's also update the messaging for the PROMPT_TOO_LONG error. We can make it more transparent what the issue is, and remove the plural 's' on collections.