Filter message history to ensure it does not exceed model.maxInputTokens

XMLWordPrintableJSON

    • Type: Task
    • Resolution: Fixed
    • Priority: Major - P3
    • 1.11.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Not Needed
    • Developer Tools

      Currently we do not have any limits set on how many messages we send to the model as context but for very long running chats, we can likely reach its input token limit and start getting errors. We should consider the size of our history and potentially filter older messages as it gets overly large.

       

      We'd want to use model.maxInputTokens as our limit.

            Assignee:
            Gagik Amaryan
            Reporter:
            Gagik Amaryan
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: