Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17520

Add support for pluggable FTS tokenizers

    • Type: Icon: Task Task
    • Resolution: Done
    • Priority: Icon: Major - P3 Major - P3
    • 3.1.1
    • Affects Version/s: None
    • Component/s: Text Search
    • None
    • Fully Compatible
    • Platform 1 04/03/15
    • None
    • 0
    • None
    • None
    • None
    • None
    • None
    • None

      To support third-party tokenizers, Mongo needs to create an abstract interface for document tokenization.

      1. Create an abstract interface
      2. Move all code to use the new interface except V1 legacy
      3. Create an implementation for our
      class FtsTokenizer {
          virtual ~FtsTokenizer()
          virtual void reset(const char* document) = 0; // Process a new doc
          virtual bool moveNext() = 0;	    // Moves to the next token
          virtual StringData& getStem() = 0;  // Returns stemmed form
      };
      

            Assignee:
            mark.benvenuto@mongodb.com Mark Benvenuto
            Reporter:
            mark.benvenuto@mongodb.com Mark Benvenuto
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: