Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-17520

Add support for pluggable FTS tokenizers

    XMLWordPrintableJSON

Details

    • Icon: Task Task
    • Resolution: Done
    • Icon: Major - P3 Major - P3
    • 3.1.1
    • None
    • Text Search
    • None
    • Fully Compatible
    • Platform 1 04/03/15

    Description

      To support third-party tokenizers, Mongo needs to create an abstract interface for document tokenization.

      1. Create an abstract interface
      2. Move all code to use the new interface except V1 legacy
      3. Create an implementation for our

      class FtsTokenizer {
          virtual ~FtsTokenizer()
          virtual void reset(const char* document) = 0; // Process a new doc
          virtual bool moveNext() = 0;	    // Moves to the next token
          virtual StringData& getStem() = 0;  // Returns stemmed form
      };
      

      Attachments

        Activity

          People

            mark.benvenuto@mongodb.com Mark Benvenuto
            mark.benvenuto@mongodb.com Mark Benvenuto
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: