Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-5042

Implement support for reliable change notifications

    • Type: Icon: New Feature New Feature
    • Resolution: Duplicate
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: Querying
    • Labels:
    • Query

      This features enables users to subscribe to change event stream (notifications) resulting from changes made to documents in a collection in mongodb. The user can specify a filter to be applied to change events. The resulting stream would go into a capped collection created specifically for this subscription. In addition, mongodb will keep a position in this capped collection reflecting the last message consumed by the client so that the client could resume reading the event stream where it left off.

      Here is a more detailed version of how this should work:
      1. Let's assume a mongodb has a collection called Item
      2. User registers a subscription to events of interest in the collection Item
      • A subscription filter is provided
      • A destination capped collection name is specified, example: Item_changes; it serves as subscription queue
      • The _id field in this collection contains a monotonously increasing sequence counter in the lower bits and date in the upper bits
      o Host name, mac address, or any other such things are not present in the _id making it possible for the client to construct the _id by date only; this makes it possible for consumer of these records to indicate the starting point for scanning the notification records having just the starting date or the id the last record consumed, or nothing at all
      3. Every time there is any mutation in collection Item, right after applying this mutation to the main collection (Item in this case) mongodb evaluates the list of all active subscriptions in Item, and if they match - appends a new record to corresponding destination capped collection.
      4.When any of the mongodb clients call getLastError after submitting a modification to Item collection the system responds only after all subscriptions have been evaluated and corresponding notification events inserted into subscription queues and made durable with the same arguments as the current getLastError call.
      • This is very important as it insures consistency of delivery of subscription events
      5. Subscription life cycle management
      • Create a subscription given a collection name and a filer; result - subscription ID
      • List subscription details given a subscription ID
      • Active/Deactivate subscription by ID
      • Delete subscription by ID

      This feature is basically very similar to how mongodb replication works.
      The key differences are:
      • subscription filter makes it possible to filer out only events of interest for a given consumer
      • Instead of using this technique for replication only, mongodb suddenly becomes a source of customizable business events that are placed in durable queues and managed by mongo itself

            Assignee:
            backlog-server-query Backlog - Query Team (Inactive)
            Reporter:
            yfinkelstein Yuri Finkelstein
            Votes:
            28 Vote for this issue
            Watchers:
            35 Start watching this issue

              Created:
              Updated:
              Resolved: