[SERVER-13932] Change Notification Stream API Created: 13/May/14 Updated: 29/Jan/18 Resolved: 07/Dec/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.0 |
| Type: | New Feature | Priority: | Major - P3 |
| Reporter: | Osmar Olivo | Assignee: | Alyson Cabral (Inactive) |
| Resolution: | Done | Votes: | 30 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||||||||||||||
| Description |
|
Currently the oplog can be used as a means to track changes on the system. You may want to do this for things such as detecting changes to determine cache invalidations. Naturally, tailing the oplog is not an ideal medium for this as it's format can change at any time and MongoDB provides no guarantees when basing things off the oplog. Furthermore, the oplog stores all events that occur on the system including internal events like removes and inserts from migrations that I may not want or may want to filter out. Finally, the oplog is a mongod-wide component which under heavy traffic means I may be tailing much more data than I actually need if my application is focused on just one collection or DB. Ideally, creating an activity stream that is optionally published out would expose this through an API layer independent of the oplog. This would allow for applications like this to no longer directly rely on the oplog. If we could also filter on the activity stream similar to how audit filters are applied it would allow for only reading the changes you care about as oppose to filtering on the application side and transmitting a ton of data needlessly. You could also do things like only allowing people with certain permissions to receive certain changes. I.E. if I don't have permission to a DB, I can't view the changes that have occurred on it. Currently if you want to tail some subset of the oplog, you have to be able to tail the whole oplog which causes a bit of a security permissions problem. |
| Comments |
| Comment by Alyson Cabral (Inactive) [ 06/Dec/17 ] |
|
I'm happy to announce that MongoDB 3.6 launched with Change Streams. As a new feature in MongoDB 3.6, change streams enable applications to stream real-time data changes by leveraging MongoDB’s underlying replication capabilities. The characteristics of change streams include: 1. Resumable For more detailed information check out the following resources: Aly Cabral |
| Comment by Shane Tolmie [X] [ 29/Jul/14 ] |
|
I am interested in this feature, as it will allow any service to receive notifications when a portion of the database is edited by a 3rd party (even a human being typing on the console). This sort of behavior is very useful in a Service Oriented Architecture. SQL Server has already solved this problem by providing SQL Service Broker. We use this feature heavily, so a service can get notified when anything at all edits a central database. The service has to react to edits can be generated by anything, e.g. a human being typing manually on the MongoDB console or someone manually editing the database using RoboMongo. |