Summary
Support the new Bulk Command use at the Database and Collection levels and direct driver developers on how to implement.
Motivation
The new bulk write command allows for grouped operations to be sent to the server.
However, the new bulk API specification eludes to this being implemented at the MongoClient level only:
This API is distinct from the collection-level bulkWrite method defined in the CRUD specification and the deprecated bulk write specification.
I think the new command is a great step forward but only implementing it at the Client level, while easy for us driver developers doesn't provide the best experience for our users. However, the new specification should be flexible and ensure naming of options doesn't preclude the use of this API at the Database and Collection levels. Eg:
If naming conflicts arise, drivers SHOULD prepend "Client" to the new type names (e.g. ClientBulkWriteOptions).
The rationale for this is:
- Discoverability - All documentation / courses / stackoverflow posts etc of existing use of bulk operations are based at the Collection level. There are no write based operations at the Client level.
- Covention - The Changes Stream specification was the first to introduce a cross Client, Database, Collection API. The new bulk API command could easily follow this convention. Building bulk operations at the database and collection level would be easy with helpers that can feed into the new underlying operation.
- Migration - All current users of bulk operations will have to fully rewrite their logic to ensure a Client is available at the same place where they currently have a Collection instance. Depending on which driver this could be a large impediment. For example, the Java driver has no way to get a Client from a Collection.
- Confusion - Two separate bulk APIs with different performance is surprising and will cause confusion to our users. If a new driver was implemented would it be expected to implement both? Is the collection API actually deprecated?
- Delight - there is the opportunity to delight our users by using the new command if supported. This would improve performance by reducing batching and should come for free. The requirement would be to add translation of the bulk write result to the exisitng type.
I feel if we don't address this now, feedback from customers and users of the existing API will quickly request it. So to head off this possibility and ensure that design decisions aren't set in stone causing further migration issues, a plan should be considered for the eventual addition of Database and Collection bulk operation builders.
- is related to
-
DRIVERS-716 Improved Bulk Write API
- In Progress
-
DRIVERS-2862 Benchmark Collection and Client bulkWrite
- In Review