-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Atlas Streams
-
Fully Compatible
-
None
-
None
-
None
-
None
-
None
-
None
-
None
See the scope for expected user behavior
What
In this ticket, we'd like to allow users to choose the outputFormat that their documents will be written to S3 in.
Users can choose between relaxedJson, canonicalJson, and basicJson.
Relaxed and Canonical formats are existing Extended JSON v2 formats described on in the documentation here
basicJson will be a streams-only serialization format that we will be creating in this PR. The format is described here on the PD
Suggested Implementation
1. Modify IDL file to add outputFormat option to S3Config
2. Create a recursive function that accepts a mongo::Document and spits out a string
3. Modify s3_emit_operator.cpp to choose between the 3 output formats
Note: we should research how to write JSON "fast". We could try to see what the CXX driver does