[SERVER-81260] Streams: Perf numbers for various workloads Created: 20/Sep/23  Updated: 19/Oct/23  Resolved: 19/Oct/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Aadesh Patel (Inactive) Assignee: Aadesh Patel (Inactive)
Resolution: Done Votes: 0
Labels: big, init-337-m3
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Problem/Incident
Related
related to SERVER-81719 Streams: Integrate mongostream into p... Closed
Assigned Teams:
Atlas Streams
Backwards Compatibility: Fully Compatible
Sprint: Sprint 32, Sprint 33
Participants:
Linked BF Score: 135

 Description   

https://mongodb.slack.com/archives/C04AH2TF7E1/p1695164357496619

Conversation above ^


aadesh  13 days ago

@ kenny.gorman re: throughput numbers for streams 
 __ 
aadesh  13 days ago

was chatting with Sandeep, so tomorrow im planning on getting a bunch of numbers together with various set ups. We have a few genny workloads setup for streams right now but those are using the in-memory source/sink operators so not super reflective of production setups. So plan is to run those same workloads against a kafka source in different regions with streams running in us-east-1 so that we have throughput numbers on a kafka source pipelines, and then send over a bunch of throughput numbers to you for each workload and source operator setuphows that generally sound? * in-memory source operator

  • same region kafka -> mstreams
  • different regions kafka -> mstreams

will run that set up for every workload we have in genny ^ along with the avg document size (in bytes) that we're using for those workloads
 __ 
aadesh  13 days ago

@ kenny.gorman
 __ 
aadesh  13 days ago

each workload will be a diff type of stream pipeline and document size
 __ 
Sandeep Dhoot  13 days ago

@aadesh we will want to change pipelines later and get these numbers a few times. So please do try to make the whole process repeatable.
 __ 
Joe Niemiec  13 days ago

I think it makes since it'll be good to have some baseline idea of the impact of a bandwidth delay product with cross region
 __ 
Joe Niemiec  13 days ago

I would make sure you really document how your Kafka setup is as well, Kafka tends to rely heavily on Linux page cache so there could be a difference between a Kafka which has buffered properly versus one that isn't because it's cold (edited) 
 __ 
Joe Niemiec  13 days ago

We also have some customers where reading change streams could potentially be cross region or  merging to a cluster cross region (edited) 
 __ 
kenny.gorman  13 days ago

Maybe I missed it but we need source and sink variations. Like Kafka to Kafka and Kafka to Mongo. To a lesser degree we need change stream source to Mongo.
 __ 
kenny.gorman  13 days ago

Yeah exact Kafka config is important. Repeatable is critical. Maybe something anyone can run not just engineering (thinking field) but maybe I am being too optimistic
 __ 
kenny.gorman  13 days ago

This is awesome guys. Can’t wait to see the results
 __ 
Sandeep Dhoot  13 days ago

Btw sources/sinks will introduce variability in results for N different reasons (source is in different region, unique kind of Kafka deployment). It would require too much effort to try to cover all the different scenarios. I hope we can just test with only a couple of difference scenarios and use those as ballpark numbers. (edited) 
 __ 
kenny.gorman  12 days ago

A couple different scenarios is what I meant yes, not all
 __ 
kenny.gorman  12 days ago

The main one is intra-region from kafka to mongodb, and intra region mongodb to mongodb. I am not sure (@ joe) if we have lots of Kafka to kafka use cases just yet.

Joe Niemiec  12 days ago

some rough telemetry based on data I have for combinations over 30 customers (a customer may do more then 1 pattern)CS 2 Kafka - 7
Kafka 2 Col - 12
Kafka 2 Kafka - 8
CS 2 Collection - 14
 __ 
Joe Niemiec  12 days ago

so really Kafka to Collection and CS to Collection are the top dogs
 __ 
aadesh  12 days ago

perf thats super helpful
 __ 
aadesh  11 days ago

re: repeatability, might take a bit more time on getting to system where we can easily repeat different setups for different stream pipelines

need to make various changes to existing perf tooling infra (DSI specifically) to distinguish mongod vs mstreams, but looking into that more today so that we can get to a place where its easy to do all this



 Comments   
Comment by Githook User [ 11/Oct/23 ]

Author:

{'name': 'Aadesh', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81260: [streams] additional workloads (#1038)
Branch: master
https://github.com/mongodb/genny/commit/952bea6f93b2713a925ef8688af07dd13f5fb696

Comment by Githook User [ 09/Oct/23 ]

Author:

{'name': 'Aadesh', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81260: support separate mongostream URI (#1026)
Branch: master
https://github.com/mongodb/genny/commit/abbea5d3970831db187f392bf08ce71efa2aeab5

Comment by Githook User [ 06/Oct/23 ]

Author:

{'name': 'Aadesh', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81260: support ^ClientURI in preprocessing (#1025)
Branch: master
https://github.com/mongodb/genny/commit/18bd36fcaa9019c9246c48a2eb21452580e3949f

Comment by Githook User [ 04/Oct/23 ]

Author:

{'name': 'Aadesh', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81260: custom throughput reporter for streams (#1020)
Branch: master
https://github.com/mongodb/genny/commit/9af1ccd6887c5320cd08801b6bc02a1b9a13af83

Comment by Githook User [ 28/Sep/23 ]

Author:

{'name': 'Aadesh Patel', 'email': 'aadesh.patel@mongodb.com', 'username': 'Aadeshp'}

Message: SERVER-81260: [sys-perf] use stream suffix for stream flavored mongod s3 tar file
Branch: master
https://github.com/mongodb/mongo/commit/988a39efdc8e11857e81c77fb64ea7d2bf23c2bd

Generated at Thu Feb 08 06:46:00 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.