[SERVER-68365] Add C++ implementation of gRPC to `mongo/third_party` Created: 27/Jul/22  Updated: 26/Mar/23  Resolved: 15/Mar/23

Status: Closed
Project: Core Server
Component/s: Build
Affects Version/s: None
Fix Version/s: None

Type: Task Priority: Major - P3
Reporter: Amirsaman Memaripour Assignee: Daniel Moody
Resolution: Won't Do Votes: 0
Labels: dbdpfy23q3, third-party-integration
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-74624 Make sure commit-queue build times re... Closed
is depended on by SERVER-74012 Generate code for MongoDB gRPC service Closed
Gantt Dependency
has to be done after SERVER-73788 Switch SCons cache dir to per variant Closed
Problem/Incident
Participants:
Linked BF Score: 58

 Description   

Integrate the latest stable version of gRPC's C++ implementation with the core-server's code-base.



 Comments   
Comment by Daniel Moody [ 15/Mar/23 ]

Closing in favor of https://jira.mongodb.org/browse/SERVER-73467

Comment by Daniel Moody [ 08/Feb/23 ]

I had talked with brian.mccarthy@mongodb.com about creating multiple separate EFS for the more common builders (commit queue and required compile builders). We would need to make sure if we change our required builders around that we correctly maintain the EFS's in use.

I'm not sure about SCons cache support for multiple cache directories in the same instance of a build. The cache dir is maintained in the environment and SCons allows multiple environments so I feel like its possible but I have never tried it before.

Comment by Alex Neben [ 08/Feb/23 ]

I am not a huge fan of that solution - it feels like a gross bandaid that is kicking the can down the road for a serious issue. Can you think of anything else? Can we somehow split up the cache (look for third party stuff is location x, look for everything else in location y)?

Comment by Daniel Moody [ 07/Feb/23 ]

Another workaround for the time being is we can mark the grpc stuff as NoCache and then exclude the grpc unittest from any of the AIB labels so it is not autoinstalled (and therefore not run by any compile_all tasks which build all the install- targets). Then it would not be built in evergreen unless we setup a special task for it.

Comment by Daniel Moody [ 07/Feb/23 ]

I don't think it's a file limit, but number of concurrent file operations. A write is 5 times more expensive than a read per amazon, and we added 1200 new files x ~100 compile tasks (including nightly tasks) so 120000 operations on top of the existing workload. 

 

I think we are near the limit with currently, although some of the graphs that I saw didn't seem to indicate that, and they quotas from amazon indicate quite a large number of operations: https://docs.aws.amazon.com/efs/latest/ug/limits.html#limits-client-specific

 

I wish there was more conclusive explanation from amazon.

Comment by Alex Neben [ 06/Feb/23 ]

This seems pretty serious then - if it is not due to grpc then we are bound to hit this limit soon. Is there a file limit or something like that in the scons cache?

Comment by Daniel Moody [ 03/Feb/23 ]

some data regarding local scons cache and the reverted changes:
I compared two cache dirs created from +grpc_lib_test target (builds all the grpc stuff and runs a helloworld unittest) on a local build (no icecream) and the two cache dirs show no difference, showing that the changes are generating reproduceable cache entries.

 

Then I tested some metrics on a local cache dir of the merged change and the previous commit.

local test of a scons_cache dir:
num files:
(python3-venv) Feb.03 01:50 ubuntu[mongo]: find ../protobuf_cache/with -type f | wc -l
17965
(python3-venv) Feb.03 01:50 ubuntu[mongo]: find ../protobuf_cache/without/ -type f | wc -l
15719size:
(python3-venv) Feb.03 01:52 ubuntu[mongo]: du -shm ../protobuf_cache/with
56605	../protobuf_cache/with
(python3-venv) Feb.03 01:52 ubuntu[mongo]: du -shm ../protobuf_cache/without/
55670	../protobuf_cache/without/ 

Comment by Daniel Moody [ 02/Feb/23 ]

alternatively the changes do support using system libraries so if we can update the images to install the correct versions of protobuf and grpc to the system, we can in the meantime avoid building/caching grpc ourselves.

Comment by Daniel Moody [ 02/Feb/23 ]

This was reverted because it was suspected to be overloading the EFS cache system in evergreen. This may be blocked on improving or increasing the capacity of EFS in evergreen.

Comment by Githook User [ 02/Feb/23 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: Revert "SERVER-68365 integrate third_party grpc 1.39.1 into the build"

This reverts commit fbea56b2b78c7961a95a8b3bfd2725bbac6cb5f6.
Branch: master
https://github.com/mongodb/mongo/commit/bf15a84f7979cbf5ad69621b2ce6fa0732caeaad

Comment by Githook User [ 30/Jan/23 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 integrate third_party grpc 1.39.1 into the build
Branch: master
https://github.com/mongodb/mongo/commit/fbea56b2b78c7961a95a8b3bfd2725bbac6cb5f6

Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 finishing up integration
Branch: SERVER-68365_testing4
https://github.com/mongodb/mongo/commit/a920ce544c6c343abaed1c1b7eefd3baa3f9dd0d

Comment by Githook User [ 03/Oct/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 finishing up integration
Branch: SERVER-68365_testing3
https://github.com/mongodb/mongo/commit/6239752e2ce22c84c27ca5ebb47a1d7ec147b7e0

Comment by Githook User [ 27/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 a few small fixes
Branch: SERVER-68365_collecting_configs
https://github.com/mongodb/mongo/commit/690f68168e253bad8a060fcf5d25d4d0afdc76c0

Comment by Githook User [ 27/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 added third party source
Branch: SERVER-68365_collecting_configs
https://github.com/mongodb/mongo/commit/a2b185a76513420a07b83867b9050da1d9d95850

Comment by Githook User [ 27/Sep/22 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-68365 added grpc third party integration (excluding third party source)
Branch: SERVER-68365_collecting_configs
https://github.com/mongodb/mongo/commit/be60e8d6c6ad9ac938583db103f49706ab8faa27

Generated at Thu Feb 08 06:10:36 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.