|
shreyas.kalyan, I would say at a high level, this ticket is made of the following pieces:
- Introducing a new command line option --spawnUsing=python|jasper with --spawnUsing=python being the default and identical to the existing behavior.
- Creating a version of the resmokelib.core.Process class that sends an HTTP request to the jasper service rather than using the subprocess package, but otherwise has an identical API. There's no API documentation for jasper but the appropriate endpoints to use for spawning a process, sending it a signal, and waiting for it to exit should hopefully be self-evident from their names in this list. It should be fine for now to have their logs go to separate files in Fixture.get_dbpath_prefix() rather than being logged to the console.
- When --spawnUsing=jasper, resmoke.py should download a pinned version of the curator binary. You can use https://s3.amazonaws.com/boxes.10gen.com/build/curator/curator-dist-<variant>-<githash>.tar.gz as the URL, where variant could be "ubuntu1604", "macos", or "windows-64" and should be determined at runtime via sys.platform. Take a look at https://evergreen.mongodb.com/waterfall/curator?task_filter=push to see the other build variants if you'd like, but I'd say it isn't worth supporting the ability to download zSeries, ARM, or PowerPC binaries because we'll never run those locally for genny. The binary should be downloaded into a temporary directory and extracted into the build/ directory like with do with clang-format-3.8. It should be checked for at the beginning to avoid downloading the curator binary if it is already present and of the same version.
- When --spawnUsing=jasper, resmoke.py should run curator jasper rest --port=<basePort - 1 = 19999 by default> at the beginning to start the service and terminate the process before exiting. This means spawning the curator process should be the only usage of the subprocess package (really the subprocess32 package) when --spawnUsing=jasper. The reasoning behind having the usage of jasper be aware of resmoke.py base port is to ensure it is possible to run resmoke.py with --spawnUsing=jasper on a shared Linux workstation. (Longer term we would want there to be a single instance of jasper running on the shared Linux workstation but that would mean resmoke.py could run processes as a different user than the resmoke.py Python process, which we haven't instruction Server engineers how to protect themselves against yet.)
- Every usage of the resmokelib.core.Process class should be replaced by a means of using resmokelib.core.Process when --spawnUsing=python and using the new class when --spawnUsing=jasper. One way to achieve this would be to add a make_process() function that constructs an instance of the appropriate class after consulting a variable defined in the config.py module.
|