[SERVER-56624] Surface SCons cache hit statistics to Splunk Created: 04/Nov/19  Updated: 29/Oct/23  Resolved: 04/May/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 4.9.0

Type: New Feature Priority: Major - P3
Reporter: Andrew Morrow (Inactive) Assignee: Daniel Moody
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Backwards Compatibility: Fully Compatible
Sprint: Dev Platform 2021-05-17
Participants:
Linked BF Score: 32
Story Points: 1

 Description   

We create a file on every server compile task that contains cache hit data for the SCons cache.

It would be nice to capture that data to somewhere we could use to monitor the effectiveness of the cache(s).

You can see an example file here: https://s3.amazonaws.com/mciuploads/mongodb-mongo-master/enterprise-rhel-62-64-bit/80e0497ebaaaf87707239f661cac2ab99ca9a88c/scons-cache-mongodb_mongo_master_enterprise_rhel_62_64_bit_80e0497ebaaaf87707239f661cac2ab99ca9a88c_19_11_04_17_32_44-0.log



 Comments   
Comment by Githook User [ 04/May/21 ]

Author:

{'name': 'Daniel Moody', 'email': 'daniel.moody@mongodb.com', 'username': 'dmoody256'}

Message: SERVER-56624 added json cachedir logs for splunk
Branch: master
https://github.com/mongodb/mongo/commit/43c6b0bda4f2dce486344a03d03c697178e765b3

Comment by Zakhar Kleyman [ 28/Apr/21 ]

I've added scons_splunk_server and scons_splunk_token variables to mongodb-mongo-master evergreen project.

Comment by Andrew Morrow (Inactive) [ 21/Apr/21 ]

Re-opening this as it may be important again per SERVER-54732

Comment by April Schoffer [ 25/Jan/21 ]

We're closing this to focus on other higher priorities, we can revisit if it becomes more pressing in the future.

Comment by Zakhar Kleyman [ 10/Mar/20 ]

I've got the creds, I can add them to the project and you can update the task to use `cutator splunk` command

Comment by Zakhar Kleyman [ 27/Jan/20 ]

I've opened TECHOPS-7268 requesting a new splunk index and creds for this.

Comment by Sam Kleinman (Inactive) [ 08/Nov/19 ]

You'll need to get a splunk token from TECHOPS

Alternatively you can use configurable logging (with the same backend/caveat) to just put the output of a command to splunk directly in evergreen that's this:

https://github.com/evergreen-ci/evergreen/wiki/Project-Files#customizing-logging

Comment by Sam Kleinman (Inactive) [ 08/Nov/19 ]

I would attempt to convert these files in some way to JSON or at least remove the line break between the two lines (it seems that each message is really made up of two lines.)

then to upload it to splunk, use curator:

GRIP_SPLUNK_SERVER_URL=""
GRIP_SPLUNK_CLIENT_TOKEN=""
 
cutator splunk <--json> --annoatate=project:<projectId> --annotate=<key:value> command --exec="cat <file>"

Generated at Thu Feb 08 05:39:46 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.