[SERVER-24699] Update Evergreen config to publish repos in push tasks Created: 21/Jun/16  Updated: 23/Nov/16  Resolved: 02/Aug/16

Status: Closed
Project: Core Server
Component/s: Packaging
Affects Version/s: None
Fix Version/s: 3.0.13, 3.2.9, 3.3.11

Type: Task Priority: Major - P3
Reporter: Justin Cohler Assignee: Sam Kleinman (Inactive)
Resolution: Done Votes: 0
Labels: code-only
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Backwards Compatibility: Fully Compatible
Backport Completed:
Sprint: Build 16 (06/24/16), Build 17 (07/15/16), Build 18 (08/05/16)
Participants:

 Comments   
Comment by Githook User [ 02/Aug/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: bump curator pin and add support for signing with legacy key
Branch: v3.0
https://github.com/mongodb/mongo/commit/b1b948de7ebbd23b3191aa88b8bddb07712d7a2a

Comment by Githook User [ 02/Aug/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: remove seperate repo tarball from push artifacts (backport)
Branch: v3.0
https://github.com/mongodb/mongo/commit/d55a9e3b2af22b350d93c9ac05ac0d389fdb1614

Comment by Githook User [ 02/Aug/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: fix mkdir command in evergreen config causing push failures
Branch: master
https://github.com/mongodb/mongo/commit/554616d30eace3fe723ecd4ded6f8941945ab892

Comment by Sam Kleinman (Inactive) [ 01/Aug/16 ]

I've restarted the failed push tasks (community build) on this master commit: https://evergreen.mongodb.com/version/mongodb_mongo_master_27e090c6d8727145852bcecad1910ad9ec86159a

Interesting data point: for the enterprise nightlies which have been working since this commit, the push task now takes about about ~10 minutes to complete.

Comment by Sam Kleinman (Inactive) [ 01/Aug/16 ]

Here's a successful community push task on 3.2: https://evergreen.mongodb.com/task/mongodb_mongo_v3.2_debian81_push_c469a43c816043cb3620c02ecc126d90b0da4571_16_07_30_16_53_05

Comment by Githook User [ 01/Aug/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: bump pin and fix arch argument in curator invocation
Branch: v3.0
https://github.com/mongodb/mongo/commit/d7ef183cf9a5ba8a9e22e41a26d19d54568852ea

Comment by Sam Kleinman (Inactive) [ 01/Aug/16 ]

zakhar.kleyman and I have amended access for the community targeted builds, and I've rescheduled all the failed push tasks on this commit, and they appear to be running correctly now.

I've filed a bug for the underlying argument parsing issue that's affecting 3.0 pushes, have a fix pending for the 3.0 push.

Comment by Sam Kleinman (Inactive) [ 31/Jul/16 ]

Success on a previously failing 3.2 variant: https://evergreen.mongodb.com/task/mongodb_mongo_v3.2_enterprise_linux_64_amazon_ami_push_c469a43c816043cb3620c02ecc126d90b0da4571_16_07_30_16_53_05

Success on a selection of diverse master variants:

It looks like we're still failing on community repos (The access denied error, but that's tractable.)

Second, looking at 3.0, it seems like the "--arch" flag isn't getting populated by the evergreen expansion packager_arch, which doesn't exist in this version of the configuration) that exists in 3.2 master, and I expect, though I need to do some more testing, that this is impacting our argument parsing. We only publish packages for 64 bit repositories in 3.0, so we can (I think) just hard code x86_64 in the curator command line.

Comment by Ernie Hershey [ 30/Jul/16 ]

Some notes for 3.0 troubleshooting -
Failed tasks:
https://evergreen.mongodb.com/task/mongodb_mongo_v3.0_suse11_push_a219a818ef702696c4e92975e0ccf2c8b34e1035_16_07_30_11_08_16
https://evergreen.mongodb.com/task/mongodb_mongo_v3.0_enterprise_debian71_64_push_a219a818ef702696c4e92975e0ccf2c8b34e1035_16_07_30_11_08_16

Equivalent tasks on 3.2 which haven't run yet -
https://evergreen.mongodb.com/task/mongodb_mongo_v3.2_suse11_push_c469a43c816043cb3620c02ecc126d90b0da4571_16_07_30_16_53_05
https://evergreen.mongodb.com/task/mongodb_mongo_v3.2_enterprise_debian71_64_push_c469a43c816043cb3620c02ecc126d90b0da4571_16_07_30_16_53_05

Comment by Githook User [ 30/Jul/16 ]

Author:

{u'username': u'ehershey', u'name': u'Ernie Hershey', u'email': u'ernie.hershey@10gen.com'}

Message: SERVER-24699 Bump Curator version

(cherry picked from commit 27e090c6d8727145852bcecad1910ad9ec86159a)
Branch: v3.2
https://github.com/mongodb/mongo/commit/c469a43c816043cb3620c02ecc126d90b0da4571

Comment by Githook User [ 30/Jul/16 ]

Author:

{u'username': u'ehershey', u'name': u'Ernie Hershey', u'email': u'ernie.hershey@10gen.com'}

Message: SERVER-24699 Bump Curator version
Branch: master
https://github.com/mongodb/mongo/commit/27e090c6d8727145852bcecad1910ad9ec86159a

Comment by Githook User [ 30/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: bump curator version and fix shell invocation

(cherry picked from commit ba4b3ae46e6d96bf52fc3b7f38845dd107fc5173)
Branch: v3.2
https://github.com/mongodb/mongo/commit/c85976c222764fbf9495349671eb74045a862d76

Comment by Githook User [ 30/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: bump curator version and fix shell invocation
Branch: v3.0
https://github.com/mongodb/mongo/commit/a219a818ef702696c4e92975e0ccf2c8b34e1035

Comment by Githook User [ 30/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: bump curator version and fix shell invocation
Branch: master
https://github.com/mongodb/mongo/commit/ba4b3ae46e6d96bf52fc3b7f38845dd107fc5173

Comment by Githook User [ 27/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: include repo config in artifacts archive
Branch: master
https://github.com/mongodb/mongo/commit/5c81ec7f025a610287f6020b56d9ad33505f5b1a

Comment by Githook User [ 27/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: include repo config in artifacts archive
Branch: v3.2
https://github.com/mongodb/mongo/commit/edbe839bf3123c89dee7acd9a698033dce7ab655

Comment by Githook User [ 27/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: include repo config in artifacts archive
Branch: v3.0
https://github.com/mongodb/mongo/commit/c90f272025102a3ba48d379a46bf0f31d5144187

Comment by Sam Kleinman (Inactive) [ 27/Jul/16 ]

https://mongodbcr.appspot.com/86250001/

Comment by Githook User [ 27/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: fix push task shell syntax
Branch: v3.0
https://github.com/mongodb/mongo/commit/040839ca15d514d42452150cf028cd1f02ee2f4e

Comment by Githook User [ 26/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: make all repository publishing synchronous
Branch: v3.0
https://github.com/mongodb/mongo/commit/f42ad98cb9a3c8d5ef13dafa40bb7333deaab0b5

Comment by Githook User [ 26/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: fix signing key for push task
Branch: v3.2
https://github.com/mongodb/mongo/commit/945cb73a7a443f3b282a072449adc81072c3eced

Comment by Githook User [ 26/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: fix signing key for push task
Branch: master
https://github.com/mongodb/mongo/commit/a724ac885e2b7d8c413d8ba7b2f50a2c04f7f666

Comment by Githook User [ 26/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: make all repository publishing synchronous

(cherry picked from commit 688ab8a44a8c89b1b68bfe7c4968b3a8d7e43f5a)
Branch: v3.2
https://github.com/mongodb/mongo/commit/40a0f5c8f6ccfd2b4dab84ade6077ecdb6071847

Comment by Githook User [ 26/Jul/16 ]

Author:

{u'username': u'tychoish', u'name': u'Sam Kleinman', u'email': u'samk@10gen.com'}

Message: SERVER-24699: make all repository publishing synchronous
Branch: master
https://github.com/mongodb/mongo/commit/688ab8a44a8c89b1b68bfe7c4968b3a8d7e43f5a

Comment by Ernie Hershey [ 15/Jul/16 ]

I'm testing this with package tarballs from this build -
https://evergreen.mongodb.com/version/mongodb_mongo_master_e5e2ecb49da979c5cc3534ae3cb76212a72e3986

RHEL 7 Enterprise:
https://s3.amazonaws.com/mciuploads/mongodb-mongo-master/enterprise-rhel-70-64-bit/e5e2ecb49da979c5cc3534ae3cb76212a72e3986/artifacts/mongodb_mongo_master_enterprise_rhel_70_64_bit_e5e2ecb49da979c5cc3534ae3cb76212a72e3986_16_07_15_12_58_40.tgz
https://evergreen.mongodb.com/task/mongodb_mongo_master_enterprise_rhel_70_64_bit_compile_e5e2ecb49da979c5cc3534ae3cb76212a72e3986_16_07_15_12_58_40

Ubuntu 16.04 Community:
https://s3.amazonaws.com/mciuploads/mongodb-mongo-master/ubuntu1604/e5e2ecb49da979c5cc3534ae3cb76212a72e3986/artifacts/mongodb_mongo_master_ubuntu1604_e5e2ecb49da979c5cc3534ae3cb76212a72e3986_16_07_15_12_58_40.tgz
https://evergreen.mongodb.com/task/mongodb_mongo_master_ubuntu1604_compile_e5e2ecb49da979c5cc3534ae3cb76212a72e3986_16_07_15_12_58_40

Generated at Thu Feb 08 04:07:09 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.