Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-5180

Exclude .git from the evergreen artifact tar ball

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 4.3 Required
    • Component/s: None
    • Labels:
      None

      Description

      The .git folder is occasionally causing permission issue when extracting the tar ball and unexpectedly failing subsequent tasks.

      Here's one example in the patch build log:

      [2019/10/10 21:19:14.637] Task logger initialized (agent revision: 49e6f044e070bba4ae4fdb05d640f8c796a8b9f0).
      [2019/10/10 21:19:14.670] Starting task wiredtiger_ubuntu1804_unit_test_bucket07_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58, execution 0.
      [2019/10/10 21:19:14.772] Running pre-task commands.
      [2019/10/10 21:19:14.787] Running command 'shell.exec' in "cleanup" (step 1 of 1)
      [2019/10/10 21:19:14.790] Finished running pre-task commands.
      [2019/10/10 21:19:14.790] Running command 's3.get' in "fetch artifacts" (step 1 of 2)
      [2019/10/10 21:19:14.806] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 1 of 10)
      [2019/10/10 21:24:14.806] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 2 of 10)
      [2019/10/10 21:24:16.842] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 3 of 10)
      [2019/10/10 21:24:19.536] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 4 of 10)
      [2019/10/10 21:24:21.947] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 5 of 10)
      [2019/10/10 21:24:32.125] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 6 of 10)
      [2019/10/10 21:24:41.926] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 7 of 10)
      [2019/10/10 21:24:56.161] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 8 of 10)
      [2019/10/10 21:25:16.206] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 9 of 10)
      [2019/10/10 21:25:36.229] fetching wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket build_external (attempt 10 of 10)
      [2019/10/10 21:25:56.265] Command failed: S3 get failed after 10 attempts
      

      Here's the detailed log in the agent log:

      [2019/10/10 21:19:14.790] Set command timeout for ''s3.get' in "fetch artifacts" (#1)' () to 2h0m0s
       [2019/10/10 21:24:14.806] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: net/http: request canceled (Client.Timeout exceeded while reading body)]
       [2019/10/10 21:24:16.842] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:24:19.536] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:24:21.947] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:24:32.125] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:24:41.926] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:24:56.161] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:25:16.206] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:25:36.229] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:25:56.265] problem getting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from s3 bucket, retrying. [problem extracting wiredtiger/ubuntu1804/8d9f164f5cd5f69b48f786b91d5451823ae97dae/artifacts/compile_wiredtiger_ubuntu1804_patch_8d9f164f5cd5f69b48f786b91d5451823ae97dae_5d9fd2c09ccd4e6068b5ae4b_19_10_11_00_54_58.tgz from archive: error extracting /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger: open /data/mci/a8fdeb39f8c4fd0f291cd230e61cfa52/wiredtiger/.git/objects/1f/423e90cf0d6a88be5f2818f8df88408698d35b: permission denied]
       [2019/10/10 21:25:56.265] Finished running task commands in 6m41.475088272s.
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                backlog-server-storage-engines Backlog - Storage Engines Team
                Reporter:
                chenhao.qu Chenhao Qu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: