Uploaded image for project: 'Evergreen'
  1. Evergreen
  2. EVG-14648

Show s3 copied/uploaded path

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major - P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: current_quarter
    • Component/s: app
    • Labels:
      None

      Description

      In our evergreen config for mongo-ruby-toolchain, we have s3 copy tasks:

        - name: push
          patchable: false
          depends_on:
            - name: build
          commands:
            - command: s3Copy.copy
              params:
                aws_key: ${aws_key}
                aws_secret: ${aws_secret}
                s3_copy_files:
                  - source:
                      path: mongo-ruby-toolchain/${build_variant}/${revision}/${build_id}.tar.gz
                      bucket: mciuploads
                    destination:
                      path: build/toolchain-drivers/mongo-ruby-driver/ruby-toolchain-${build_variant}-${revision}.tar.gz
                      bucket: boxes.10gen.com
            
            - command: s3Copy.copy
              params:
                aws_key: ${aws_key}
                aws_secret: ${aws_secret}
                s3_copy_files:
                  - source:
                      path: mongo-ruby-toolchain/${build_variant}/${revision}/${build_id}.tar.xz
                      bucket: mciuploads
                    destination:
                      path: build/toolchain-drivers/mongo-ruby-driver/ruby-toolchain-${build_variant}-${revision}.tar.xz
                      bucket: boxes.10gen.com
      

      When these are run, evergreen produces the following output:

       
      [2021/05/20 12:51:12.323] Running command 's3Copy.copy' (step 1 of 2)
      [2021/05/20 12:51:12.711] destination bucket names containing dots that are created after Sept. 30, 2020 are not guaranteed to have valid attached URLs
      [2021/05/20 12:51:12.711] Finished 's3Copy.copy' in 387.579235ms
      [2021/05/20 12:51:12.711] Running command 's3Copy.copy' (step 2 of 2)
      [2021/05/20 12:51:13.180] destination bucket names containing dots that are created after Sept. 30, 2020 are not guaranteed to have valid attached URLs
      [2021/05/20 12:51:13.180] Finished 's3Copy.copy' in 469.352543ms
      

      The actual paths that are being copied/uploaded are not shown. Since these paths are generated by evergreen, a user doesn't know where the files have gone to.

      Can all tasks that write to s3 and utilize expansions please output the destination path that is being written.

        Attachments

          Activity

            People

            Assignee:
            backlog-server-evg Backlog - Evergreen Team
            Reporter:
            oleg.pudeyev Oleg Pudeyev
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated: