[SERVER-24315] Enable rsync on Linux Repos Created: 31/May/16 Updated: 06/Mar/19 Resolved: 06/Mar/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Packaging |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | New Feature | Priority: | Minor - P4 |
| Reporter: | stephan.schultchen | Assignee: | Matt Lord (Inactive) |
| Resolution: | Done | Votes: | 3 |
| Labels: | repositories | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Participants: |
| Description |
|
It would be great, if you could enable rsync as a protocol for mongodb yum/deb repositories in order to easily mirror them locally. This is needed quite frequently due to company policies that keep datacenter machines isolated from the internet. |
| Comments |
| Comment by James Glenn [ 06/Mar/19 ] | ||
|
Thanks! | ||
| Comment by Matt Lord (Inactive) [ 06/Mar/19 ] | ||
|
I'm going to close this as done (let me know if you disagree). We can't enable the rsync protocol because the repos live in S3 and it's not an option there, but you can achieve the underlying goal of mirroring our repos locally this way:
The awscli is available for all platforms, is free and open source software, and when using the --no-sign-request option no AWS credentials are needed. | ||
| Comment by Matt Lord (Inactive) [ 01/Mar/19 ] | ||
|
mcdaniel, you can solve the underlying pain and sync/mirror the entire repo this way (with --no-sign-request, no AWS credentials are required):
You can also just sync various sub-sections if you prefer, e.g.:
The awscli is available in many ways. And you can find the source on GitHub. Is this a workable solution for you? | ||
| Comment by Alex McDaniel [ 25/Feb/19 ] | ||
|
Thank you Matt! Would it be possible to get a time estimate? | ||
| Comment by Matt Lord (Inactive) [ 19/Feb/19 ] | ||
|
Thank you again for the helpful feedback, mcdaniel! I'll get this on our backlog and discuss with the team. | ||
| Comment by Alex McDaniel [ 19/Feb/19 ] | ||
|
Thanks for the quick response! Exactly, without being updated when a new release came out it would be difficult to keep the repo syncs up to date, that is if I were to reposync each cross section individually. Being able to mirror https://repo.mongodb.org in its entirety would be ideal. Please let me know if you find any options! | ||
| Comment by Matt Lord (Inactive) [ 15/Feb/19 ] | ||
|
Thank you, mcdaniel! That's a good point. There's a difference between sync'ing a repo – e.g. RHEL7-x86_64-mongodb-4.0 – and mirroring the full suite of repos available on a host. What you really want to do is mirror https://repo.mongodb.org in its entirety? I want to make sure I understand the specific want/need so that I can look at existing options/work-arounds and look into future work. | ||
| Comment by Alex McDaniel [ 15/Feb/19 ] | ||
|
With reposync you cant sync across different versions without manually adding them to the config files. That is I believe you can only sync at specific cross sections. I hope to sync new versions if they are introduced without having to manually check if there is a new version and adding it to the config. Any ideas? | ||
| Comment by Matt Lord (Inactive) [ 31/Jan/19 ] | ||
|
The general need for local mirrored repos is quite common in large orgs as the machines in the DC don't have internet access. Can anyone help me understand why rsync is desired here over debmirror and reposync, which both support using HTTP? These tools also have a big advantage in that you don't have to mirror every bit in the repo as usually an org is only interested in a small subset of the packages at the cross section between: MongoDB versions, OS variants, and hardware platforms. Can you help me understand the key reasons you want rsync protocol support? Thanks! | ||
| Comment by Alex McDaniel [ 28/Jan/19 ] | ||
|
Also in need of this. Any updates? | ||
| Comment by James Glenn [ 03/Nov/17 ] | ||
|
Also need this. | ||
| Comment by stephan.schultchen [ 07/Jun/16 ] | ||
|
our internal servers have no access to the internet, therefore we run mirrors of needed repositories. rsync is the tool/protocol of choice for syncing those mirrors. syncing mirrors via http/https is a strong pain point. | ||
| Comment by Sam Kleinman (Inactive) [ 07/Jun/16 ] | ||
|
Thanks for your suggestion. We're very committed to providing unfettered access to these repositories via HTTP and HTTPS, and don't have plans- Could you please provide more information about your use case for this kind of access to help us understand and prioritize this feature. Regards, | ||
| Comment by stephan.schultchen [ 31/May/16 ] | ||
|
a ftp server would also be nice, if possible. |