[KAFKA-288] Using brew installs on mac doesn't seem to allow mongo connector installation Created: 02/Feb/22 Updated: 27/Oct/23 Resolved: 10/Feb/22 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Task | Priority: | Unknown |
| Reporter: | Alexander Boese | Assignee: | Robert Walters |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | external-user | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Description |
|
I realize there's about 50 things I could do wrong to get here. Adding a --component-dir parameter gets further but still doesn't run, and a worker-configs option I don't believe does anything particularly useful, so I think that's a general error that hasn't been paired off to be very specific in this instance.
I realize this is not for production, but was just trying a basic setup on Mac for testing. I have downloaded the component, but I'm missing a good walk through. There are some directories that need to be set up in profile according to some postings, but that has not seemed to help either. Is there a good page I can go to that would be a suitable walk through for this? And to be very clear, the connector itself needs to be downloaded unzipped and placed in the right place. The primary necessary components as far as I can tell have been downloaded and installed with brew.
|
| Comments |
| Comment by Alexander Boese [ 10/Feb/22 ] |
|
It looks like some of the connector name have maybe changed, and that is leading to the mix up. That's one thing....but I'll figure out the others. Thank you. Mr. Alex Boese EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender. https://jira.mongodb.org/s/en_USp8swtz-1988229788/6109/25/_/jira-logo-scaled.png Re: Using brew installs on mac doesn't seem to allow mongo connector installation<https://jira.mongodb.org/browse/KAFKA-288> then just copy the JAR file of whatever connector you want to install into the plug in path of your connector. It most likely is [Add Comment] <https://jira.mongodb.org/browse/KAFKA-288#add-comment> Add Comment<https://jira.mongodb.org/browse/KAFKA-288#add-comment> This message was sent from MongoDB's issue tracking system. To respond to this ticket, please login to jira.mongodb.org<https://jira.mongodb.org> using your JIRA, MongoDB Cloud Manager, or MongoDB Atlas credentials. ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. |
| Comment by Robert Walters [ 10/Feb/22 ] |
|
then just copy the JAR file of whatever connector you want to install into the plug in path of your connector. It most likely is |
| Comment by Alexander Boese [ 10/Feb/22 ] |
|
I mean most of what I've gotten working has actually been manual on a stand alone instance. Just wanting to continue with that for mongo connector as possible. Mr. Alex Boese EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender. https://jira.mongodb.org/s/en_USp8swtz-1988229788/6109/25/_/jira-logo-scaled.png Re: Using brew installs on mac doesn't seem to allow mongo connector installation<https://jira.mongodb.org/browse/KAFKA-288> I think there is some confusion with regard to what it means to install a connector on Kafka Connect. What I am suggesting is use docker to launch Kafka Connect and the rest of the kafka services. This just helps with cleanup as nothing is left installed on your mac. within Kafka Connect you can install as many connectors as you'd like MongoDB, MySQL and whatever other connectors you'd like to use. It doesn't have anything to do with Docker at that point they are just JARs added to a folder on that Kafka Connect instance. if it is easier I'd be happy to jump on a zoom and go over this, just lmk and Ill email you directly. [Add Comment] <https://jira.mongodb.org/browse/KAFKA-288#add-comment> Add Comment<https://jira.mongodb.org/browse/KAFKA-288#add-comment> This message was sent from MongoDB's issue tracking system. To respond to this ticket, please login to jira.mongodb.org<https://jira.mongodb.org> using your JIRA, MongoDB Cloud Manager, or MongoDB Atlas credentials. ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. |
| Comment by Alexander Boese [ 10/Feb/22 ] |
|
Asking for those other things might be a stretch...okay I get that. Is there at least a known working properties file for mongo connector that you can point to? I see a slew of different examples and none of them as far as I can tell are working for me. Maybe tell me which is good for whatever is recent, and I can pick at the problems I am having locally loading that? Mr. Alex Boese EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender. https://jira.mongodb.org/s/en_USp8swtz-1988229788/6109/25/_/jira-logo-scaled.png Re: Using brew installs on mac doesn't seem to allow mongo connector installation<https://jira.mongodb.org/browse/KAFKA-288> |
| Comment by Robert Walters [ 10/Feb/22 ] |
|
I think there is some confusion with regard to what it means to install a connector on Kafka Connect. What I am suggesting is use docker to launch Kafka Connect and the rest of the kafka services. This just helps with cleanup as nothing is left installed on your mac. within Kafka Connect you can install as many connectors as you'd like MongoDB, MySQL and whatever other connectors you'd like to use. It doesn't have anything to do with Docker at that point they are just JARs added to a folder on that Kafka Connect instance. if it is easier I'd be happy to jump on a zoom and go over this, just lmk and Ill email you directly. |
| Comment by Alexander Boese [ 10/Feb/22 ] |
|
But I am trying to install three different connectors. You're suggesting I install the docker image with the mongo connector that works....okay great. But I also need the other connectors working. So if the others have similar issues, do I need to have a docker instance per connector? I mean, I'm not faulting the docker approach, but if you're really going to have a docker image for connectors, why not make one image with all the connectors? Then I only have to turn things off, which relatively speaking is trivial. Or maybe one with the most popular ones? Or, if you really have this stuff figured out, a way to select the connectors I need to supply me with a conforming image. I'd honestly much rather have the documentation. If the only way to get that is by going through the code, then great. But I see no documentation atm and no recommendations on how to best troubleshoot these besides "use the docker instance" which again forces me to cobble things, so still not ideal. Thoughts? Mr. Alex Boese EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender. https://jira.mongodb.org/s/en_USp8swtz-1988229788/6109/25/_/jira-logo-scaled.png Re: Using brew installs on mac doesn't seem to allow mongo connector installation<https://jira.mongodb.org/browse/KAFKA-288> |
| Comment by Robert Walters [ 10/Feb/22 ] |
|
Once your kafka connect service is running you can install as many connectors as you'd like on that kafka connect instance. You do not have to stand up multiple kafka connect containers. Just one container that is running kafka connect, that's it. I find Docker to be the best way to quickly stand up a Kafka environment. It leaves nothing behind on my mac when I'm done. There might be other options but it isn't anything I've needed to explore. If you are not that familiar with Docker and Kafka, it might be easiest to install the Confluent Platform. https://docs.confluent.io/platform/current/quickstart/ce-docker-quickstart.html Run through the Docker install on this quickstart and you'll a web UI, the Control Center to use to add connectors as well.
|
| Comment by Alexander Boese [ 10/Feb/22 ] |
|
No I'm not looking for a quick test. I'm doing some investigational testing, and I don't need or want to spin up a new docker for every single connector I plan on using. Isn't there a better way to access default documentation than through the docker instance? Mr. Alex Boese EXTERNAL EMAIL - This email originated from outside of CACI. Do not click any links or attachments unless you recognize and trust the sender. https://jira.mongodb.org/s/en_USp8swtz-1988229788/6109/25/_/jira-logo-scaled.png Re: Using brew installs on mac doesn't seem to allow mongo connector installation<https://jira.mongodb.org/browse/KAFKA-288> Alexander Boese<https://jira.mongodb.org/secure/ViewProfile.jspa?name=alexander.boese%40caci.com> if what you are seeking is to spin up a quick dev/test on a mac, use docker and follow the quickstart https://docs.mongodb.com/kafka-connector/master/quick-start/ [Add Comment] <https://jira.mongodb.org/browse/KAFKA-288#add-comment> Add Comment<https://jira.mongodb.org/browse/KAFKA-288#add-comment> This message was sent from MongoDB's issue tracking system. To respond to this ticket, please login to jira.mongodb.org<https://jira.mongodb.org> using your JIRA, MongoDB Cloud Manager, or MongoDB Atlas credentials. ________________________________ This electronic message contains information from CACI International Inc or subsidiary companies, which may be company sensitive, proprietary, privileged or otherwise protected from disclosure. The information is intended to be used solely by the recipient(s) named above. If you are not an intended recipient, be aware that any review, disclosure, copying, distribution or use of this transmission or its contents is prohibited. If you have received this transmission in error, please notify the sender immediately. |
| Comment by Robert Walters [ 09/Feb/22 ] |
|
alexander.boese@caci.com if what you are seeking is to spin up a quick dev/test on a mac, use docker and follow the quickstart https://docs.mongodb.com/kafka-connector/master/quick-start/
|