[KAFKA-365] The data should exceed 1.4mb, causing the producer send to fail Created: 24/Apr/23 Updated: 27/Oct/23 Resolved: 22/May/23 |
|
| Status: | Closed |
| Project: | Kafka Connector |
| Component/s: | None |
| Affects Version/s: | 1.9.1 |
| Fix Version/s: | None |
| Type: | Question | Priority: | Major - P3 |
| Reporter: | 晓阳 刘 | Assignee: | Robert Walters |
| Resolution: | Gone away | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Attachments: |
|
| Comments |
| Comment by Jeffrey Yemin [ 22/May/23 ] |
|
OK liuxiaoyang904.61@gmail.com thanks for following up. |
| Comment by 晓阳 刘 [ 22/May/23 ] |
|
dbeng-pm-botjeff.yemin@mongodb.comrobert.walters@mongodb.com Hi~, |
| Comment by PM Bot [ 18/May/23 ] |
|
Hi liuxiaoyang904.61@gmail.com! If this is still an issue for you, please open Jira to review the latest status and provide your feedback. Thanks! |
| Comment by PM Bot [ 11/May/23 ] |
|
Hey liuxiaoyang904.61@gmail.com, We need additional details to investigate the problem. If this is still an issue for you, please provide the requested information. |
| Comment by 晓阳 刘 [ 04/May/23 ] |
|
robert.walters@mongodb.com Hi,my kafka config is 10m |
| Comment by Robert Walters [ 01/May/23 ] |
|
liuxiaoyang904.61@gmail.com Can you ensure the following properties are set in your kafka broker? This would make 15MB the max
|
| Comment by 晓阳 刘 [ 25/Apr/23 ] |
|
jeff.yemin@mongodb.com log: |
| Comment by 晓阳 刘 [ 25/Apr/23 ] |
|
robert.walters@mongodb.com |
| Comment by Robert Walters [ 24/Apr/23 ] |
|
By default Kafka only handles messages about 1MB in size, it might be that your Kafka is not setup to handle more. I found this stackoverflow that suggests a few Kafka configurations to change the default size https://stackoverflow.com/questions/21020347/how-can-i-send-large-messages-with-kafka-over-15mb
|
| Comment by Jeffrey Yemin [ 24/Apr/23 ] |
|
Hi liuxiaoyang904.61@gmail.com, It's difficult to say with the information provided. If you give us more context around this error we may be able to assist you:
|
| Comment by 晓阳 刘 [ 24/Apr/23 ] |
|
|
| Comment by 晓阳 刘 [ 24/Apr/23 ] |
|
Caused by: org.apache.kafka.common.errors.RecordTooLargeException: The message is 1445052 bytes when serialized which is larger than the maximum request size you have configured with the max.request.size configuration. |