kafka 生产者报at kafka.producer.SyncProducer

 

当topic 创建的副本数超过kafka 的节点数时就会包如下错误：

log4j:ERROR Could not find value for key log4j.appender.R
log4j:ERROR Could not instantiate appender named "R".
Verifying properties
Property key.serializer.class is overridden to kafka.serializer.StringEncoder
Property metadata.broker.list is overridden to 221.122.77.10:9092
Property serializer.class is overridden to kafka.serializer.StringEncoder
Property zookeeper.connect is not valid
Fetching metadata from broker BrokerEndPoint(0,221.122.77.10,9092) with correlation id 0 for 1 topic(s) Set(car_test)
Connected to 221.122.77.10:9092 for producing
Disconnecting from 221.122.77.10:9092
Connected to slave101:9092 for producing
Disconnecting from slave101:9092
Failed to send producer request with correlation id 2 to broker 101 with data for partitions [car_test,0]
java.nio.channels.ClosedChannelException
	at kafka.network.BlockingChannel.send(BlockingChannel.scala:110)
	at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:75)
	at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:74)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:105)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:105)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:105)
	at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
	at kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:104)
	at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:104)
	at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:104)
	at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
	at kafka.producer.SyncProducer.send(SyncProducer.scala:103)
	at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:257)
	at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:108)
	at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:100)
	at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
	at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
	at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
	at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
	at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
	at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:100)
	at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:73)
	at kafka.producer.Producer.send(Producer.scala:77)
	at kafka.javaapi.producer.Producer.send(Producer.scala:33)
	at com.zhuoshi.jcbigdata.save.kafkaCeShi.main(kafkaCeShi.java:65)
Back off for 100 ms before retrying send. Remaining retries = 3
Fetching metadata from broker BrokerEndPoint(0,221.122.77.10,9092) with correlation id 3 for 1 topic(s) Set(car_test)
Connected to 221.122.77.10:9092 for producing
Disconnecting from 221.122.77.10:9092
Disconnecting from slave101:9092

解决方法:

创建topic的副本数要小于kafka 的节点数