记一次kafka事务提交失败

线上业务报错：

2019-10-29/20:27:48.621|l-fin-ycb-sync-server1.toc.pre.ali.dm|-|-|^_^|[org.springframework.kafka.KafkaListenerEndpointContainer#1-1-C-1] ERROR o.s.k.l.KafkaMessageListenerContainer$ListenerConsumer 864 - Consumer exception org.springframework.kafka.KafkaException: Stopped container; nested exception is java.lang.RuntimeException: Transaction rollback,cant be recovered,ListenerContainer will be aborted,you should restart application! at org.springframework.kafka.listener.ContainerStoppingBatchErrorHandler.handle(ContainerStoppingBatchErrorHandler.java:64) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.handleConsumerException(KafkaMessageListenerContainer.java:856) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:709) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.RuntimeException: Transaction rollback,cant be recovered,ListenerContainer will be aborted,you should restart application! at com.vipkid.commons.springboot.kafka.KafkaClientConfiguration$AbortedAfterRollbackProcessor.process(KafkaClientConfiguration.java:202) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.batchAfterRollback(KafkaMessageListenerContainer.java:1002) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchListenerInTx(KafkaMessageListenerContainer.java:989) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchListener(KafkaMessageListenerContainer.java:944) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:930) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:749) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:698) ... 3 common frames omitted Caused by: org.springframework.transaction.CannotCreateTransactionException: Could not create Kafka transaction; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while initializing transactional state in 30000ms. at org.springframework.kafka.transaction.KafkaTransactionManager.doBegin(KafkaTransactionManager.java:150) at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:378) at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:137) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeBatchListenerInTx(KafkaMessageListenerContainer.java:957) ... 7 common frames omitted

Broker端日志报错：

[] ERROR [ReplicaManager broker=4] Error processing append operation on partition __transaction_state-32 (kafka.server.ReplicaManager) org.apache.kafka.common.errors.NotEnoughReplicasException: Number of insync replicas for partition __transaction_state-32 is [1], below required minimum [2]

根据日志提示，发现__transaction_state 这个topic的副本数为1，小于最小同步副本数2. 消费者并没有配置这个最小副本数，上官网一查，默认值果然是2。

解决方案：将__transaction_state这个topic的副本数扩充