'Kafka 1.0.2 Exiting because log truncation is not allowed for partition X
First of all I have already tried setting the unclean.leader.election
to true and I am still having the same problem. The brokers are still exiting with this exception
[2022-05-13 12:10:39,988] FATAL [ReplicaFetcher replicaId=48, leaderId=38, fetcherId=0] Exiting because log truncation is not allowed for partition topic-46, current leader's latest offset 553608 is less than replica's latest offset 1440464418 (kafka.server.ReplicaFetcherThread)
Fact: The are no consumer groups currently on the topic. I disabled the subscribed consumers since I thought that by doing so I'd let the cluster recover. Unfortunately, this was not really the case.
My questions though is on this part of the error
current leader's latest offset 553608 is less than replica's latest offset 1440464418
To my understanding setting the leader offset to 1440464418 or the replica's offset to 553608.
The offsets though are only shifted using consumer groups. So here are my questions:
- Shouldn't these offsets be scoped with a consumer group dimension? What does it mean that a partition's latest offset is x? The information given should be: partition's
x
's latest offset on topicy
for consumer groupz
is w - Am I right? Is there actually a way to change the offset without consumer group info? The only command I've used and have found being used around for this situation is:
./usr/local/kafka/bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 --group <groupid> --topic <topic_name>:<partition_number> --reset-offsets --to-earliest --execute
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|