'Extracting a particular part of Kafka Topic Msg in Python
I am trying to extract the VALUE part from a bunch pf Kafka Topic Msg's in Python. I am trying to subscribe to a Kafka Topic and read the Latest Message and parse it. While trying to split the below Kafka Topic Msg by '=',i get the below error -
AttributeError: 'ConsumerRecord' object has no attribute 'split'
Also the object type of the below string is -
<class 'kafka.consumer.fetcher.ConsumerRecord'>
Sample Kafka Topic Msg -
ConsumerRecord(topic=u'xxxxxxxx', partition=0, offset=55, timestamp=XXXXXXXXXX, timestamp_type=0, key=None, value='{"snapshot":[{"active":"XXXXXXXXXX","buffers":"0","cached":"XXXXXXXXXX","inactive":"313581568","memory_free":"XXXXXXXXXX","memory_total":"XXXXXXXXXX","swap_cached":"0","swap_free":"0","swap_total":"0"}],"action":"snapshot","name":"memory_info","hostIdentifier":"XXXXXXXXXX","calendarTime":"Wed Sep 6 18:54:48 2017 UTC","unixTime":"XXXXXXXXXX","epoch":"0"}', checksum=-XXXXXXXXXX, serialized_key_size=-1, serialized_value_size=375)
Could someone please guide as to what's the best way to extract the Highlighted (bold) part of the Kafka Topic Msg.
Solution 1:[1]
Solved!
Just use value
attribute of that record.
For example, the following is my Kafka Consumer code.
from kafka import KafkaConsumer
consumer = KafkaConsumer(
bootstrap_servers='localhost:9092',
auto_offset_reset='latest',
group_id='test'
)
consumer.subscribe('test')
for msg in consumer:
print(msg.value.decode('UTF-8'))
In the above code, we have written msg.value
in the lastline, which gives the value of the data we want to extract from that ConsumerRecord of Kafka.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
Solution | Source |
---|---|
Solution 1 | vagdevi k |