'How to transform a nested array into a table in ksqldb/kafka

I started learning ksqldb for Kafka and ran into a problem, I have a Product stream and the structure is the following:

 ID          | VARCHAR(STRING)
 NAME        | VARCHAR(STRING)
 ...
 INGREDIENTS | ARRAY<STRUCT<NAME VARCHAR(STRING), ISALLERGEN BOOLEAN, RELATIVEAMOUNT DOUBLE>>

And as a result, I want to create a table and get the last state of the product using 'LATEST_BY_OFFSET' function, but the problem is that I can't apply this function to Array or Struct.

Should I EXPLODE each prop and create a separate entry in the table for each ingredient?(I think it's very strange). How can I deal with this situation? Can you tell me if you have any ideas?

apache-kafka ksqldb

Solution 1:^[1]

The problem was solved in release 0.25.1 https://github.com/confluentinc/ksql/pull/8878 https://github.com/confluentinc/ksql/commit/adac45855dce1c413073e5cbeb474cb022013a2b

Solution 2:^[2]

This is a known issue. It is documented here: https://github.com/confluentinc/ksql/issues/5437 and I happen to be working on it presently. Hopefully a solution will be available in an upcoming version!

I'm not sure how your workaround would work given that after you explode the array that there could be several instance of each component of the struct per key...

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Artsiom Krukouski
Solution 2	GeoJim

'How to transform a nested array into a table in ksqldb/kafka

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]