'Can I direct Apache Beam late data into another output?
Is it possible to route late data into another output instead of dropping it? I see that Flink supports this - getting late data as a side output.
Solution 1:[1]
You can update a Beam pipeline to not drop late data by specifying Window.withAllowedLateness.
This data can be processed or pushed to a different output using a subsequent ParDo transform.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | chamikara |