'What is the best practice of streaming JSON objects in Nodejs, if it is bigger than 10M and split it into multiple streams?
I have a JSON object, it is a big object. I need to send (write) it as a Kafka message and it takes a lot of time.
I need a mechanism to stream the JSON object and if it is bigger than 10M, split it into multiple stream messages and then send it.
Please help me to find a best practice.
Thank you.
Solution 1:[1]
If it is possible to do decompress in the client. I would do :
- compress 10MB json file. It will reduce to about 115KB
- encode base64 the file. It will add the size to 160KB
160KB is a lot smaller than 10MB and should be easy to streamed. It's similar with what netflix does ( compression ) that's why their movie stream so fast in our TV
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | surga |