Converting a Buffer into a ReadableStream in Node.js

Question

I have a library that takes as input a ReadableStream, but my input is just a base64 format image. I could convert the data I have in a Buffer like so:

var img = new Buffer(img_string, 'base64');

But I have no idea how to convert it to a ReadableStream or convert the Buffer I obtained to a ReadableStream.

Is there a way to do this?

Answer 1

You can create a ReadableStream using Node Stream Buffers like so:

// Initialize stream
var myReadableStreamBuffer = new streamBuffers.ReadableStreamBuffer({
  frequency: 10,      // in milliseconds.
  chunkSize: 2048     // in bytes.
}); 

// With a buffer
myReadableStreamBuffer.put(aBuffer);

// Or with a string
myReadableStreamBuffer.put("A String", "utf8");

The frequency cannot be 0 so this will introduce a certain delay.

Answer 2

For nodejs 10.17.0 and up:

const { Readable } = require('stream');

const stream = Readable.from(myBuffer.toString());

Answer 3

something like this...

import { Readable } from 'stream'

const buffer = new Buffer(img_string, 'base64')
const readable = new Readable()
readable._read = () => {} // _read is required but you can noop it
readable.push(buffer)
readable.push(null)

readable.pipe(consumer) // consume the stream

In the general course, a readable stream's _read function should collect data from the underlying source and push it incrementally ensuring you don't harvest a huge source into memory before it's needed.

In this case though you already have the source in memory, so _read is not required.

Pushing the whole buffer just wraps it in the readable stream api.

Answer 4

Node Stream Buffer is obviously designed for use in testing; the inability to avoid a delay makes it a poor choice for production use.

Gabriel Llamas suggests streamifier in this answer: How to wrap a buffer as a stream2 Readable stream?

Answer 5

You don't need to add a whole npm lib for a single file. i refactored it to typescript:

import { Readable, ReadableOptions } from "stream";

export class MultiStream extends Readable {
  _object: any;
  constructor(object: any, options: ReadableOptions) {
    super(object instanceof Buffer || typeof object === "string" ? options : { objectMode: true });
    this._object = object;
  }
  _read = () => {
    this.push(this._object);
    this._object = null;
  };
}

based on node-streamifier (the best option as said above).

Answer 6

You can use the standard NodeJS stream API for this - stream.Readable.from

const { Readable } = require('stream');
const stream = Readable.from(buffer);

Note: Don't convert a buffer to string (buffer.toString()) if the buffer contains binary data. It will lead to corrupted binary files.

Answer 7

Here is a simple solution using streamifier module.

const streamifier = require('streamifier');
streamifier.createReadStream(new Buffer ([97, 98, 99])).pipe(process.stdout);

You can use Strings, Buffer and Object as its arguments.

Answer 8

This is my simple code for this.

import { Readable } from 'stream';

const newStream = new Readable({
                    read() {
                      this.push(someBuffer);
                    },
                  })

Answer 9

Try this:

const Duplex = require('stream').Duplex;  // core NodeJS API
function bufferToStream(buffer) {  
  let stream = new Duplex();
  stream.push(buffer);
  stream.push(null);
  return stream;
}

Source: Brian Mancini -> http://derpturkey.com/buffer-to-stream-in-node/