'How to backup all streams in JetStream server?

Problem:

I have to backup all the streams of my JetStream server. I don't know what streams are there. I want to backup all the streams into a single snapshot. Currently, we can backup only a single stream using the following command.

$ nats stream backup <stream-name> /backup/dir/<stream-name>.tgz

What I have tried so far:

I have tried providing a wildcard instead of <stream-name>. It does not work.

$ nats stream backup * /backup/dir/backup.tgz
nats: error: "*" is not a valid stream name, try --help

Possible Workaround:

At first, I can list all the streams using nats str ls command. Then, I can loop through all the streams and backup them individualy.

However, this does not satisfy my requirement as I want to backup all the streams into a single snapshot. My snapshot should represent a complete state of the JetStream server just not a single stream.



Solution 1:[1]

[Edited as there is now a better way to do this through new functionality]

Updated answer:

You can backup all of the streams for an account using nats account backup [<flags>] <directory to create the backup in>

Original answer:

Yes creating a script to capture the list of all currently defined streams using nats stream ls and then backing them up in a loop using nats stream backup is the best you can do.

It is not possible to create a single 'frozen at a point in time' complete backup of all the streams at the same time because:

  • NATS and JetStream continue to operate while the backup is happening
  • NATS and JetStream are distributed systems: servers and streams are independent from each other

The only way to create such a backup would be to synchronously suspend all writes (publications or consumptions of messages) on all streams and their source subjects, perform the backup (which could take a fair amount of time), and then allow operations on the streams and subject to resume. This interruption in service would not be acceptable to many (most?) nats users.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1