SSE/Redis - how to recover messages sent when SSE goes offline

Question

On a website I have a very simple Live chat setup that uses SSE/Redis and pub/sub structure.

The basic setup (without going into details) is:

1. Client-side using EventSource

Opens SSE connection and subscribes to live events sent by SSE daemon. Sends messages to an API endpoint

connect(hash, eventListener) {

    const url = `${url}?client=$hash=${hash}`;
    sseSource = new EventSource(url);

    sseSource.onopen = function(e) {
      reconnectFrequencySeconds = 1;
    }

    sseSource.onerror = err => {
      this.closeSSEStream();
      this.reconnectSSEStream(hash, eventListener);
    };
    sseSource.addEventListener('messages', event => {
      const messages = JSON.parse(event.data);
      eventListener(messages);
    });
  },

2. API endpoint

That stores message in the database and pushes it to a Redis channel.

3. Redis DB

That keeps and serves the messages.

4. Server-side SSE daemon

Subscribes client to a channel in a Redis DB and forwards messages to the subscribers using SSE stream.

const subscriber = redis.createClient();
subscriber.select(config.redisDatabase);

subscriber.on('message', function (channel, message) {
    log(connectionId, 'Redis: new msg on channel: ' + channel, message);
    let event = {
        event: 'messages',
        data: message
    };
    currentClient.connection.write(event);
  });

The whole thing works pretty well, however, it is one tweak away from perfection. During deploy we restart our workers (including SSE daemon) and while it goes offline users do not receive LIVE updates. It reconnects just fine but messages that have been sent during down time are lost (as daemon starts listening to messages on reconnect only).

My only idea for a workaround involves an overengineered solution where "lost" messages are collected with a separate API endpoint on reconnect and displayed to the user.

Is there an out-of-the-box way to receive messages that have been stored to Redis BEFORE subscribing to a channel? E.g. "pop" unprocessed messages or something like that?

Answer 1

when you have reconnected send request to check if you are new msg with time of last msg and if you are newer msg send it in result msg to avoid new request