1

I'm new to server sent events,
I red trough many tutorials and watched many helpful videos and I still don't understand "what to do with the data that keep growing on every user request" ??.

I use NodeJS so I started here..
How To Use Server-Sent Events in Node.js to Build a Realtime App

this following example was taken from the above link..

// in the following function they push each fact to facts array in order to later send it to the client
// the FACTS array can grow and grow and ... ??

async function addFact(request, respsonse, next) {
  const newFact = request.body;
  facts.push(newFact);
  respsonse.json(newFact); 
  return sendFactToAll(newFact);
}


// this is optimal, only 1 fact is send.
function sendFactToAll(newFact) {
   clients.forEach(client => 
   client.response.write(`data:${JSON.stringify(newFact)}\n\n`))
}

// what about sending all facts
function sendFactsToAll(facts) {
   clients.forEach(client => 
   client.response.write(`data:${JSON.stringify(facts)}\n\n`))
}
  1. when to clear the FACTS array?
  2. if clearing the array, what about those users who need to get the data from that array?

THANK YOU!

1
  • If I understand correctly you want to keep all of the facts that will be stored (possibly more than can fit in memory). You might want to write them to a JSON file or save them to a database before clearing them from the array.
    – soma
    Commented Jun 20, 2021 at 18:00

1 Answer 1

1

what to do with the data that keep growing on every user request?

Currently the demo stores all facts, and sends them all out as the starter history for a client. As it is a demo they've not gone into the messy details of how to scale it for running forever.

In a real app you are likely to want to add an id (or timestamp) to each fact. Then you could use this with last-event-id on reconnect, to just get the facts they missed.

You are also likely to want to store the facts in a SQL DB of some kind, and maybe offer a separate web service to access the history. A client would first request as much history as they need, then start the event source request.

By the way, I'd make this history request service a separate web service if you wanted to scale. With event source the limiting factor for scaling is the number of simultaneous connections, whereas for storing history the limit is the disk space.

Another approach would be to only keep the most recent 100 or so facts, and not worry about a full history. You could do this by having addFact() first do facts = facts.slice(-99) before doing the push (see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/slice if that syntax is unfamiliar).

1
  • In my case, I'll take your advice and keep an x amount of facts with combination of using the last-event-id on reconnect. thank you Darren !! Commented Jun 21, 2021 at 8:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.