I have a novel approach to this problem for those that want to limit this to one request. Event Stream requests pass cookies to the server the same as any AJAX request. My approach is to set cookies containing the data before making the request, and then immediately discard the cookies upon the first response.
Now there are a few limitations with cookies to be aware of. First of all, a cookie's max length is 4096 bytes including the name as well as potentially 3 bytes of overhead. Secondly, the safe amount of cookies per browser is 50. Different browsers have different maximum amounts ranging from Google's 180 to Android's 50.
I'll leave it up to you to determine how you'll implement logic around these limitations, but I'll provide an example of my implementation. Note that this implementation uses js-cookies to manipulate the cookies. We're also making use of the cookie-parser middleware on the server side.
Browser JS
const productIDs = ["b0708d2c-fe46-4251-96e0-1cfb3cd05eb0", "244d1e73-b5b4-4c59-8d4e-006fc1b190fe"];
const size = new TextEncoder().encode(JSON.stringify(productIDs)).length;
const chunkSize = Math.floor(1024 * 3.5);
const segments = Math.ceil(size / chunkSize);
const currentCookies = Object.keys(Cookies.get()).length;
if (segments + currentCookies <= 50){
let payloads = [];
for (let segment = 0; segment < segments; segment++){
const payloadLength = Math.ceil(productIDs.length / segments);
const startPosition = segment * payloadLength;
payloads[segment] = productIDs.slice(startPosition, startPosition + payloadLength);
}
for (const [index, payload] of Object.entries(payloads)){
Cookies.set(`qp[${index}]`, JSON.stringify(payload));
}
const source = new EventSource("start-processing");
source.addEventListener("connected", () => {
for (const [index, payload] of Object.entries(payloads)){
Cookies.remove(`qp[${index}]`);
}
});
}else{
console.log(`ERROR: Cookie limit was reached.`);
}
As you can see, we set the chunkSize
to 3584 bytes. This seems like a pretty safe amount of padding but you can tweak it to suit your needs. Keep in mind that this method will evenly distribute array values across all payload arrays. That means you're unlikely to be hitting that chunkSize value exactly.
You can adjust the maximum cookies from 50 to whatever you want. If you don't care about mobile or legacy and just want to support modern desktop browsers, 150 is the minimum to support Firefox. Check out this page for more information on browser limitations.
After we've split the productIDs into their own individual arrays, we generate cookies for each chunk. In this case, I've labelled it qp
and I follow it with a [0]
identifier indexed to the chunk.
Finally we initiate our event stream. You'll want to have an event that occurs as an immediate response so your browser knows to clear the newly created cookies. It would also be wise to have an independent function somewhere that clears these cookies in case a response never comes, or the browser is refreshed before the response comes through.
I recommend adding the following code to the start of the above code block in order to clean up any prior data that may be left over.
for (const cookie of Object.keys(Cookies.get()).filter(x => x.startsWith("qp"))){
Cookies.remove(cookie);
}
Server-side JS (Express)
async (request, response) => {
response.writeHead(200, {
'Content-Type': 'text/event-stream',
'Connection': 'keep-alive',
'Cache-Control': 'no-cache'
});
const productIDs = Object.entries(request.cookies).filter(([name, value]) => name.startsWith("qp")).map(([index, value]) => JSON.parse(value)).flat();
const sendEvent = (event, data) => response.write(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`);
sendEvent("connected", {});
}
Pretty straightforward stuff here. That rather long one liner is a really simple way of obtaining all of the arrays and flattening them into a single one. Just make sure that startsWith("qp")
part is set to your cookie name and obviously make sure no other cookies on your site start with that string.
sendEvent()
is a very simple function to respond to your client. Just make sure to respond immediately with the connected
event to make sure the cookies are cleaned up properly.
And that's it! It really is such a simple approach, and while it has its flaws in terms of cookie limitations, I feel that a forewarned developer will be able to make the judgement as to whether to make use of the technique. Once made into an easily reusable function, it would be a piece of cake to implement anywhere you need.