1

I need some help with Firestore building a chat app. I've looked at the documentation but I couldn't find the answer I need.

I'm building a real-time chat (many-to-many) that must handle thousands of messages, and those messages can also be edited, deleted and undeleted. The main problem is that loading all the messages once (as Firebase suggests) and then manage them on the FrontEnd side freezes my frontend application for the huge amount of messages. I tried to do that with the pagination API but I got some edge cases e.g.

let last = null;

// Execute the first query saving the last doc
const ref = db
 .collection('chat')
 .orderBy('timestamp')
 .limit(10)
 .onSnapshot(({docs}) => {
    last = docs[docs.length - 1]
   // my logic
 })

// When needed I'll execute the second query starting from the last doc
// of the previous query
const ref = db
 .collection('chat')
 .orderBy('timestamp')
 .limit(10)
 .startAfter(last)
 .onSnapshot(({docs}) => {
    last = docs[docs.length - 1]
    // my logic
 })

If the last document returned by the first query is deleted, I get the update only from the first query, e.g.

[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9] // original docs from the first query
[10,11,12,13,14,15,16,17,18,19] // original docs from the second query

Deleting the document 9 triggers the update from the first query so on my frontend I will have the document 10 duplicated.

[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 10] // updated docs from the first query
[10,11,12,13,14,15,16,17,18,19] // original docs from the second query

I'm guessing that there are also other edge cases that I hadn't considered. Do you have some suggestion aboutn how can I handle this big amount of messages with Firebase? Is there any error in my approach with the pagination? Or maybe this is the wrong approach?

0

1 Answer 1

1

Your approach is fine, but it turns out that pagination and realtime updates with a cursor based API lead to some tricky edge-cases.

You'd have to either remove the duplicate documents based on their ID and then have pages with different sizes, or update the starting point of the second query (and then later queries as you have more pages).

There are more such edge cases, which is one of the reasons the FirestorePagingAdapter in FirebaseUI doesn't handle realtime updates.

2
  • Thanks for the response. I'd like to ask one more think. Is there a limit to a real-time update per second that firestore-js can handle ? I try writing thousand of messages with a couple of load test tool and even if my onSnapshot just execute a console.log my CPU goes to 100% and the UI got freeze. If I add some sleep in some way, the CPU won't go to 100%. Is it normal ?
    – lucataglia
    Nov 12, 2021 at 17:06
  • 1
    On the client it could be all kinds of things, so that's hard to say. On the server, you're likely to run into hotspotting since you have an index on timestamp. See cloud.google.com/firestore/docs/best-practices#indexes Nov 12, 2021 at 17:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.