7

1. The problem

Lately, it seems that many note managers with "infinite" tree structure are choosing a block model (where each paragraph is an entry in the DB), instead of a document or file model.

Blocks Documents
Notion
Workflowy
Remnote
Dynalist
Roam Research
Evernote
Obsidian
Bear app

If you find any errors in the table, please let me know.

We have been developing an app very similar to Notion for 8 months now, also using the block model, but we are considering making a radical change and switching to the document model. The structure of our blocks in MongoDB currently looks like this:

_id: "61fd3ede7f6d2cc7a53ca669"
children: Array
    0: "61fd3ee87f6d2cc7a53ca66b"
    1: "61fd3ef37f6d2cc7a53ca671"
    2: "61fd3ef77f6d2cc7a53ca673"
backlinks: Array
type: "bullet"
parentPage: Array
    _id: "61fd3ede7f6e2ccra53ca664"
    userParent: "german-jablo"
    permisionParent: "edit, comment, read"
parentParagraph: "61fd3ede7f6d2cc7a53ca668"
content: "<p>This is a paragraph</p>"
isCollapsed: false
createdAt: 2022-02-04T14:57:34.280+00:00
updatedAt: 2022-02-04T14:57:59.585+00:00

Many pages talk about the differences of both approaches (example) although in a very vague way, so we decided to open this thread to find a more scientific answer to the question.

Features of our app

app Blocks that can be opened as documents Blocks that can collapse or expand their children
Notion Page type blocks Toggle type blocks
Workflowy All All
Evernote Documents None
Our app Page type blocks All others

Our app has two types of "blocks". The page type (which, like in Notion, can be inserted into any note and generate a document "inside" the current document), and the rest of the blocks, which are equivalent to the "toggle" block type in Notion (i.e. they can be collapsed or their nested children can be expanded).

2. What we have tried

In trying to answer our question (which DB model would work best for our application), we've realized that the answer is probably "it depends". Perhaps both models have strengths or weaknesses in different types of operations or situations. That is why we formulated this comparison table describing how we believe the performance of both models would be for each of these operations.

Operation Blocks Documents Apparent Winner
Fetch the contents of a page Find all paragraphs in the DB. Search the document in the DB. Document
Render the content of a page** Build the tree from the paragraphs recursively. You can omit the children of paragraphs whose isCollapsed property is true Render the document Document
Update the content of a paragraph in the DB Only the modified paragraph is rewritten The whole document is rewritten Block
Alternatives for rendering very large documents * Blocks can be fetched or rendered as you scroll (as Workflowy does), or as you expand child paragraphs that were collapsed. I thought that Grifds could achieve similar behavior, breaking the document into smaller chunks and bringing them in piecemeal, but it doesn't support updating an individual chunk, or even the entire document. It could also corrupt an HTML by splitting it into binary format. Block
Import or paste content In addition to converting the clipboard to HTML and/or sanitizing it, you must set up paragraphs with tree structure recursively. Note: Roam Research e.g. supports importing in JSON format, but generally users do not handle this format beforehand. Only convert the clipboard to HTML and/or sanitize the clipboard Document
Copy content** Clipboard must be sanitized and/or transformed Correct by default** Document
Real-Time Collaboration At the document level, could use some tree-based (Json) library like Automerge, or combine with some CRDT library for paragraph level. Could use tinymce solution. Tie? Both seem to have their advantages and disadvantages.

*Render very large documents: Most users probably do not use notes larger than 250 kb (considering that multimedia files are referenced in a separate collection). Still, in the document model, the question arises: how can we load, render or edit large documents in manageable chunks? One idea we came up with is to split HTML documents that reach large dimensions into portions of a certain size in kb, instead of splitting them into paragraphs. (It would be like a kind of Gridfs that allows you to modify the file in parts.) Could this be a good idea?

**Should the DOM be nested? In order to be able to collapse or expand nested child paragraphs, note managers with a block model structure the DOM in a nested way (paragraphs are in divs, inside their parent divs, etc.). However, an alternative in the document model could be that when the user presses tab, only that block (HTML tag such as <p> or <li>) is assigned an attribute with a number less than or equal to 1, representing the nesting levels relative to the previous block. This way when you press tab to nest or shift-tab to un-nest, you only have to modify one attribute of an HTML element instead of many elements; and the DOM stays simple, without having nested blocks.

3. Our conclusions

We believe that for each of the rows in the comparison table, benchmarks could be done measuring the performance of both models. Other people have done something similar here and here, comparing the performance of note managers using both models. The problem with those tests is that it is difficult to draw an accurate conclusion about the goodness of both models. Obsidian uses documents locally, so you don't have to sync notes. Roam Research is a very new and poorly optimized app. Standard Notes encrypts notes locally. In other words, it's not always apples to apples.

And even if tests could be done, we believe that the answer may even depend on how each user uses the application. Suppose user A usually organizes his notes in long documents using paragraph nesting (to collapse or expand them). On the other hand user B usually organizes his notes by creating new documents within documents. It is likely that a block model based manager would work better for user A while a document based one would work better for B.

So, we tried to push our doubt as far as we could, but we are still not sure of the answer Which of the two models do you think would offer better performance for our app and why?

4. Update

I just found some very interesting information. It seems that both TinyMCE and CKEditor (up to version 4), the view and the model converge with the HTML being based on content-editable. However, CKEditor 5 switched to MVC [source 1], [source 2]. image taken from tinymce's blog

I've done a short test pasting a large clipboard of a few MB in TinyMCE 5, CKEditor 4 and CKEditor 5, and the latter has been a bit slower. I hope soon to be able to do more tests with other things like dragging blocks or rendering large documents.

In a GitHub thread about CKEditor 5's performance when working with large documents, one of the contributors said "It works slower than the native content-editable element obviously, but the typing experience is pretty well".

8
  • What's your expectation posting here?
    – wahwahwah
    Feb 7, 2022 at 20:16
  • 1
    Get answers, ideas, or information about my question :) Feb 7, 2022 at 20:18
  • I love this post, but it's 100% against the the rules.
    – wahwahwah
    Feb 7, 2022 at 23:37
  • I have read the rules several times. It would be more helpful if you said which rule is being violated. Feb 8, 2022 at 0:58
  • "Against the rules" might have been a misstatement. The help section on questions to avoid asking might be helpful here.
    – wahwahwah
    Feb 8, 2022 at 14:27

3 Answers 3

0

Looks like you've done your homework. Database modeling is sometimes a bit of an art rather than a science. I think that with both models, you can achieve good performance if you optimize them well. So I would recommend that you go for the one that requires less work. Since you've been working on the block model for 8 months already, that's probably the best option for you.

0

Love how thorough you thought about the design of your application. I just want to add some suggestions you might look:

  • Obsidian is using a hybrid approach. It started document-based, but now supports block links and embeds while still being super-fast. From all programs tested in my benchmark you linked above, Obsidian was the fastest.
  • One of the most important functions of a notetaking tool is effectively searching the probably thousands of notes. I created an amazingly simple test (the "Spaghetti Parmesan"-test) where all block-based approaches currently fail. It is about searching for two ingredients (spaghetti and parmesan) in a recipe. When both ingredients are in different blocks, all common block-based applications ultimately fail to find the recipe. You can read more about this here. I also tried to start a discussion with some of the authors on Twitter but failed to get any serious results. If you want to continue the block-based approach, you might start designing a search algorithm that can handle search terms spread about blocks (if you haven't yet). I tried to outline an algorithm in the thread linked above, but not sure if it will really work with hundreds of thousands of blocks.
1
  • Thanks for your input. I wouldn't say that Obsidian has a hybrid approach. It is based on documents. Just that it replicates block functions using techniques like the ones I describe. It is true that it is very fast, but it is also true that it would be unfair to compare it with apps that are synchronized in the cloud. Still, it doesn't seem to lag far behind apps like Notion in the other comparison I linked. Feb 7, 2022 at 21:25
-1

You seem to be making your choice based on what is easier to model or develop.

If you intend to compete in this increasingly crowded market, you need to consider what will give you an edge, and what will be sustainable in the long run.

Two of the main issues Evernote (longtime user and former employee) faced were:

  • a very badly designed back-end that was very expensive to scale and made sharing and collaboration a nightmare (engineering-wise)
  • a too loose document model that made it very hard to add new features to the editor (real time collaboration) and made collaboration very finicky even for simple features (checkboxes would often generate so many note conflicts as to be unusable)

On the other hand, block-based tools usually have really bad web clippers, because it's very hard to go from HTML to blocks.

Solve some of these hard problems!

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.