52

While DOM still totally dominates the way we create UIs, does it make sense to create a bunch of entirely canvas-based UI components, like buttons, lists, horizontal/vertical groups, etc?

I know for sure that there will be a lot of drawbacks, but what would the possible advantages of such be?

For one, I'd say in general the visual integration with canvas will be much tighter.

5
  • 1
    This reminds me of the fashion a few years ago to create navigation buttons in Java. There were a small few advantages but the disadvantages were huge. IMO it's not a good idea.
    – Tak
    Jul 28, 2011 at 10:33
  • 1
    @Tak ... this was a thing that people actually... did? And they thought it was a good idea?!!
    – Alex K
    Mar 5, 2013 at 19:27
  • May be not a problem at all but I guess people which have to use screenreaders would not be able to use the page.
    – Werner
    Jun 23, 2017 at 5:36
  • Try bitbucket.org/nikola_l/visual-js , you can found online examples ... Jun 23, 2017 at 6:52
  • I'm really surprised by how many answers suggest this is a good idea, it is really not. The downsides are huge!
    – Jonathan
    May 22, 2022 at 14:47

10 Answers 10

51

The Zebra project has created a full component set which is rendered into a HTML5 canvas element. Here is a screenshot of the component sampler. I haven't used the framework, but it should give you an idea of how well the different browsers can render UI components.

enter image description here

Rotate the components and check the quality of line rendering (anti-aliasing), which is very different depending on the browser you use. Here's some more information regarding that problem:

Another project is Makepad, a webGL worker-based library and live code editor. Every visible part of the UI is rendered in WebGL, including all text on screen, rendered through the integrated text rendering engine.

Makepad - a webGL worker-based library and live code editor

It is still early phase for the project, but you can try out a live demo here. Makepad is open source, the Git repo can be found at github.com/makepad/makepad.github.io.

0
33

Using Canvas as a UI base is an excellent idea if you have > 200 elements. It's much, much faster to render than using DOM elements.

On iPhone Safari, 300 animated DOM elements runs at 3fps (frames per second), very slow.

If you use canvas, you can render > 300 elements and still achieve 30fps, which means smooth animation and transitions. I've tested this at length so I know it works.

The downside to Canvas (as someone else mentioned), is that search engines can't see your content. But if you are building an app that shouldn't be spidered and needs to run on mobile, then Canvas is the way to go.

2
  • 2
    Who cares if search engines can't see your content in the context of how this sort of UI would be deployed?
    – Alan B
    Jul 20, 2016 at 8:28
  • Good post. The advantage of self-rendering is potentially faster performance because the browser is built to be optimized for the general use case.
    – Pacerier
    Aug 18, 2016 at 21:52
25

Yes and no.

In terms of accenting the UI of the page, it's somewhat ideal if the page navigation and operation can also work well if JavaScript is not enabled.

It is okay to have UI elements that are prettified by canvas, but remember that these elements will not be accessible to web crawlers (like Google), or to users who have scripts disabled.

No:

Don't try to remake text editing in canvas. Even the HTML Canvas Spec has a section where they strongly advise against trying to create text-editing controls in Canvas.

There's a long history of trying to do that and failing (look up Mozilla Bespin)

I know for sure that there will be a lot of drawbacks, but what would the possible advantages of such be?

Yes:

Visualizations (that have no fallback except text description) and controls like dials (that fall back to some HTML, like input type="range" sliders) can be greatly enhanced with Canvas.

If you search for canvas controls, canvas diagrams, canvas charts, etc you'll find a lot of libraries that offer stuff like this. Just understand very well that many of these come at the expense of accessibility.

Maybe:

There are a lot of pretty elements you can add to a page with Canvas. Some things can get really beautiful without being intrusive or altering the page navigation in any way. Perhaps the logo of a website would "grow" procedurally or glow or otherwise get more complex. Other background animation effects might be really neat.

There are also interactive images, like on sites where you want a diagram or breakdown or exploded view that you would navigate to inspect the various parts of something (a chemical structure, a biological organism, a new car). Visual interactive media such as diagramming and games are some of the best use-cases for Canvas.

9
  • 3
    Actually, while I agree with most of your assertions, I would argue that this is not such a clear cut in some circumstances. For example, if you don't need your UI to be globally accessible (e.g. for a purely intranet-facing app), and of course in the cases mentioned by nick fallon in his answer (if you have a lot of items and have to run on mobile) -- and especially if those circumstances combine (as in my case, where we have what amounts to an in-browser spreadsheet with potentially thousands of cells, running strictly internally on the local network), canvas is definitely a viable option. Nov 19, 2013 at 12:09
  • 8
    Who says it's not indexable? Put all your meta data in some display:none elements, then make your entire ui in canvas, and leverage WebGL. Totally doable while maintaining indexability.
    – trusktr
    Apr 16, 2014 at 15:10
  • 5
    What about a canvas game that needs menu buttons?
    – Josh
    Jul 14, 2014 at 9:39
  • Often the game HUD/menus are done separately from the canvas, which makes internationalization a little easier, but I've seen it both ways. @AshleysBrain do you want to weigh in? Jul 14, 2014 at 13:04
  • 9
    The future will show that this answer is wrong. It will take time but the persisting diversity among browser implementations of DOM and CSS are about to finally drive every toward a more unified UI system. Flex, the UI component framework for Flash, existed because HTML had already failed once before, for this very reason. Now that we have canvas, it is just a matter of time until people get their heads on straight again. -1 for ignoring history. Aug 2, 2015 at 1:07
22

For the last four years I have been building components for the canvas including buttons, dials, sliders, check boxes, radio buttons, color pickers, panes, windows, indicators, waiters, steppers, tabs, pads, etc. see http://zimjs.com/components/ for working examples.

Dial and Slider in ZIMjs for Canvas

The advantages are as follows:

  1. The components can be customized in more or different ways than traditional HTML/CSS components and we can make more types of components. See http://zimjs.com/docs.html for examples.
  2. It is often important when making interactive works to embed the components right in the app or game. This is difficult to do by overlaying HTML components when taking into account scaling apps.
  3. There is definitely tighter integration when dragging panels, animating components, scaling components, working with canvas library events (ZIM and CreateJS use on() method which has benefits - canvas components can make use of this).

I love working with Canvas components - it saves lines of code and I don't have to switch between systems. Just a reminder... the format of CSS is basically the same as an object literal in code. I would rather format my components in code any day rather than CSS - personally, I find it much easier to work in one system.

In terms of screen reader results for interface - many canvas creations are not suited for visually impaired. It can still be done, as pointed out, if applicable.

One final comment... consistency is an important design principle but variety is the spice of life. I do not think we should be relying on a homogeneous interface system. There should be room for growth, experimentation and exploration.

5
  • 1
    Since the time of the response above, ZIM has added Accessibility for all its canvas components. As this is not directly supported by a single bitmap canvas, the support is by placing HTML tags in behind the canvas. zimjs.com/accessibility But this does give an example of how Accessibility and canvas components can work.
    – Dan Zen
    Mar 13, 2018 at 15:04
  • Looks interesting, but... the product page is a bit offputting. Here's a quote: "Digital advertising has surpassed traditional media buys. ZIM provides easy interaction and animation and can often be distilled to 40k to keep ad sizes low. See MORE for ideas and tips..." Those are in fact words there, but they are meaningless. The entire product page seems to be full of snips like that to the extent that it actually gave me a headache. Sep 4, 2019 at 15:33
  • PS... upvoting anyway because I really do like this concept, just need to mellow the product page a bit to make it less chaotic feeling. Sep 4, 2019 at 15:39
  • Thanks for the note @slumtrimpet. That description is from one of ten types of features where coding on the Canvas makes sense - in this case interactive advertising (admusements). These sections are somewhat promotional - compared to the rest of the site - sorry they gave you a headache and thanks for the upvote.
    – Dan Zen
    Sep 5, 2019 at 23:18
  • Amazing work! I agree that we need to experiment with interfaces, and I hope to see an increase of such projects in future. May 27, 2020 at 19:16
4

That sounds like a bad idea. You will lose much accessibility that the user expect e.g. focus and tabbing. Or it will be a lot of work for you to implement all that.

It's much better to use HTML5 and CSS3 for such things. There is many JavaScript GUI Frameworks available e.g. see 15 Javascript Web UI Libraries, Frameworks and Toolkits.

1
  • 3
    The two billion Javascript UI libraries, frameworks and toolkits.
    – Alan B
    Jul 20, 2016 at 8:27
3

We've tried something like this but finally came up with the idea that the world is not ready yet )

You should keep in mind following

  • js always should be enabled. Nowadays one can consider it not a big deal, but nevertheless it worth to mention.
  • html/css is actually traditional and constantly evolving stack of standards, sooner or later you'll feel the need in having some descriptive language to reduce repeating code in your canvas rendered UI-components. And there are two options here - to try to invent something proprietary, which actually could be fun and interesting, but can have some very sad consequences. The second way is to reimplement html/css not to confuse third party developers. But, wait a minute, we've already have html/css engine )))
  • events and, therefore, user experience. Jonas is right. Trying to reimplement even a subset of js event model to make it more comfortable to develop canvas rendered components is hard. Some issues even are unsolvable.

So, it is actually interesting experience, but I would not recommend.

1

Here is why I am choosing to do this:

  1. My resume will render the same across all browsers.

  2. I can re-use the code in my Go (Google Go) project. I am using SFML to set pixels, so as long as I don't try anything super "clever" the code will transcribe easily.

  3. It's good practice, and while it's bad to re-invent the wheel, someone has to know how to do this.

1

I developed a JavaScript library that lets users create user interfaces using the canvas element. You can build UI elements of different types and you can event create your own types of elements like a tetris piece, a sudoku...

This is the library: canvasui-js.com

1
  • As it’s currently written, your answer is unclear. Please edit to add additional details that will help others understand how this addresses the question asked. You can find more information on how to write good answers in the help center. May 23, 2022 at 6:54
0

I thought about the idea to do so because DOM rendering and the available ecosystem of libraries seems to miss a lot of my use cases.

As others already mention, you may have shortcomings of a non-DOM based UI in the following areas :

  • Search engine crawling
  • Accessibility using assistive technologies
  • Text typing experience

Evaluate your use case and chances are that you do not need all of these. You do not require crawling ability by search engines everywhere. A password-protected admin area does not have to be indexed on Google. Regarding accessibility, at least make sure that widgets can be navigated using the keyboard using Tab and arrow keys, just like desktop apps have been doing for ages. Support for assistive technologies and text typing can be a different story, though, because of the variety of input methods (physical keyboard, on-screen keyboard for touch apps, pinyin / complex input methods for Asian languages...)

On the other side, here are the shortcomings of a DOM-based UI I found out regarding my use cases:

  • Different support of CSS properties over browser versions. If something useful comes out, for instance a new CSS property, I still need to wait for some months or years for the browser user base figures to catch up in order to make sure the new feature will be widely supported, or use hacky polyfills. If relying on a single UI library that is not dependent on browser CSS, I can update it as soon as it is released and start using it right away.

  • Scrolls by default: the assumption that everything is on a scrollable webpage. I have to fight against this system by strikes of overflow:hidden and similar. There are quirks everywhere, such as those 3px of overflow and some parasiting unwanted "giggling" scrolling for instance you spend hours hunting with the CSS debugging tools... Components that grow from the inside are able to "push the walls" and make their containers overflow in turn by default.

  • The 300ms touch feedback delay on touchscreens, which is quite difficult to counter. Having a button that lights up as soon as your finger lands on the glass is still a struggle without hacking your way around, because the browser takes this time to evaluate if the gesture is a scroll... even on content that does not scroll. This is the very aspect that speaks out "sluggish" or "webview" to me, along with accidental selection of text labels or the browser context menu popping up on long press.

  • Most DOM UI libraries are opinionated for mobile "website" aspect and have limits on customizability. For example, using Bootstrap, you have to stick to those 12 columns and preset breakpoint sizes, spacings between components are fixed, you can choose sizes among a set number of options only. Else you have to get your hands dirty in CSS anyway or recompile the framework.

  • Widgets used to build a desktop-app-like UI are few and far between in the DOM UI library world. I mean data grids with resizable columns, tree views, draggable split panes, menu bars, toolbars that are able to hide some of their buttons as they are sized down and show a "more" menu automatically instead, among others. Some offering exists but they are paid and not cheap, while such components are available for free in some desktop UI libraries such as Qt.

Some of those aspects can be worked around by a bit of hacking, while others are more difficult.

Actually, the process of showing widgets in the browser without using the DOM and directly drawing those on screen is widely used in the world of game engines. For example, the Godot engine uses webassembly to render on the browser window in its HTML5 exports, and this includes a comprehensive set of UI widgets you can use.

0

Google Docs uses canvas based rendering since 2021, reason:

improve performance and improve consistency in how content appears across different platforms.

It also confirms some of the points mentioned in other responses about performance

Accesibility integration

We will continue to ensure assistive technology is supported, and work on additional accessibility improvements enabled by canvas-based rendering.

Not the answer you're looking for? Browse other questions tagged or ask your own question.