44

Writing an app in Python, and been playing with various ORM setups and straight SQL. All of which are ugly as sin.

I have been looking at ZODB as an object store, and it looks a promising alternative... would you recommend it? What are your experiences, problems, and criticism, particularly regarding developer's perspectives, scalability, integrity, long-term maintenance and alternatives? Anyone start a project with it and ditch it? Why?

Whilst the ideas behind ZODB, Pypersyst and others are interesting, there seems to be a lack of enthusiasm around for them :(

2
  • -1: Value Judgement: "beginning to hate the RDBMS/Object mismatch" Could you please eliminate the value judgement from the question? Is there a way to focus on the facts and reduce the emphasis on your personal hates?
    – S.Lott
    Mar 6, 2010 at 5:56
  • 16
    @S.Lott - A reluctant edit. Not because you are incorrect, but because the -1 came inline with a request which is just rude. This isn't the bayeux tapestry.
    – Aiden Bell
    Mar 6, 2010 at 11:18

5 Answers 5

27

I've used ZODB for more than ten years now, in Zope and outside. It's great if your data is hierarchical. The largest data store a customer operates has maybe. I don't know, 100GB in it? Something on that order of magnitude anyway.

Here is a performance comparison against Postgres.

If you're writing a WSGI web app, these packages may be useful:

4
  • 1
    Accepted. Not just because of the answer, but I keep coming across your posts/articles/presentations when googling ZODB stuff I need. I know who to direct further questions at! ;)
    – Aiden Bell
    Mar 7, 2010 at 23:49
  • This docs.zope.org/zope3/Code/BTrees/index.html is also useful
    – amirouche
    Jan 21, 2011 at 18:08
  • Really interesting link about the performance there. Worth a read!
    – fatuhoku
    Sep 21, 2013 at 19:30
  • 1
    "performance comparison against Postgres" the links expired, you can see cached page version at web.archive.org/web/20090218122454/http://…
    – d9k
    Oct 6, 2017 at 17:04
15

Compared to "any key-value store", the key features for ZODB would be automatic integration of attribute changes with real ACID transactions, and clean, "arbitrary" references to other persistent objects.

The ZODB is bigger than just the FileStorage used by default in Zope:

  • The RelStorage backend lets you put your data in an RDBMS which can be backed up, replicated, etc. using standard tools.
  • ZEO allows easy scaling of appservers and off-line jobs.
  • The two-phase commit support allows coordinating transactions among multiple databases, including RDBMSes (assuming that they provide a TPC-aware layer).
  • Easy hierarchy based on object attributes or containment: you don't need to write recursive self-joins to emulate it.
  • Filesystem-based BLOB support makes serving large files trivial to implement.

Overall, I'm very happy using ZODB for nearly any problem where the shape of the data is not obviously "square".

0
5

I would recommend it.

I really don't have any criticisms. If it's an object store your looking for, this is the one to use. I've stored 2.5 million objects in it before and didn't feel a pinch.

2

ZODB has been used for plenty of large databases

Most ZODB usage is/was probably Zope users who migrated away if they migrate away from Zope

Performance is not so good as relatonal database+ORM especially if you have lots of writes.

Long term maintenance is not so bad, you want to pack the database from time to time, but that can be done live.

You have to use ZEO if you are going to use more than one process with your ZODB which is quite a lot slower than using ZODB directly

I have no idea how ZODB performs on flash disks.

2
  • Thanks for the answer! I am wondering if ZODB will fit in place of MySQL using ZEO in a webfarm setup. Serious caching will hopefully negate the slow performance of the store.
    – Aiden Bell
    Mar 5, 2010 at 19:10
  • 3
    ZODB usually smokes relational databases for repeated queries when the data doesn't change, even with ZEO in the mix, because there's already a client-side cache. Mar 5, 2010 at 21:43
0

With pickling you should be able to use any key value database in a similar fashion.

3

Not the answer you're looking for? Browse other questions tagged or ask your own question.