For https://developer.ubuntu.com we use Django CMS 3.0.6 (mptt) and we're very happy with it.

Recently we added the functionality for us to import markdown docs from bzr branches of projects, so we can have cronjobs update articles for us whenever things changed in some of the upstream projects we are interested in. It took me some time to complete it, but we got it working. Locally I used sqlite (the default) and was happy. Once it was deployed to staging, the import worked as well, but the articles which were added or updated by the script ended up in a different place in the article tree. After some time I could reproduce this with a local postgres installation.

We are currently tracking the issue here: https://bugs.launchpad.net/developer-ubuntu-com/+bug/1506861

I'm a bit lost and don't quite know how to debug this.

I put together a test-case for this and it's basically:

$ bzr branch lp:~developer-ubuntu-com-dev/developer-ubuntu-com/debug-md-importer
$ cd debug-md-importer
$ virtualenv ./env
$ ./env/bin/pip install -r requirements.txt
$ ./env/bin/pip install django psycopg2

Then apply a diff like http://pastebin.ubuntu.com/12798510/ to developer_portal/settings.py and run

$ sudo apt install python-pip python-dev libpq-dev postgresql postgresql-contrib
$ sudo -u postgres psql -c "CREATE USER developerportal_user WITH PASSWORD '123abc';"

To see the issue in action then run:

$ ./postgres-retry.sh

in a postgres setting and

$ ./sqlite-retry.sh

in the sqlite setting.

The scripts will create a new db from scratch, run the importer with debug messages and run runserver for you, so you can easily check in the pages list how the structure changed.

Current state of things: sqlite does fine and postgres has the article tree messed up.

All the relevant code for this problem can be found in developer_portal/management/commands/import-external-docs-branches.py

If anyone could give me a pointer, it'd be much much appreciated.

Thanks a lot in advance.

Have a great day, Daniel

share|improve this question
    
If you don't use bzr, you can get content of the example branch here too: daniel.holba.ch/temp/debug-md-importer.tar.xz – dholbach Oct 22 '15 at 15:00
    
Its is a pity that you have wrapped your schema into two layers of tooling. Maybe if you also supplied a database dump, more people could take a look at it. (it seems to me that this could be a simple DBA thingy, since this is your first port) – wildplasser Oct 22 '15 at 18:45
    
There's db dump included in the branch? Or is it missing anything? What do you mean by "first port"? Could you elaborate? – dholbach Oct 23 '15 at 6:14
    
database dump, as in: pg_dump -U user_name database_name >dumpfile.dmp – wildplasser Oct 23 '15 at 7:36
    
Hum... this can be reproduced easily with the instructions above, so it doesn't seem to matter if it's a default installation with a default Django+Django CMS set up or the actual developer.u.c production host. The instructions above should get you a working db and can show the actual issue. – dholbach Oct 27 '15 at 17:09
up vote 3 down vote accepted

I see from http://bazaar.launchpad.net/~developer-ubuntu-com-dev/developer-ubuntu-com/debug-md-importer/view/head:/requirements.txt#L18 that you are on django CMS 3.0.6, which means you're using Django MPTT.

Generally, most of the tree problems people have encountered with django CMS have turned out to be corruption after moves, related to Django MPTT.

If you haven't already done so, you might try one of the following:

  • instead of using django-mptt==0.6.1 try the lastest version of https://github.com/django-mptt/django-mptt
  • I haven't had an opportunity to run through your process, but if it's possible, try running the manage.py cms fix-mptt command after each operation
  • in django CMS 3.1, we began using Treebeard instead of MPTT. If you're able to try using 3.1 instead of 3.0.x without too much additional work, that might be a useful experiment.
share|improve this answer
    
Thanks a lot. I'll try that out. – dholbach Oct 29 '15 at 8:52
    
Preliminary testing suggests that manage.py cms fix-mptt indeed helps in this regard. I'll figure out how to embed this into our process until we switch to a newer Django CMS treebeard. Thanks a lot for your help! – dholbach Oct 29 '15 at 9:15
    
This looks very good. I put up a merge proposal for our team to review and test in staging - I'll let you know how it goes. Thanks again! – dholbach Oct 29 '15 at 12:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.