15

I'd like to save my files using the primary key of the entry.

Here is my code:

def get_nzb_filename(instance, filename):
    if not instance.pk:
        instance.save() # Does not work.
    name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
    name_slug = re.sub('[-]+', '-', name_slug)
    return u'files/%s_%s.nzb' % (instance.pk, name_slug)

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename)
    name = models.CharField(max_length=256)

I know the first time an object is saved the primary key isn't available, so I'm willing to take the extra hit to save the object just to get the primary key, and then continue on.

The above code doesn't work. It throws the following error:

maximum recursion depth exceeded while calling a Python object

I'm assuming this is an infinite loop. Calling the save method would call the get_nzb_filename method, which would again call the save method, and so on.

I'm using the latest version of the Django trunk.

How can I get the primary key so I can use it to save my uploaded files?


Update @muhuk:

I like your solution. Can you help me implement it? I've updated my code to the following and the error is 'File' object has no attribute 'create'. Perhaps I'm using what you've written out of context?

def create_with_pk(self):
    instance = self.create()
    instance.save()
    return instance

def get_nzb_filename(instance, filename):
    if not instance.pk:
        create_with_pk(instance)
    name_slug = re.sub('[^a-zA-Z0-9]', '-', instance.name).strip('-').lower()
    name_slug = re.sub('[-]+', '-', name_slug)
    return u'files/%s_%s.nzb' % (instance.pk, name_slug)

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename, blank=True, null=True)
    name = models.CharField(max_length=256)

Instead of enforcing the required field in my model I'll do it in my Form class. No problem.

8 Answers 8

4

It seems you'll need to pre-generate your File models with empty file fields first. Then pick up one and save it with the given file object.

You can have a custom manager method like this;

def create_with_pk(self):
    instance = self.create()
    instance.save()     # probably this line is unneeded
    return instance

But this will be troublesome if either of your fields is required. Because you are initially creating a null object, you can't enforce required fields on the model level.

EDIT

create_with_pk is supposed to be a custom manager method, in your code it is just a regular method. Hence self is meaningless. It is all properly documented with examples.

2
  • 1
    No need to instance.save() after instance = self.create().
    – claymation
    May 1, 2013 at 18:36
  • I was unable to extrapolate whatever extra steps are involved in this solution. I created a custom manager, defined the method mentioned above, but ended up with a double save, where the first save created an empty object and the second failed to modify the correct object. I think that anyone trying to use this method will also have to modify their save function. Ultimately switched to @Giles solution below.
    – seanmus
    May 27, 2016 at 21:03
4

You can do this by setting upload_to to a temporary location and by creating a custom save method.

The save method should call super first, to generate the primary key (this will save the file to the temporary location). Then you can rename the file using the primary key and move it to it's proper location. Call super one more time to save the changes and you are good to go! This worked well for me when I came across this exact issue.

For example:

class File( models.Model ):
    nzb = models.FileField( upload_to='temp' )

    def save( self, *args, **kwargs ):
        # Call save first, to create a primary key
        super( File, self ).save( *args, **kwargs )

        nzb = self.nzb
        if nzb:
            # Create new filename, using primary key and file extension
            oldfile = self.nzb.name
            dot = oldfile.rfind( '.' )
            newfile = str( self.pk ) + oldfile[dot:]

            # Create new file and remove old one
            if newfile != oldfile:
                self.nzb.storage.delete( newfile )
                self.nzb.storage.save( newfile, nzb )
                self.nzb.name = newfile 
                self.nzb.close()
                self.nzb.storage.delete( oldfile )

        # Save again to keep changes
        super( File, self ).save( *args, **kwargs )
2
  • I was able to modify this so that I could have a primary key as part of the path to files served. Thanks.
    – seanmus
    May 27, 2016 at 20:59
  • @seanmus Did it work out well? I also want to use the pk for the generation of paths, but there is some serious yak shaving necessary (relative path concatenations after editing, problems with the exchange of files with similar names, deleting, etc.). Do you have a link to an elegant and robust example?
    – armin
    Aug 29, 2018 at 17:05
2

Context

Had the same issue. Solved it attributing an id to the current object by saving the object first.

Method

  1. create a custom upload_to function
  2. detect if object has pk
  3. if not, save instance first, retrieve the pk and assign it to the object
  4. generate your path with that

Sample working code :

class Image(models.Model):
    def upload_path(self, filename):
        if not self.pk:
            i = Image.objects.create()
            self.id = self.pk = i.id
        return "my/path/%s" % str(self.id)
    file = models.ImageField(upload_to=upload_path)
2
  • 2
    No need to i.save() after i = Image.objects.create().
    – claymation
    May 1, 2013 at 18:32
  • 1
    This doesn't work, as the new object that is created stays empty, and the object that you are saving is saved as a different object.
    – Giles
    May 15, 2013 at 18:39
2

You can create pre_save and post_save signals. Actual file saving will be in post_save, when pk is already created. Do not forget to include signals in app.py so they work. Here is an example:

_UNSAVED_FILE_FIELD = 'unsaved_file'


@receiver(pre_save, sender=File)
def skip_saving_file_field(sender, instance: File, **kwargs):
    if not instance.pk and not hasattr(instance, _UNSAVED_FILE_FIELD):
        setattr(instance, _UNSAVED_FILE_FIELD, instance.image)
        instance.nzb = None


@receiver(post_save, sender=File)
def save_file_field(sender, instance: Icon, created, **kwargs):
    if created and hasattr(instance, _UNSAVED_FILE_FIELD):
        instance.nzb = getattr(instance, _UNSAVED_FILE_FIELD)
        instance.save()
1
+500

Here are 2 possible solutions:

Retrieve id before inserting a row

For simplicity I use postgresql db, although it is possible to adjust implementation for your db backend.

By default django creates id as bigserial (or serial depending on DEFAULT_AUTO_FIELD). For example, this model:

class File(models.Model):
    nzb = models.FileField(upload_to=get_nzb_filename)
    name = models.CharField(max_length=256)

Produces the following DDL:

CREATE TABLE "example_file" ("id" bigserial NOT NULL PRIMARY KEY, "nzb" varchar(100) NOT NULL, "name" varchar(256) NOT NULL);

There is no explicit sequence specification. By default bigserial creates sequence name in the form of tablename_colname_seq (example_file_id_seq in our case)

The solution is to retrieve this id using nextval :

def get_nextval(model, using=None):
    seq_name = f"{model._meta.db_table}_id_seq"
    if using is None:
        using = "default"
    with connections[using].cursor() as cursor:
        cursor.execute("select nextval(%s)", [seq_name])
        return cursor.fetchone()[0]

And set it before saving the model:

class File(models.Model):
    # fields definition

    def save(
        self, force_insert=False, force_update=False, using=None, update_fields=None
    ):
        if not self.pk:
            self.pk = get_nextval(self, using=using)
            force_insert = True
        super().save(
            force_insert=force_insert,
            force_update=force_update,
            using=using,
            update_fields=update_fields,
        )

Note that we rely on force_insert behavior, so make sure to read documentation and cover your code with tests:

from django.core.files.uploadedfile import SimpleUploadedFile
from django.forms import ModelForm
from django.test import TestCase

from example import models


class FileForm(ModelForm):
    class Meta:
        model = models.File
        fields = (
            "nzb",
            "name",
        )


class FileTest(TestCase):
    def test(self):
        form = FileForm(
            {
                "name": "picture",
            },
            {
                "nzb": SimpleUploadedFile("filename", b"content"),
            },
        )
        self.assertTrue(form.is_valid())
        form.save()

        self.assertEqual(models.File.objects.count(), 1)
        f = models.File.objects.first()
        self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")

Insert without nzt then update with actual nzt value

The idea is self-explanatory - we basically pop nzt on the object creation and save object again after we know id:

def save(
    self, force_insert=False, force_update=False, using=None, update_fields=None
):
    nzb = None
    if not self.pk:
        nzb = self.nzb
        self.nzb = None

    super().save(
        force_insert=force_insert,
        force_update=force_update,
        using=using,
        update_fields=update_fields,
    )

    if nzb:
        self.nzb = nzb
        super().save(
            force_insert=False,
            force_update=True,
            using=using,
            update_fields=["nzb"],
        )

Test is updated to check actual queries:

def test(self):
    form = FileForm(
        {
            "name": "picture",
        },
        {
            "nzb": SimpleUploadedFile("filename", b"content"),
        },
    )
    self.assertTrue(form.is_valid())
    with CaptureQueriesContext(connection) as ctx:
        form.save()

    self.assertEqual(models.File.objects.count(), 1)
    f = models.File.objects.first()
    self.assertRegexpMatches(f.nzb.name, rf"files/{f.pk}_picture(.*)\.nzb")

    self.assertEqual(len(ctx.captured_queries), 2)
    insert, update = ctx.captured_queries
    self.assertEqual(
        insert["sql"],
        '''INSERT INTO "example_file" ("nzb", "name") VALUES ('', 'picture') RETURNING "example_file"."id"''',
    )
    self.assertRegexpMatches(
        update["sql"],
        rf"""UPDATE "example_file" SET "nzb" = 'files/{f.pk}_picture(.*)\.nzb' WHERE "example_file"."id" = {f.pk}""",
    )
1
  • Thank you. The get_nextval() method is cool. This works, I use PostgreSQL, too.
    – guettli
    Jan 28, 2022 at 16:02
0

Ty, is there a reason you rolled your own slugify filter?

Django ships with a built-in slugify filter, you can use it like so:

from django.template.defaultfilters import slugify

slug = slugify(some_string)

Not sure if you were aware it was available to use...

2
  • I think your answer is correct, but it does not fit to the question. Or am I missing something?
    – guettli
    Jan 27, 2022 at 8:23
  • This is verrrry old but you're correct, it doesn't answer the question but instead points out that the author probably didn't need to created their own slugify method Jan 29, 2022 at 1:19
0

You can use the next available primary key ID:

class Document(models.Model):
    def upload_path(self, filename):
        if not self.pk:
            document_next_id = Document.objects.order_by('-id').first().id + 1
            self.id = self.pk = document_next_id
        return "my/path/document-%s" % str(self.pk)
    document = models.FileField(upload_to=upload_path)

Details

My example is a modification of @vinyll's answer, however, the problem Giles mentioned in his comment (two objects being created) is resolved here.

I am aware that my answer is not perfect, and there can be issues with the "next available ID", e.g., when more users will attempt to submit many forms at once. Giles's answer is more robust, mine is simpler (no need to generate temp files, then moving files, and deleting them). For simpler applications, this will be enough.

Also credits to Tjorriemorrie for the clear example on how to get the next available ID of an object.

2
  • Getting the next id like this will fail, if there are multiple uploads at the same time.
    – guettli
    Jan 25, 2022 at 7:50
  • Yes, I am aware of that, and that is what I wrote in my answer as a disclaimer. In the case of my program there will be only one upload at a time (single account/user), so getting the next id like that will be sufficient. Jan 25, 2022 at 15:36
0

Well I'm not sure of my answer but -

use nested models, if you can -

class File(models.Model):
    name = models.CharField(max_length=256)

class FileName(models.Model):
    def get_nzb_filename(instance, filename):
        return instance.name
    name = models.ForeignKey(File)
    nzb = models.FileField(upload_to=get_nzb_filename)

And in create method -

File_name = validated_data.pop(file_name_data)
file = File.objects.create(validated_data)
F = FileName.objects.create(name=file, **File_name)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.