9

I'm writing a Python module to perform IO on a O_DIRECT context. One of the limitations of O_DIRECT is you must read into a buffer aligned on a 4096 byte boundary for 2.4 and 2.5 kernels, and 2.6 and up will accept any multiple of 512.

The obvious memory allocation candidate for this is posix_memalign(void **memptr, size_t alignment, size_t size)

In my code, I allocate an area like so:

char *buffer = NULL;

int mem_ret = posix_memalign((void**)&buffer, alignment, size);

if (!buffer) {
    PyErr_NoMemory();
    return NULL;
}

/* I do some stuff here */

free(buffer);

When I compile and import the module with python3.2, this (and the rest of the unshown module) work fine.

When I attempt the same with python2.7 (I'd like to preserve compatibility) it throws the PyErr_NoMemory exception, and mem_ret == ENOMEM, indicating it was unable to allocate.

Why would the version of Python I compile against affect how posix_memalign operates?

OS: Ubuntu 12.04 LTS

Compiler: Clang + GCC Show same behaviour

UPDATE

I now have a working piece of code, thanks to user694733
However the fact that it works has me even more confused:

#if PY_MAJOR_VERSION >= 3
char *buffer = NULL;

int mem_ret = posix_memalign((void**)&buffer, alignment, count);
#else
void *mem = NULL;

int mem_ret = posix_memalign(&mem, alignment, count);

char *buffer = (char*)mem;
#endif

Can anyone explain why the incorrect first block works under Python3, but not 2.7, and more importantly why the correct second block does not work under Python3?

UPDATE 2

The plot thickens, having settled on the correct form of the code below, I tested on 4 different version of Python.

void *mem = NULL;

int mem_ret = posix_memalign(&mem, alignment, count);

char *buffer = (char*)mem;

if (!buffer) {
    PyErr_NoMemory();
    return NULL;
}

/* Do stuff with buffer */

free(buffer);

Under Python 2.7: This code operates as expected.
Under Python 3.1: This code operates as expected.
Under Python 3.2: This code generates mem_ret == ENOMEM and returns NULL for buffer
Under Python 3.3: This code operates as expected.

The Python versions not included in the Ubuntu repositories were installed from the PPA at https://launchpad.net/~fkrull/+archive/deadsnakes

If the version tagged Python binaries are to be believed, the versions I have installed are:

python2.7 
python3.1
python3.2mu (--with-pymalloc --with-wide-unicode)
python3.3m (--with-pymalloc)

Could the use of the wide-unicode flag in the default Python3 distribution be causing this error? If so, how is this happening?

For clarity, the ENOMEM failure to allocate will occur with any variant of malloc(), even something as simple as malloc(512).

8
  • 3
    Note that casting char** to void** is not valid. Pointers might have different sizes. Use void *buffer = NULL;, and later you can assign it to char* if you need byte access.
    – user694733
    Commented Jan 30, 2014 at 15:11
  • Have you tried printing size and alignment to see if they make sense? Perhaps they're changed as a side-effect of some UB elsewhere in the code.
    – Fred Foo
    Commented Jan 30, 2014 at 15:29
  • Even if I hardcode those params and call a literal posix_memalign(buffer, 512, 512) it still fails to allocate under 2.7
    – bradfier
    Commented Jan 30, 2014 at 15:44
  • Python 3.3 is AFAIK always built as if you had defined --with-wide-unicode.
    – filmor
    Commented Jan 31, 2014 at 10:40
  • @filmor If we look at the ancient history that is: docs.python.org/2.3/whatsnew/section-pymalloc.html In particular, In 2.3, pymalloc has had further enhancements and is now enabled by default This would indicate that PyMalloc should be built into 2.7 and 3.1, yet they lack the m tag, is this an inconsistency or are those versions genuinely built without it? The 2.7 on my machine is the version that ships with Ubuntu LTS.
    – bradfier
    Commented Jan 31, 2014 at 11:04

2 Answers 2

1

For a quick work-around, stick to mmap instead of malloc+memalign

0

posix_memalign may not be the same body of code in one compilation environment as another. You could easily imagine that Python 3 would use different feature test macros to Python 2. That could mean it ends up running different code.

You might have a look at the symbols that are used... often times the output of ldd or nm will have mangled names that indicate what version is actually being used.

Additionally, what does an strace show of the allocation system call? I find that's a good way of seeing if the arguments passed in are incorrect, which can be a reason for getting ENOMEM.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.