4

I have a requirement to call C function from python, and that C function in turn invokes python methods. Ex: test.py invokes C function in hello.c, and that C function calls method in hello.py. In this case, when I try to load .py module (using PyImport_ImportModule) in C function, causing Segmentation fault (core dumped).
Can you please help debug/root cause this issue? Let me know, if I am missing anything or is there any other way to get this working?

test.py is as follows which uses ctypes and invokes C function.

import time
import ctypes

def _run():
    _hello = ctypes.CDLL('./hello.so')
    result = _hello.hello_main()
    time.sleep(10)

if __name__ == "__main__" :
    _run()

hello.c is as follows, which is loading hello py module.

#include <Python.h>

int hello_main()
{
     PyObject *pModule = NULL;
     PyObject *pFunc   = NULL;

     Py_Initialize();
     printf("Import Py module\n");
     pModule = PyImport_ImportModule("hello");
     if (pModule == NULL) {
         printf("Py module import fail\n");
         return 0;
     }

     pFunc = PyObject_GetAttrString(pModule, "Hello");
     if (pFunc == NULL) {
         printf("Py module get attr fail \n");
         return 0;
     }
     PyEval_CallObject(pFunc, NULL);

     Py_Finalize();

     return 0;
}

hello.py is as follows -

#Filename hello.py
def Hello():
    print "Hello, Py world!"

The C code is compiled as shared lib as below:

[host]# gcc -g -o hello.so hello.c -shared -fPIC -I/usr/include/python2.7 -L/usr/lib/python2.7 -lpython2.7

and i get Segmentation fault, when i run test.py

[host]# python test.py 
Import Py module
Segmentation fault (core dumped)

Following is the backtrace of the core:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b116a0 in threadstate_getframe () from /lib64/libpython2.7.so.1.0
(gdb) bt
#0  0x00007ffff7b116a0 in threadstate_getframe ()
   from /lib64/libpython2.7.so.1.0
#1  0x00007ffff7aef5c9 in PyEval_GetGlobals () from /lib64/libpython2.7.so.1.0
#2  0x00007ffff7b0b222 in PyImport_Import () from /lib64/libpython2.7.so.1.0
#3  0x00007ffff7b0b48a in PyImport_ImportModule ()
   from /lib64/libpython2.7.so.1.0
#4  0x00007fffefdcc980 in hello_main () at hello.c:15
#5  0x00007ffff01dcdcc in ffi_call_unix64 () from /lib64/libffi.so.6
#6  0x00007ffff01dc6f5 in ffi_call () from /lib64/libffi.so.6
#7  0x00007ffff03efc8b in _ctypes_callproc ()
   from /usr/lib64/python2.7/lib-dynload/_ctypes.so
#8  0x00007ffff03e9a85 in PyCFuncPtr_call ()
   from /usr/lib64/python2.7/lib-dynload/_ctypes.so
#9  0x00007ffff7a5d8e3 in PyObject_Call () from /lib64/libpython2.7.so.1.0
#10 0x00007ffff7af2036 in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#11 0x00007ffff7af64bd in PyEval_EvalFrameEx () from /lib64/libpython2.7.so.1.0
#12 0x00007ffff7af8e3d in PyEval_EvalCodeEx () from /lib64/libpython2.7.so.1.0
#13 0x00007ffff7af8f42 in PyEval_EvalCode () from /lib64/libpython2.7.so.1.0
#14 0x00007ffff7b1237f in run_mod () from /lib64/libpython2.7.so.1.0
#15 0x00007ffff7b1353e in PyRun_FileExFlags () from /lib64/libpython2.7.so.1.0
#16 0x00007ffff7b147c9 in PyRun_SimpleFileExFlags ()
   from /lib64/libpython2.7.so.1.0
#17 0x00007ffff7b2591f in Py_Main () from /lib64/libpython2.7.so.1.0
---Type <return> to continue, or q <return> to quit---
#18 0x00007ffff6d4db35 in __libc_start_main () from /lib64/libc.so.6
#19 0x0000000000400721 in _start ()
(gdb) quit

EDIT 1: The reason for the crash is that, there is no thread state in my example case, leading to crash in the call flow below. The tstate is NULL.

PyEval_GetGlobals(void)
 - PyEval_GetFrame();
     - PyThreadState *tstate = PyThreadState_GET();
     - return _PyThreadState_GetFrame(tstate);      ==> Crashing here

As explained in Python PyGILState_{Ensure/Release} causes segfault while returning to C++ from Python code

adding below code after Py_Initialize(), fix the crash problem.

PyGILState_STATE d_gstate;
d_gstate = PyGILState_Ensure();

Thanks for all the help.

7
  • 1
    Have you looked into the dumped core? Which function caused it, in which file, on which line?
    – DYZ
    Apr 29, 2017 at 5:46
  • The CDLL call doesn't hold the GIL. Then to add to this insanity you're trying to reinitialize the interpreter. Why can't you give your C function a ctypes callback function to call back into Python? ctypes will handle the details of reacquiring and releasing the GIL for you.
    – Eryk Sun
    Apr 29, 2017 at 11:57
  • Thanks for reply. The requirement is that - we have some feature code that was written C and python. The C provides the API's for that featue. Then, we have another Python application, where we want use that existing feature by calling C API's (which inturn invokes python module internally). We dont have much flexibility to rewrite/modify the exiting C API code, but just find glue to use the C APIs from python application. Is there way to make this working? Thanks a lot for help. May 1, 2017 at 3:32
  • with gbd, following is backtrace. May 2, 2017 at 2:49
  • It sounds to me like you need to wrap your pre-existing C API in a C extension. Functions in a C extension can be used to convert Python variables to C variables, make calls to the C API and return the value to Python. I've done just this for academia and for my occupation. Read more here.
    – JacaByte
    May 2, 2017 at 3:24

0

Your Answer

Reminder: Answers generated by Artificial Intelligence tools are not allowed on Stack Overflow. Learn more

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.