EDIT: New example that explode without changing the ulimit, and it does not matter what is in the c part.
I am a python user (I learned from an edX course), and I have been working with ctypes and c (but I dont really know c). I have a "Segmentation fault: 11" problem with my code when I run it with some specific parameters (basically, a big array size), here is a small example that replicates what my code do:
test1.py:
import numpy as np
from numpy.ctypeslib import ndpointer
import ctypes as cy
Lib_Path = './lib.so'
class Simulacion:
def __init__(self, ss,tm):
self.tm = tm;
self.ss = ss;
def ejecutar(self):
self.data = np.empty((int(self.ss), self.tm), dtype = float)
lib = cy.CDLL(Lib_Path)
dblc = cy.c_double; pntrc = ndpointer(dblc); intc = cy.c_long
lib.trisolve.argtypes = [intc, pntrc, intc]
lib.trisolve(self.tm, self.data, self.ss)
return self.data
ss = 10
tm = int(1e6);
sim = Simulacion(ss,tm)
data = sim.ejecutar()
test1.c
void trisolve(int tm, double* data, int ss){
}
makefile
SRC=test1
GCC=gcc-6
all:
$(GCC) -fPIC -fopenmp -lm -c -O3 $(SRC).c
$(GCC) -shared -lgomp -o lib.so $(SRC).o
clean:
rm lib.so
rm $(SRC).o
This code explode without changing the ulimit.
for my real code, i am using "ulimit -s 65532" which is the maximum stack size in my mac. This limit the size of the arrays that I am using, and currently I need to duplicate the size of it. For what I have found, the problem is that the arrays are being stored in the stack instead of the heap, so I have this hard limit due to the SO. So my question is, how I can pass that big array to C, store it in the heap and then bring it back to python ?
I mostly use python, and the c part of the code I did it without a good formation in this lenguaje, so "stacks", "heap" and probably "malloc" are new terms for me.
Thanks!
intandlong, let alone the more complex parts of working with NumPy arrays in C.omp parallel forwould cause that problem either. But CPython itself needs at least 32K of stack.while, inside it a fewforto set the terms of the matrix, then the solution of it, then some modification of the result, and then the iteration again. @eryksun if is not using the stack, why when I useulimit -s 16, I got a segmentation fault, but if I set it to, for example, 65532, the code works without a problem?. theomp parallel foris not the problem here, the program doesn't even start running the c code. (if I putprintthere, the segmentation fault occurs before it).