3

I'm trying to malloc a huge 4d array (192 gig available); but what goes in does not match what comes out (see assert() in code below). (I made the defined sizes small; but the real numbers are: 20,9000,195,120)

#define SIZE_A 1
#define SIZE_B 3
#define SIZE_C 4
#define SIZE_D 2

#define offSet(a,b,c,d) ( ((size_t) SIZE_A * SIZE_B * SIZE_C * a) + ((size_t) SIZE_B * SIZE_C * b) + ((size_t) SIZE_C * c) + d)

void xall(void)
{
int *aray = (int *) malloc( (size_t) SIZE_A * SIZE_B * SIZE_C *  SIZE_D  * sizeof(int));


int counter = 0;

    for (int a = 0; a < SIZE_A; ++a){
        for (int b = 0; b < SIZE_B; ++b){
            for (int c = 0; c < SIZE_C; ++c) {
                for (int d = 0; d < SIZE_D; ++d){
                    aray[ offSet(a,b,c,d) ] = counter++;

                }}}}


counter = 0;    
    for (int a = 0; a < SIZE_A; ++a){
        for (int b = 0; b < SIZE_B; ++b){
            for (int c = 0; c < SIZE_C; ++c) {
                for (int d = 0; d < SIZE_D; ++d){       
                    int value = aray[ offSet(a,b,c,d) ] ;
                    assert(value == counter++);

                }}}}
}

4 Answers 4

5

Your macro is wrong, as others have mentioned. Fixing that is fine, but I'd suggest instead that you allocate a multi-dimensional array directly rather than hand-rolling the offset macro. Check out this example:

#include <assert.h>
#include <stdlib.h>

#define SIZE_A 1
#define SIZE_B 3
#define SIZE_C 4
#define SIZE_D 2

int main(void)
{
  int counter = 0;
  int (*array)[SIZE_A][SIZE_B][SIZE_C][SIZE_D] = 
          malloc(sizeof(int) * SIZE_A * SIZE_B * SIZE_C *  SIZE_D);

  for (int a = 0; a < SIZE_A; ++a)
    for (int b = 0; b < SIZE_B; ++b)
      for (int c = 0; c < SIZE_C; ++c) 
        for (int d = 0; d < SIZE_D; ++d)
          (*array)[a][b][c][d] = counter++;

  counter = 0;    
  for (int a = 0; a < SIZE_A; ++a)
    for (int b = 0; b < SIZE_B; ++b)
      for (int c = 0; c < SIZE_C; ++c)
        for (int d = 0; d < SIZE_D; ++d)
        {
          int value = (*array)[a][b][c][d];
          assert(value == counter++);
        }

  return 0;
}

The in-memory layout of the array allocated in this example is exactly the same as it is in your question, but why not let the compiler do the work for you?

On a bit of an editorial aside - don't cast the return value of the malloc() call in a C program. The conversion from void * is implicit, and having the explicit cast can hide implicit function declaration warnings that you would get otherwise - for example if you forgot to include stdlib.h.

8
  • Good point, but the problem is the dimensions are too large for Visual c: The total size of the array is more than 4 gigabytes, and I get the C2148 error about the array size too large. Feb 10, 2012 at 22:58
  • But you can allocate that much and assign it to a simple pointer? I don't get it.
    – Carl Norum
    Feb 10, 2012 at 23:01
  • If your array is larger than 4 gb you should put more memory into your comp and use 64-bit OS. Feb 10, 2012 at 23:03
  • The C2148 error says that the total size of an array must not exceed 2^31-1 bytes. But you can still malloc more than that. So it's just a compiler limitation.
    – Mysticial
    Feb 10, 2012 at 23:04
  • @mikithskegg: I'm using a 64bit OS, and my workstation is maxed out at 192 gig (that's right, gigabytes, not megabytes). Any other suggestions? Feb 10, 2012 at 23:17
1

Your macro isn't quite correct. Change it to this:

#define offSet(a,b,c,d) ( ((size_t) SIZE_B * SIZE_C * SIZE_D * a) + ((size_t) SIZE_C * SIZE_D * b) + ((size_t) SIZE_D * c) + d)

You basically had your SIZE_X shifted over by 1 letter.

I also suggest putting () around your macro parameters:

#define offSet(a,b,c,d) ( ((size_t) SIZE_B * SIZE_C * SIZE_D * (a)) + ((size_t) SIZE_C * SIZE_D * (b)) + ((size_t) SIZE_D * (c)) + (d) )
0

I seem the macros should be defined this way:

#define offSet(a,b,c,d) ( ((size_t)  SIZE_B * SIZE_C *SIZE_D * a) + ((size_t)  SIZE_C *SIZE_D * b) + ((size_t) SSIZE_D * c) + d)
0

In some OS, there is a limit of memory allocation per process. Usually the default may be changed.

Others, don't even allocate real memory when calling malloc, like Linux for example, and you may fail later on when trying to write to it, but I guess this is not your case.

Try checking if you can control the maximum allocates memory per process on your system. Hope you find an answer, it is indeed interesting.

Some reading stuff:

http://duartes.org/gustavo/blog/post/anatomy-of-a-program-in-memory http://www.linux-mag.com/id/827/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.