1

I have a program in C++ that performs mainly matrix multiplcations, additions and so on.

The problem is, a EXC_BAD_ACCESS happens when the calculation performs for about 3 million times.

Is there any possible problems that can arise when a problem is executed for millions of times and for several hours?

Details of the program:

The program is simply calculations on different ranges of values, so it is executing on 6 threads at the same time. There is no resource sharing between the threads.

There seems be no evident problem in the program since:

  1. there is no memory leak, I've confirmed this using Instruments, and the memory size of the program is stable.
  2. the program can execute for at least 2 million times on each thread without any problem, but it is almost guaranteed that the EXC_BAD_ACCESS exception arises some time, on some thread. (the exception happens in my 2 tries of the program (2/2) )

About the matrix multiplication:

Sometimes the size of the matrices is about 2*2 multiply 2*1000.

The elements of the matrix is a custom Complex Number class.

the values of the elements are randomly generated by rand() and converted to float.

the structure is like this:

class Complex
{
private:
    float _real, _imag;
public:
   // getters, setters and overloaded operators
};

class Matrix
{
private:
    Complex **_values;
    int _row,_col;
public:
     getters, setters and overloaded operators
};

Thank you very much!

Any possible reason for the crash is greatly welcomed!

14
  • 2
    EXC_BAD_ACCESS usually implies either a bug in the code or unstable hardware.
    – Mysticial
    Sep 21, 2013 at 23:41
  • 4
    Programs do not age. If a program does not contain any errors it can run indefinitely. A debugger will help you find the bug. Sep 22, 2013 at 0:08
  • 4
    If you're driving your CPUs really hard for several hours and eventually getting crashes, but you're confident that your code is correct, one thing to check is to make sure your computer is not overheating. (sorry if this sounds stupid, but you did ask for any possible reason...)
    – NYCdotNet
    Sep 22, 2013 at 0:08
  • 2
    Ubuntu isn't going to make your program suddenly run more correctly :) Sep 22, 2013 at 0:36
  • 1
    Where are you writing the results to? Is there a chance of thread overwrite there? Are you calling any runtime functions that aren't thread safe? Sep 22, 2013 at 0:40

3 Answers 3

2

EXC_BAD_ACCESS means that you dereferenced a pointer which doesn't point into your process's current memory space. This is a bug in your code. Run it under a debugger until it fails and then have a look at the variable values in the statement where it fails. It could be simple or exceedingly subtle.

2
  • Yes, the process's memory space... it seems to be the solution, as #Graham Perks has also pointed above, the problem seems to be about the threads.
    – Lewen
    Sep 22, 2013 at 1:27
  • The problem does have something to do with memory space, maybe #Graham Perks has pointed it out more clearly, but when I run the program without multithreading, it can loop no matter how many times I want.
    – Lewen
    Sep 22, 2013 at 8:12
1

There's too little information in your post to make a decisive answer. However, it might be that no information available to you now would change it, and you need to debug the case more carefully. Here's what I'd do.

To debug, you want repeatability. But… you say that you're using random numbers. It seems though, that what your program does is some scientific-ish computations. In most cases you don't actually need “true” randomness, but “repeatable” randomness—randomness which passes statistical tests, but where you have enough data to reset the random number generator so that it will produce the exactly the same results as in a previous run. For that, you can just write down the current RNG state (e.g. seed) every time you start a new block of computation.

Now, write some piece of code that will store all the state necessary to restart computations (including RNG) once every few minutes, and run the program. This way, if your code crashes, you will be able to restart the computations with the same exact state and get to the point where it crashed without waiting for millions of iterations. I am putting a strong assumption here, that except for RNG your code does not depend on any other kind of external state (like, network activity, IO, process scheduler making certain choices when scheduling your threads…)

With this kind of data it will be easier to test if the problem is due to a machine fault (overheating, bad memory, etc.). Simply restart the computation with the last state before crashing—preferably after letting the machine cool down, maybe restarting it… if you'll encounter another crash (and it will happen every time you try to restart code), it's quite certain it's due to a bug in your code.

If not, we still cannot say that it's machine fault—your code might (by pure accident/mistake in code) crash due to an undefined behavior which depends on factors out of your control. Examples include using an uninitialized pointer in a rarely-taken code path: it might throw bad access sometimes, and go unnoticed if by pure luck the pointer points to memory you allocated. Try valgrind, this is probably the best tool to check for memory problems… except that it slows down execution so much that you'll again prefer to rerun the computations from a state known to be suspicious (the last state before crash) instead of waiting for millions of iterations. I've seen slowdowns of 5x to 100x.

In the meantime, try running your code on another machine. If you'll also get crashes after a similar number of iterations (to be sure wait for at least 3 times more iterations than it took to crash on the original machine), then it's quite probable that it's a bug in your code.

Happy hacking!

1
  • thank you for the tips! Yeah, I'm doing some scientific-ish computations and using C++ simply because it's much clearer to write OO code and way faster than Matlab. I'll check out valgrind for memory access problems and try to record the RGN state.
    – Lewen
    Sep 22, 2013 at 1:25
0

Calculations with finite precision that fail after a few million iterations? That could be accumulated round-off error. Problem is, those usually exhibit themselves as division by zero or other mathematical errors. EXC_BAD_ACCESS is not. However, there's one case in which this can happen: when you use the mathematical result as an array index.

1
  • Thank you, but the problem is most likely to have something to do with thread and memory overwrite as #Graham Perks and #EJP has pointed above. I have changed the program from 6 threads to 4 threads, it has been running for 11h 54 min now and there is no problem.
    – Lewen
    Sep 23, 2013 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.