6

There are printfs, asserts, edit and continue, logging frameworks? Whats your favorite poison?

4

20 Answers 20

20

ASSERTs ASSERTs ASSERTs.

I have 300000 loc (not counting comments) of highly factored and reused code in my personal libraries of which about 15%(a guess) is templates and 50000 loc is test code.

If an idiom is duplicated then it is made a function/method. Personally I view the ease of cut and paste as the invention of the DEVIL purposely put there to bloat code and propagate defects.

About 4% of the library is ASSERTS and debug code (very few printfs and almost all of the output is queued output to a cout stream custom low priority task because screen IO is just so expensive and thus timing changing). Maybe 50% of the asserts are there to guarantee class invariants and post conditions on method execution.

I refactor mecilessly when I revisit a piece of code that I may have rushed or maybe just made a mistake in the interface/object coupling design, say the subject object of a method really belongs as an object object and the method belonged in one of the originally object objects (parameter objects). Being liberal with asserts seems to protect me from some dumb mistakes if I makea substantial refactoring. This doesnt happen much but there are times.

I have a DEBUGGING macro that acts much like ASSERT so I can have code surrounded by

DEBUGGING(... code....);

and it is not compiled in non debug builds.

I do not use the vendor supplied assert. My asserts DO NOT abort & core dump they merely throw up a message box and call the debugger. If it is new code and the method is a const method, being able to return to the method and then re-execute it(the method) with the same set of parameters is pretty useful. Sometimes even the fact that some data is changed is irrelevant to the problem and one can re-invoke with knowledge gain.

I absolutely HATE command line debuggers. It is like going back 25 years in time - might as well be using a teletype and a 2400 baud line. I need and want a full blown IDE where one can right click on a data structure and open it up, close it, chase pointers execute methods etc etc etc.

I step through every line of my new code and examine every (one of my) variables for expected behaviour. The use of an IDE that highlights changes is invaluable here. To do that with GDB one must be a concert pianist with the memory of Carnac the Magnificent ;-).

For new development I also try to capture stream data/message data when an abnormal condition is encountered. This is especially useful for udp servers and frequently gives a head start on reproducibility.

I also like to have simulators that can "surround the application and drive it and consume from it and do verification. (cojoined/coupled source/sink simulators) Almost all of my code is headless or at least the human interaction is irrelavant to functionality so "surrounding" the app is frequently possible. I find it very important to have good support and management that understands that creation of the test data is very important and the the test data collection is what builds up into a suite of tests that can evolve into a comprehensive regression/smoke test.

I also used to like to set the OS scheduling quantum way down With multithreaded apps such short quanta can more easily bring out threading bugs. I especially like to drive thread safe object methods with many threads - tens if not hundreds. In general a threadsafe object cannot be tested in situ if the application is human driven - just impposible to drive it. So there is real need for custom test drivers that are at a much lower (component oriented) level. And it is in these test that the asserts can let you know if something broke. Obviously doesn't prove the code is right but does give some confidence.

It is also true that these preferences probably reflect the more class library/reusability oriented views and roles that I have had. When you write library code there are usually few "production" problems as the library is by definition heavily used and heavily tested. Logging and that sort of history seems to be more app oriented than library oriented.

2
  • 3
    wow, good, i like that answer
    – knittl
    Aug 25, 2009 at 9:18
  • 1
    Agree everywhere (especially the custom assert) except with the comment about gdb; ease of use is an acquired attribute, not an inherited one. While a visual interface provides for a greater span of data display it does not inherently provide greater utility.
    – ezpz
    Aug 25, 2009 at 11:00
13

Multi-level logging systems. I find use for at least 5 levels:

  • verbose: stuff you only want to see when debugging low-level stuff like protocol errors; may be compiled out of release binaries, so you can put stuff in this level that you don't want your end users to find

  • internal: lower-level tracing than normal level, which is frequently useful, but not so often that you want to see it all the time

  • normal: default output level, for stuff useful to anyone watching the system run normally

  • problem: run-time errors that the program knows how to cope with; you might choose to run at this level for release versions, instead of normal

  • fatal error: "scream and die" type messages, like out-of-memory

Multi-level logging lets you build plenty of logging info into the program without having to see it all the time. You turn up the log level only when you have to debug something, then put it back to its normal level. When doing "printf" type debugging -- temporary messages -- I put them in at normal level, so I don't have to crank up the log level to see them, and have them be obscured by the noisy internal or verbose level messages.

1
  • 2
    I like a separate debug level for temporary messages. The debug level is always displayed. This avoids the "oh I don't know if it's going to show up or not, whatever I'll just use printf for this message" train of thought. It also serves as a clear reminder to delete the printout when I've finished debugging. Aug 25, 2009 at 4:04
8

In general, printf's. They allow for easy program slicing and don't require any tools other than an editor and compiler.

4
  • 9
    Exactly. "When in doubt, print more out." Aug 25, 2009 at 3:15
  • 3
    They will also help you ignore other things that are going wrong. IMHO debugging using print is better suited for languages like PHP or Python ....using printf to 'debug' C++ applications is like using binoculars to do surgery.
    – jscharf
    Aug 25, 2009 at 4:24
  • @jscharf. I couldn't agree more! conditional breakpoints and the ability to see the values in variables at all times is crucial.
    – rfcoder89
    Jan 28, 2013 at 14:43
  • I'm not opposed to debuggers, but I generally use them once I've narrowed down the problem to a particular area. Print statements can give you a good view of what your program is doing while still letting your program behave normally.
    – jason0x43
    Jan 28, 2013 at 20:21
6

Reactive measures:

  • Using a good debugger, especially of the integrated variety. Use a debugger that works visually with your code.

  • Debugging tactics. Familiarity with your tools. Knowledge of common pitfalls. The process of deduction. Using hypothesis and branching on different ideas.. returning to previous ideas when needed. Tracking your debugging progress. The scientific method.

  • Analytical tools (e.g. Dependancy Walker, .NET Reflector), memory dumps, stack traces and logs.

Preventative measures:

  • Small incremental builds. During active development, add to your project piece-by-piece and test each incoming piece. Don't spew out a bunch of code and then mash Run and blithely correct each error. Add to your program piece by piece and take the time to fix everything. Try to make each increment in your project 'atomic'.

  • Multi-level logging, as others have already pointed out. Have a severity scale running from trace up to fatal error and everything in between. Your production app should log every single thing that it does and why it succeeded or failed, but you should also be able to alter the logging level when an extreme level of detail is not needed. Logs should be human readable as a first priority.

  • Fail-safe logging. Don't depend on your logging mechanism to survive exceptional situations. Always have a back up plan - the event log, a flat text file, a last ditch email - for when things go bottoms up.

  • Good source control policy. Frequent check-ins on a regular schedule at minimum, plus check-ins for any and all groups of changes, particularly widespread or possibly breaking changes.

5

I like gdb -- I normally use it in commandline mode, but, if you can't stand that, there are GUI frontends to it like Insight, Ddd, etc. Logs always help, too, and so do core dump files on which you can perform "postmortem debugging" with these same tools.

3

I use the gdb command commands to create "printf" statements on the fly. Just make sure the last command is continue.

#assume two breakpoints, 1 and 2
commands 1
  silent
  echo calling baz\n
  set $print_foobar=1
  continue
end

commands 2
silent
  echo calling foobar\n
  if $print_foobar
    set $print_foobar=0
    backtrace
  end
  continue
end

I've recently fallen in love with this technique as it lets me create printf() statement for code that is already running. Plus GDB scripting though limited lets you do quite a bit when deciding what to print. I've yet to find a situation where a commands section wasn't as good or better than a printf().

2

I vote for gdb. Especially when all you have to work with is a core file.

printf's are nice when you have a general idea where the bug is. But when the test team comes back to you w/ a core file and a weird description of the problem, to be able to analyze the core dump will give you a tremendous head start in your debugging effort.

2

Doing multi-threading stuff, I couldn't live without logging. For logging in C++, I use the templog library. It allows several severities (how bad is it?) times multiple log targets (who might be interested in it?) combine with as many filtering log sinks (where to write it to?) as you want, so you don't get drowned in noise. It goes a long way towards efficiency, employing template-meta stuff to help the compiler eliminating superfluous code while not falling into the assert(do_something_important()) trap.

Also, it's small enough (I think it's below 1kLoC spread over half a dozen headers) and comes with the permissive boost license so that you don't depend on the creators to not to let the ball drop. If they do -- you can just maintain it yourself.

I only wish the guys would finally turn that branch the current code resides on into the new trunk.

1

Tracing through, modifying variables in memory to hit some obscure branch of code. Rarely edit and continue, for some reason I can't quite trust it to keep sane state, so full runs after a change.

When impossible to trace (gdb on windows is way too slow for example, hitting a breakpoint takes 30 seconds each time) then printf. Junk code and throws off timing in case of multithreading bugs, but sometimes it is the only way.

Disassembler debuggers when having to debug release builds without debug information (Olydbg is nice when it works).

Proper logging is nice when it is available, takes effort to setup and use, but very valuable when needed.

Sending stacktraces of crashes home is even nicer.

Asserts spreaded around where needed.

Minidumps for crashes on users machines. (Reproducible crashes are the best. If bugs can't be relied to show up on time then what can?)

1

For this one, I'm going to have to go with deliberately structuring the project to minimize incremental build/iteration times. As a side effect, these are the same steps it takes to keep edit-and-continue working properly.

1

Assertions. When I'm writing code, if I spot any edge cases which I think might indicate trouble, I can drop in a line which will let me know in debug builds if such a thing is happening. For bonus points they have no performance effect on release builds so you can check semi-expensive things like eg. assert(someMap.find(someKey) != someMap.end());

Obviously they're not a replacement for checking real error conditions which you have to catch, but they're great for those fringe cases where you think "maybe I should check that" while you're writing the code.

1
  • If you think 'maybe I should check that' at any point in the development process building without that check (via release compilation or other) is opening up a door for trouble down the line.
    – ezpz
    Aug 25, 2009 at 11:05
1

Next to logging, a nice technique that has helped us is dumping all useful variables in a readable way on request. This way, you can narrow down possible causes.

1

IMO one of the most powerful ways of debugging server code is Differential Debugging, that is comparing two log files.

It is especially useful with legacy code when the current developers don't know all areas of the code. It helps narrow the search area and the code that has to be analysed. It does require good logging/tracing to be present already.

This works when you have a use case that succeeds and one that fails. For example feature X used to work in version 3 of your product but now is broken in version 4, and no one knows why.

Compare the logs, use awk or sed to strip out the redundant differences (these scripts can be reused):

  • Time stamps
  • Thread IDs - sometimes filter on a particular thread
  • paths
  • etc

When you see the logs diverge, this usually indicates a wrong decisions was made just before.

I used this technique to replace a proprietary middleware solution in a legacy system with CORBA and ensure that I did not change the behaviour of the application logic. But it is very useful in lots of situations.

0

Writing carefully so I minimize bug creation in the first place.

When I do have bugs, I prefer IDEs and stepping through code.

Printfs are an extremely powerful tool, but still the weapon of last resort.

0

If you're using the Visual Studio Debugger (which I suspect you are since you mentioned 'edit and continue'),you can make good use of the "Immediate Window". You can bring it up quickly with the keyboard shortcut Ctrl+Alt+I

The 'Immediate Window' comes quite close to being a "Read-Eval-Print" loop, which is common in dynamic languages, but all but absent in C++ and similar languages. The immediate window will allow you to evaluate simple expressions, which you could also do in the watch window, but also lets you run simple statements, which the watch window isn't as good for.

If there's some hypothesis you want to explore while debugging, performing experiments in the immediate window can often get you to the answers quickly. You don't have to know ahead of time what you need to print, you'll have all the state information you need available to you while you're in the debugger. And you can change the state of your program by executing simple statements in order to test your hypothesis, something you couldn't do with simple print statements.

Debugging this way mimics the incremental style of programming popular in languages that offer a REPL out of the box (like say Python or Ruby). It can be quite helpful in your debugging.

0

If my program is too big I logically separate problem part and debug it independently. It gives an opportunity for code to be more modular also

0

Emacs gud-gdb mode is an army swiss knife!

I no longer stand gdb consol mode, well I know who does, but in my case using a GUI debugger whenever it's possible is just worth it.

IDA PRO is another thing, I might use it too from time to time when it comes to (RE)

0

Let's not forget valgrind ... it's saved my bacon on more than one occasion by helping find that obscure once-in-100-runs memory corruption or uninitialized-value error.

0

all kinds of tracing

-1

When writing code, consider how you're going to be putting breakpoints in there later. This particularly means not overdoing nested function calls - foo(bar(baz)) - and the same for field/method chains - foo().bar().baz(). In general, unless expressions are trivial, it is often worth it to put them on separate lines and assign them to variables, even if the values are only used once - you can then easily step through each, set breakpoint precisely where you want, and you'll have the values around in the watch window. When compiling with optimizations, any such variables will likely be optimized away, especially if you use the const reference trick rather than relying on RVO to kick in.