17

My boss thinks that any code we write ( in C/C++) has to comply to the standards specified by a static analysis tool(like MISRA/Lint). My take on this is since compilers are well developed today, is this really required ?

Question here is how effective is this static analysis these days?

11
  • 23
    Compilers detect a different type of errors.
    – this
    Apr 15, 2014 at 10:27
  • 2
    the first step is to compile with the highest level of warning and remove the warning: you can remove all the unused variables, cast that cause a potential loss of data and other. this does not require a new tool and will remove the most obvious one
    – Gabriel
    Apr 15, 2014 at 10:31
  • 2
    I don't think it's practical to adhere stricly to these standards. While MISRA has many good rules, some of the rules are retarded, and require unnecessary boilerplate code to get around.
    – user694733
    Apr 15, 2014 at 10:45
  • 3
    Your first question is somewhat bizarre. It is: "my boss told me I have to do X. I don't think I have to do X. Do I have to do X?" If there is some lack of clarity about the policies of your organization, asking strangers on the internet seems unlikely to help. If you want to know what is required to do your job in accordance with company policies, ask your boss, not us. The answer to your second question is: yes, static analysis is extremely effective. Full disclosure: I design static analyzers. Apr 15, 2014 at 17:07
  • 2
    If you don't want to take my word for it, use science to answer your second question. Get ten million lines of code, fix all the compiler warnings, and then run it through a number of static analyzers. Review all the defects produced. If there was a static analyzer that produced a true positive defect that your compiler did not then it was more effective at finding defects than your compiler was. Our experiments show that in a typical codebase you can expect to find about one serious defect per thousand lines of code scanned. Apr 15, 2014 at 17:11

6 Answers 6

37

Short answer: Yes.

Long answer: Compilers are indeed getting better at analysing certain kinds of "mistakes", but the "depth" that they work to is typically a lot less than proper tools for this. In particular tools that work across compile units, such as Coverity, which can understand (for example) that a function may return a pointer as NULL, and if it does, your code will crash because you don't check for NULL before accessing the pointer.

Static analysis tools can also do checking for lock usage, which the compiler typically can't.

As to "how effective" it is really does depend on both which tool you are using, what settings, and how well you otherwise test the code. So, "code coverage" comes into it too. Do you go through every branch of your code in your testing? With every possible value that causes a difference in behaviour? Static analysis tools can detect errors that your testing may not cover.

(Obviously, whether it actually makes sense in your particular business is a completely different discussion - that's for your boss, and his/her bosses to decide)

9
  • Just wondering, you haven't by any chance skipped testing Clang's ccc-analyzer before writing up this answer? Here the combination of the knowledge in the compiler backend and static analysis seems to be a lot more powerful than the usual old lint program. I've noticed that some other static analysis tools that are not free of charge also make extensive use of Clang and LLVM facilities. Apr 15, 2014 at 10:38
  • 2
    I have not, personally, used ccc-analyzer, but it's another static analyzer tool that you could add to the list. However, it is still a separate pass to the normal compile (It happens to be part of clang, but it isn't used in "normal compilation"). Apr 15, 2014 at 10:50
  • it can be but does not have to be a separate pass. The outcome will still be a binary either way. My remark was mostly triggered by the claim that compilers have less depth, I'd argue it's the opposite. But then, depends on each respective tool. So I guess the issue I see with your answer is the (IMO inappropriate) generalization. Apr 15, 2014 at 13:59
  • 1
    The word I put there was "typically", which implies that "it may not apply to every compiler" - I agree that clang, using the right passes, is able to do a lot more than most compilers in this regard. (I'm working with LLVM both at work [not directly] and with my hobby Pascal compiler, and I use clang too). But there are also a whole range of other static analysis tools that allow various other detection that, currently, isn't in clang. Apr 15, 2014 at 14:35
  • So, I just ran my compiler project with clang++ --analyzer, and aside from it taking 13 minutes (ca 10x the normal time), and finding one genuine error [and one where it clearly didn't understand what was going on inside the stl libraries allocator implementation - variadic templates...], I disagree with the "you could run it with your regular compile" - at least with clang 3.5 from end of March, it outputs the results of the analyze as the output for -o somefile.o, which means there is nothing to link. I really wouldn't want to run that every time I compile something either. Apr 15, 2014 at 15:58
6

It is not that compilers cannot do the analysis that static analysis tools do. The problem is that static code analysis takes a significant amount of time and is usually not needed for every single compilation. Compilers are generally optimized for a balance of code quality and compilation time. If a compiler happens to stumble on an error in your code it will tell you, but it has no time to actively look for bugs.

4

Also, static analysis is important to generate metrics.

Those metrics can show : cyclomatic complexity, the depth of your class inheritance and many others / dependencies graph / % of comments and much (Understand has a complete list of features for example ).

A good point too is that most static analysis tools allows you to add specific rules which can be useful if your project / company have coding rules.

You can also add a "Continuous integration" server checkout-ing your svn/git/other development branch every day and doing the analysis by night. This way, next day you can fix the code that isn't compliant to your ruleset.

4

Yes, it adds more confidence to the code.

MISRA rules are used widely in real time systems. complying to MISRA (by running misra-supported tool) is required by many software auditors.

Also MISRA (or similar standards) define some cumbersome rules to prevent user from doing fancy stuff which might go wrong on some compilers. For example, pointer arithmetic is prohibited by MISRA, but no compiler will give you a warning on it.

EDIT: Another example: The following code is doing what many people don't expect (x != 1.1 because of floating point accuracy).

int main(int agrc, char** argv){

    float x = 1.1;
    if(x == 1.1)
        printf("Yes!\n");
    else
        printf("OMG!\n");
    return 0;
}

This is not detected by compilers (tried it on gcc 4.6 and clang 3.2). But it is detected by almost all the static analysis tools.

1
  • That depends on what you are trying to do with pointer arithmetics (whether it issues a warning). Either way MISRA is meant for the automotive industry, so plenty of the items in their guidelines do not apply to just any code base. However a lot of thought went into it and it's definitely worth using as a basis for own coding standards. +1 Apr 15, 2014 at 15:34
3

Good static analysis tools analyze thing like cyclometric complexity and even things like 'code smells' (see Martin Fowler's Refactoring book). Even line counts in a file or in a function are clues that your code can be improved. I don't think Lint covers these areas, though.

0
3

Are static analysis tools still needed? Absolutely. The warnings emitted by compilers are getting more sophisticated, but there's still the limitation that a compiler generally works on one source file at a time. Each file is compiled (*.c -> *.o), and the resulting object files are linked together at the end. To do truly effective static analysis, you need to look at the whole code base at once, analyzing individual files as well as the interactions between them. Compilers generally aren't designed like that, so a compiler alone usually misses the sorts of things that static analyzers pick up. You wouldn't want to build this functionality into the normal compiler because it's a fairly heavy performance impact. A static analysis run on my current project typically takes 4x the time a normal compile takes. You don't want this extra overhead on every single build, so it's best to farm it off to a separate, specialized tool than can be run when needed (or in the case of some modern compilers, a separate set of command-line options that are disabled by default).

How effective is static analysis? Very. My team finds a large number of potential problems through static analysis tools, most of which would be nearly impossible to find using other methods. It's particularly good at finding complex problems that humans aren't good at spotting, like those that involve the interaction between multiple local and global variables. Even if you have excellent test coverage, static analyzers will find all sorts of things that are difficult to find through testing alone. This is even more true in the embedded world, where testing tends to be more difficult and less automated. In my experiences, static analyzers have even showed us problems that we didn't even know we needed to be testing for in the first place.

I would definitely recommend the use of static analysis tools for any non-trivial software project. I actually run two separate static analyzers (one is built into the compiler suite and the other is a separate utility); you might be surprised what one will catch and the other will miss. I highly recommend that you come up with a customized set of rules/tests for your analysis runs instead of simply adopting a rule set like MISRA. Every project's needs are different, and many industry-wide specs like MISRA include a lot of stuff that isn't needed for most people. The more unnecessary stuff that you're checking for, the more time it takes to analyze, the more false positives you'll have to wade through, etc etc.

Not the answer you're looking for? Browse other questions tagged or ask your own question.