Does anybody have any thoughts on Fortify vs. Ounce vs Klockwork vs. Coverity? Preferred choice?

answer: depends on what you wanna scan... I mean, really...

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Fortify is great with packs if the language and framework are supported.
http://www.cigital.com/securitypack/
The customization is decent in Fortify, but does leave a lot to be desired.
http://www.cigital.com/papers/download/Framework%20for%20Custom%20Rules.pdf
so people turn to Checkmarx --
Checkmarx works with the object models better and doesn't require compilation/build

Many are using Ounce over Fortify because of supported frameworks and/or cost. Others are using Ounce because they are looking to go deeper with O2.

Klocwork and Coverity are different... check out "Hacking Exposed Linux, 3rd Edition" for information on these two products and integration with C programming logics.

I think a lot of people are just using manual code review instead of any of the above because of the lack of capabilities. While difficult to do data flow and control flow manually in your brain (data flow is where the tools are especially useful), it is all but required in almost all scenarios where software risk is the utmost importance.

The preferred tool is Source Insight (it supports syntax highlighting languages inline with other languages, as well as macros), followed by SciTE and Brixsoft SourceEdit -- although many use the Visual Studio Express Editions or Eclipse. Developers in Microsoft environments usually have access to VSTS (Tester or Team Suite are especially nice to have), but security people generally do not (because nobody wants to buy extra licenses for people who don't code full-time). I know that the CAT.NET team is working on integrating it with the Express Editions, but right now you need at least the Visual Studio Standard Edition to integrate CAT.NET in the IDE (you could always run it from the command line instead).

Others use grep or lexical analysis command line tools for file-based analysis to great effect, so YMMV. When you're working with object-oriented managed languages such as Java Enterprise or C# -- Fortify, Checkmarx, and Ounce are extremely useful to a point -- they can save a lot of time -- but that doesn't mean that file-based (grep) or Source Insight aren't also going to be useful to solve other problems and find other types of issues.

The tools are only going to find low-hanging fruit type weaknesses and they have a lot of corner cases. You run them and they find stuff, and then you sort through it while also looking elsewhere and running other kinds of checks with tools or manual review.

Just want to add that you don't need visual studio to run CAT.NET, there is command line version of this tool (which require some VS libs, but you can get them freely). You can also customize it correctly (like LAPSE if you are familiar with it).

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

There's a new report out from Gartner where they do some comparisons, haven't read it yet myself so I don't know how relevant it is, but here it is :
http://www.fortify.com/landing/downloadLanding.jsp?path=%2Fpublic%2FGartnerMQ_StaticApplicationSecurityTesting.pdf

@ holiman:

buh........ blah blah blah Gartner blah blah blah haven't read iet myself blah blah blah relevance

@ntp

Tell us how you really feel? ;)

--thrill

---

It is not the degrees you hold, but the mind you possess. - thrill

@ntp:
Of course, after your Ultimate Answer, who could possibly be interested in some ol' report that just came out? Apologies for being so blatantly off-topic...

@ holiman: it's been out for months. i think i got to review it before it went public.