Cenzic 232 Patent
Paid Advertising
sla.ckers.org is
ha.ckers sla.cking
Sla.ckers.org
This is a place for us to start seriously talking about vendors. Who's great, who's not, what's it cost, how does it relate to their competitors and would we buy it? A place to talk about snakeoil, and brilliant products alike. Marketing fluff is forbidden. 
Go to Topic: PreviousNext
Go to: Forum ListMessage ListNew TopicSearchLog In
Source Code Analysis
Posted by: br0kan
Date: March 25, 2009 07:37AM

Does anybody have any thoughts on Fortify vs. Ounce vs Klockwork vs. Coverity? Preferred choice?

Options: ReplyQuote
Re: Source Code Analysis
Posted by: nEUrOO
Date: March 26, 2009 10:47AM

answer: depends on what you wanna scan... I mean, really...

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Options: ReplyQuote
Re: Source Code Analysis
Posted by: ntp
Date: March 26, 2009 12:17PM

Fortify is great with packs if the language and framework are supported.
http://www.cigital.com/securitypack/
The customization is decent in Fortify, but does leave a lot to be desired.
http://www.cigital.com/papers/download/Framework%20for%20Custom%20Rules.pdf
so people turn to Checkmarx --
Checkmarx works with the object models better and doesn't require compilation/build

Many are using Ounce over Fortify because of supported frameworks and/or cost. Others are using Ounce because they are looking to go deeper with O2.

Klocwork and Coverity are different... check out "Hacking Exposed Linux, 3rd Edition" for information on these two products and integration with C programming logics.

I think a lot of people are just using manual code review instead of any of the above because of the lack of capabilities. While difficult to do data flow and control flow manually in your brain (data flow is where the tools are especially useful), it is all but required in almost all scenarios where software risk is the utmost importance.

The preferred tool is Source Insight (it supports syntax highlighting languages inline with other languages, as well as macros), followed by SciTE and Brixsoft SourceEdit -- although many use the Visual Studio Express Editions or Eclipse. Developers in Microsoft environments usually have access to VSTS (Tester or Team Suite are especially nice to have), but security people generally do not (because nobody wants to buy extra licenses for people who don't code full-time). I know that the CAT.NET team is working on integrating it with the Express Editions, but right now you need at least the Visual Studio Standard Edition to integrate CAT.NET in the IDE (you could always run it from the command line instead).

Others use grep or lexical analysis command line tools for file-based analysis to great effect, so YMMV. When you're working with object-oriented managed languages such as Java Enterprise or C# -- Fortify, Checkmarx, and Ounce are extremely useful to a point -- they can save a lot of time -- but that doesn't mean that file-based (grep) or Source Insight aren't also going to be useful to solve other problems and find other types of issues.

The tools are only going to find low-hanging fruit type weaknesses and they have a lot of corner cases. You run them and they find stuff, and then you sort through it while also looking elsewhere and running other kinds of checks with tools or manual review.

Options: ReplyQuote
Re: Source Code Analysis
Posted by: nEUrOO
Date: March 26, 2009 05:34PM

Just want to add that you don't need visual studio to run CAT.NET, there is command line version of this tool (which require some VS libs, but you can get them freely). You can also customize it correctly (like LAPSE if you are familiar with it).

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Options: ReplyQuote
Re: Source Code Analysis
Posted by: holiman
Date: March 29, 2009 02:23PM

There's a new report out from Gartner where they do some comparisons, haven't read it yet myself so I don't know how relevant it is, but here it is :
http://www.fortify.com/landing/downloadLanding.jsp?path=%2Fpublic%2FGartnerMQ_StaticApplicationSecurityTesting.pdf

Options: ReplyQuote
Re: Source Code Analysis
Posted by: ntp
Date: March 29, 2009 07:47PM

@ holiman:

buh........ blah blah blah Gartner blah blah blah haven't read iet myself blah blah blah relevance

Options: ReplyQuote
Re: Source Code Analysis
Posted by: thrill
Date: March 29, 2009 08:09PM

@ntp

Tell us how you really feel? ;)

--thrill

---

It is not the degrees you hold, but the mind you possess. - thrill

Options: ReplyQuote
Re: Source Code Analysis
Posted by: holiman
Date: March 30, 2009 01:27PM

@ntp:
Of course, after your Ultimate Answer, who could possibly be interested in some ol' report that just came out? Apologies for being so blatantly off-topic...

Options: ReplyQuote
Re: Source Code Analysis
Posted by: ntp
Date: March 30, 2009 01:45PM

@ holiman: it's been out for months. i think i got to review it before it went public.

Options: ReplyQuote


Sorry, only registered users may post in this forum.