Cenzic 232 Patent
Paid Advertising
sla.ckers.org is
ha.ckers sla.cking
Sla.ckers.org
If you have some interesting news or want to throw up a link to discuss it, here's the place. Anything is okay, even shameless vendor launches (since that is often applicable to what we work on). 
Go to Topic: PreviousNext
Go to: Forum ListMessage ListNew TopicSearchLog In
Challenges faced by automated web application security assessment tools
Posted by: zeno
Date: November 29, 2006 05:39PM

If you're in the position of evaluating a web application security scanner, or use one to fullfill a compliance scanning requirement then you may want to check out an article I wrote describing some of the challenges these products face.

http://www.cgisecurity.com/articles/scannerchallenges.shtml

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: rsnake
Date: November 29, 2006 05:42PM

I actually did before you mentioned it here... very well written article, zeno.

Some of the key things I think are worth discussing are some of the "how to detect it's bad without doing anything harmful" stuff. XSS is easy, but buffer overflows and SQL injection may not be.

- RSnake
Gotta love it. http://ha.ckers.org

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: jungsonn
Date: November 30, 2006 06:18AM

Yeah there isn't a "out of the box scanner" that can do this kind of job, and even if it could, it renders itself useless, cause it requires the same programming skills to fix the problem it found, as you would prevent it in the first place before you ran the scanner. You can't replace the programmer.

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: rsnake
Date: November 30, 2006 10:42AM

I mean, I get it. The main reason for scanners isn't because they are better at finding things (maybe they are less likely to miss something that is easy to tests for). The main reason scanners exist is for scalability. It's much cheaper to run a scan than to get some guy out to the customer premise to test everything by hand. The ROI on scanners is far better than that of individuals.

- RSnake
Gotta love it. http://ha.ckers.org

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: digi7al64
Date: November 30, 2006 06:01PM

Personally I believe the concept of a web application security scanner as a means of determining web application security is a joke. Sure it might help for known server/application bugs but it fails miserably almost everywhere else.

This is because it fails to audit the source code which is the foundation from which web application security is derived from.

----------
'Just because you got the bacon, lettuce, and tomato don't mean I'm gonna give you my toast.'

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: maluc
Date: November 30, 2006 07:17PM

Well, i think it *could* be possible to write a scanner which detects the bulk of the vulnerabilities in custom web applications.. but it'll almost assuredly have to be a white box scanner (access to the source code)

Most scanners on the market right now don't test custom web app holes. which would be the top two rows of http://photos1.blogger.com/blogger2/1912/1679/1600/vulnerability_assessment_comparison_chart.1.png
(Picture from Jeremiah Grossman's blog post)

We really don't need more re-packaged nessus clones, and their customers need to realize the difference of what they scan for :T .. and jungsonn is right to suggest that it still requires a programmer who understands what XSS/SQLinj/CSRF is, in order to fix the places it found. So they'll never be a security team/audit replacement - but a well designed one could be an invaluable tool that saves alot of the code reviewing time.

I wouldn't put my faith in the current crop of scanners.. but i'd like to be able to some day.

-maluc

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: id
Date: November 30, 2006 07:38PM

I'm curious if anyone has tried this http://www.fortifysoftware.com/

-id

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: digi7al64
Date: November 30, 2006 10:00PM

Have never tried http://www.fortifysoftware/ (in fact I've never user any automated tools) but they seem to be partly on the right track with integrating the auditing process into the SDLC.

maluc - you are correct, a white box scanner will be the most efficient and practical way to audit the code, and in reality the most effective way to do this would be a tiered approach (my comments below are not complete cause i only spent 10mins thinking about it, but the idea makes sense)

1. Capture all places within the code that use
> GET and POST requests,
> Server Variables, and
> Cookies.

2. Using the information obtained from 1 follow the code logic, values and determine where they are used
> Regex's/Parsing functions,
> SQL statements,
> Select/Switch,
> Echo/Response.write,
> Include Files.

3. Report on any unfiltered/insufficiently filtered content

----------
'Just because you got the bacon, lettuce, and tomato don't mean I'm gonna give you my toast.'

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: jungsonn
Date: December 01, 2006 08:35AM

I understand what your're saying here, and it could be a way towards scanning. But im not so sure with the lists one match against, alot of programmers use there own way of writing code, using more & more classes and have much freedom of naming variables.

Is the approach purely scanning and spitting out possible lines of code that could be in danger?

That would be awfull hard to match against.

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: digi7al64
Date: December 04, 2006 08:30PM

It is a doubled edged sword from where i stand. The way i see it White box scanning is the only way to protect a site unless you want to spend the next five years making a neural network smart enough to factor in all equations.

For instance consider a simple sql injection using asp (with on error resume next included in the file).

In ASP a web application scanner might
a. Get the url h**p://example.com/test.asp?id=1 and collect the page contents for cross checking
b. In will then try to inject a simple sql query... say h**p://example.com/test.asp?id=1'. the application fails but the on error resume next ensures it still returns some page content (ie. resource not found). the scanner cross references the 2 returned files and finds a difference.
c. The scanner then performs a h**p://example.com/test.asp?id=1' or 1=1-- injection... again it fails, but the on error resume next ensures it still returns some page content (ie. resource not found). the scanner cross references the 2 returned files and finds a difference.
d. The scanner then performs a h**p://example.com/test.asp?id=1 or 1=1 injection... this time it doesn't fail. But the programmer has only returned the results for the first record (doesn't use any loop/while statements).

At this point we have 2 fails (b and c) and 2 that appear the same (a and d). now the question is (as a and d are equal in content does this mean a injection has been found?) and with (b and c) are they injectable?

Ok, so our scanner has made no real progress and we decide to try some different vectors on a different page. lets say
e. The scanner then performs h**p://example.com/login.asp?un=admin&pass=god (Which is a valid account) and we login (the scanner logs the content and or the redirection url).
f. The scanner then performs h**p://example.com/login.asp?un=admin --(Which is a valid username) and the login fails.
g. The scanner then performs h**p://example.com/login.asp?un=' or 1=1 --(and the login fails.
h. At this point our scanner is not really achieving much so we take it up another notch and we try h**p://example.com/login.asp?un=' or '1'='1&pass=' or '1'='1 (this time everything should work except it doesn't, and the login fails).

So given that with the exception of b and c which returned nothing we could assume that the site is secure...but it isn't. The programmer simply added a count check for the recordset returned and when it was more the one it failed to return the result. Furthermore as it turned out we only needed to bounce through the re-direct page (on successful login) which set up our session variables. oh and as for why none of the injections worked (c,f and g) he was using MSAccess.

And this is the problem with web scanning, it simply can't factor in every possible scenario. Instead if we white boxed it we could have simple seen that the values being submitted weren't parsed through any filters and therefore it was possible to inject. We then from there could have traced the value in the code and examined the logic.

----------
'Just because you got the bacon, lettuce, and tomato don't mean I'm gonna give you my toast.'

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: jungsonn
Date: December 05, 2006 07:39AM

Yeah correct, just like "spaces" you can do alot with spaces, tabs, carriage returns which are tough to test on. In SQL you have something like this:

UNION SELECT *

this is different then:

UNION SELECT *

Or like:

'UNI' + 'ON' + 'SELECT'

||UNI||ON||SELECT

or:

' OR1=1

is different then:

' OR 1 = 1

And could output very strange things which are hard to anticipate upon.

Or just HEX encode the whole query, this would mean you must hex encode every vector in existence to pentest against it. That's why i never put my money on such scanners. If you check the sourcecode and sift through by hand, you'll easely spot the weaknesses pretty fast.

Options: ReplyQuote
Re: Challenges faced by automated web application security assessment tools
Posted by: jungsonn
Date: December 05, 2006 07:40AM

Oops! the board removed the big spaces, anyway: having many spaces between SQL functions, can output stranged results.

Still my motto is simple: you can't automate security. there has to be a human involved who knows ways around the system dispite known attack vectors, which could depend on server-software, db type, server-side language, which encoding, etc, etc.



Edited 1 time(s). Last edit at 12/05/2006 07:43AM by jungsonn.

Options: ReplyQuote


Sorry, only registered users may post in this forum.