Cenzic 232 Patent
Paid Advertising
sla.ckers.org is
ha.ckers sla.cking
Sla.ckers.org
Whether this is about ha.ckers.org, sla.ckers.org or some other project you are interested in or want to talk about, throw it in here to get feedback. 
Go to Topic: PreviousNext
Go to: Forum ListMessage ListNew TopicSearchLog In
Intro to Web App Security lecture
Date: December 28, 2008 11:04PM

As part of MIT's SIPB IAP classes series, I am going to be giving a two-hour introductory lecture to Web Application Security.

I'm planning on focusing on XSS, HTML filtering and CSRF type attacks, since I've had the most personal experience on them. A brief outline of the talk is as follows:

1. String is not a type:
- Know where your strings are going (HTML, SQL, text?)
- Know what your string contains (plaintext, wikitext, HTML, bbcode?)
- Prefer to handle a "pure" version of the string internally
- Prefer safe APIs (prepared SQL statements, multiarg exec, dombuilders)
over escaping functions
2. String escaping/filtering functions are crypto.
(Use them but don't roll your own.) (Don't Repeat Someone Else)
3. A browser sent a request. Did the user mean to send it?

I'm thinking of posting my talk notes online and having the general public review them before January 8th, when I'm giving the talk. I'm also planning on preparing these for a web version of the talk. Would anyone be interested in helping review such a transcript?

Cheers,
Edward

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 01, 2009 06:32PM

I've posted slides for Part 1 here: http://web.mit.edu/~ezyang/Public/iap/intro-to-was.html

I suggest viewing the slides in outline form. You can toggle between formats by calling toggle() in your browser, or navigating to the bottom left of the slide and clicking on the slashed O.

You can also view rough drafts of the slides in http://web.mit.edu/~ezyang/Public/iap . They are named part1.txt, part2.txt and part3.txt

Cheers,
Edward

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: DoctorDan
Date: January 01, 2009 10:20PM

--Sorry, double post--



Edited 1 time(s). Last edit at 01/22/2009 11:54AM by DoctorDan.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: DoctorDan
Date: January 01, 2009 10:20PM

Nice! The slides look good. Who is your audience and what will their background be? I'm actually just a few T stops away (I go to Tufts)! Would love to go, but I'm not back to school for a while. Are you an MIT student?

-Dan

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: thornmaker
Date: January 01, 2009 11:29PM

Ambush Commander: I read the outline earlier and it looked really good. I'll take a look at the slides. I'm in the Boston area too, we should have a sla.ckers get together, eh :)

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 01, 2009 11:31PM

Quote

Who is your audience and what will their background be?

Probably people who have written a few small web scripts before, maybe have vaguely heard of this XSS thing, but haven't looked into it too deeply. It focuses very much on the implementor/defender side, and not so much on the pentesting/techniques side.

Quote

I'm actually just a few T stops away (I go to Tufts)! Would love to go, but I'm not back to school for a while.

Given your expertise, 80% the lecture would probably be a waste of time for you. :-) I've tried to put in some interesting bits that will be new even for veterans, but the target audience certainly shows.

One of the things I would like the wider community to pick up is the idea that escaping is not the whole story: if you think that "if you escape stuff, I'm safe", you're doing it wrong.

Quote

Are you an MIT student?

Yep!

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 01, 2009 11:33PM

Quote

I'll take a look at the slides.

Great! Any comments you have are welcome.

Quote

I'm in the Boston area too, we should have a sla.ckers get together, eh :)

Ah, nice! I'd be up for something like that, though this event probably wouldn't be that appropriate.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: iota
Date: January 02, 2009 09:04AM

Clear, Concise, Short to the point!
Excellent!

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: thornmaker
Date: January 02, 2009 10:29AM

The slides are good. I like your approach of framing things from a students perspective, one who has some php/java programming experience, but hasn't given security much thought. I have just a couple of suggestions.
1 - include some real world examples of the different bugs like XSS, CSRF - even if it is with a dumny app like web goat, it helps get the point across.
2 - perhaps include some "hands-on" activities that the students can mess with during the lecture. Again, something like web goat might fit the bill here, a home-made lab, or something completely different.

btw, your telegraph metaphor is great!

Quote

Ah, nice! I'd be up for something like that, though this event probably wouldn't be that appropriate.
i had in mind some other venue like a local pub or restaurant

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 02, 2009 11:12AM

Quote

1 - include some real world examples of the different bugs like XSS, CSRF - even if it is with a dumny app like web goat, it helps get the point across.
2 - perhaps include some "hands-on" activities that the students can mess with during the lecture. Again, something like web goat might fit the bill here, a home-made lab, or something completely different.

I agree, these are things worth looking at. Maybe step out of the slides for a moment, fire up a web browser, and demonstrate a vulnerability in a website in realtime.

A hands-on activity would probably be more appropriate for the pentesting aspect of things, so I'm not quite sure how to work it in here.

Quote

btw, your telegraph metaphor is great!

Hehe, thanks! Unfortunately, I decided to cut it from the final slides, in favor for simply showing what happens when someone inserts XSS.

Quote

Clear, Concise, Short to the point!
Excellent!

Thanks! Glad you like the slides.

HTML Purifier - Standards Compliant HTML filtering



Edited 1 time(s). Last edit at 01/02/2009 11:12AM by Ambush Commander.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: kuza55
Date: January 02, 2009 02:19PM

Ambush Commander Wrote:
-------------------------------------------------------
> One of the things I would like the wider community
> to pick up is the idea that escaping is not the
> whole story: if you think that "if you escape
> stuff, I'm safe", you're doing it wrong.

I haven't looked at the slides yet, but I don't see what the alternate recommendation for stopping xss would be?

If we have some kind of way of separating html(code) and data, e.g. having html objects with properties which auto-sanitize (which .NET seems to do a resonable job) or some kind of bound parameters implementation, then we can do better than just escaping, but since most of the time we don't: escaping properly is really the only decent method we have. It'd be nice if we didn't have to do it manually all the time, but that requires a reworking of how we construct html.

----------------------------------------------------------
Don't forget our IRC: irc://irc.irchighway.net/#slackers
[kuza55.blogspot.com]

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 02, 2009 08:30PM

That's exactly what I talk about in the lecture. Look at the Safe API slides, around slides 24-32.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 05, 2009 02:08AM

The last two parts of the lecture are up. They're a bit shorter than the rest, so tell me if you think there's something I should have covered that I didn't.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: thornmaker
Date: January 05, 2009 01:56PM

perhaps you don't want to go into click jacking much, but if you do go beyond 1 slide, i suggest using the same approach to introduce it as you use for the other topics. e.g. "say you are surfing and see this form with a button that says 'submit'. you would assume that clicking the button submits the form, right? oops! you just turned on your web cam to the world... "

or something like that

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 05, 2009 03:50PM

So, the idea is when I show the "Now for something new" slide, I show them a video demo of a clickjacking exploit, with me narrating a bit. I feel like that will be a lot more vivid than a hypothetical.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: thornmaker
Date: January 05, 2009 10:27PM

i didn't mean to use a hypothetical but i didn't explain myself very well (was in a bit of a rush when i posted). a video example would be perfect, considering it's a visual attack that you need to see to really understand.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: thornmaker
Date: January 12, 2009 03:53PM

so how did it go?

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 12, 2009 06:23PM

It went quite well, and a lot of people showed up. :-)

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: id
Date: January 12, 2009 08:14PM

Cool, when can we expect the video online?

-id

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 12, 2009 11:05PM

Sorry, but it wasn't videotaped.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: rvdh
Date: January 13, 2009 02:56AM

Hi Edward.

While reading: http://web.mit.edu/~ezyang/Public/iap/intro-to-was.html I don't agree on that the used examples are insecure. You just presented them in a insecure manner, in such a way that htmspecialchars without flags is insecure, but you always have to set flags. So the function isn't insecure, it's insecure if you do not understand what it's meant for.


This is rather a faux representation, not sure what you mean but is it a personal preference or something? like, having everything javascript-esque, because it looks interesting? (just guessing here) since if you set the proper flags nothing will be more insecure:

#

$html = '<b>' .
            htmlspecialchars($text) .
        '</b>';

# Use:

$b = $doc->createElement('b');
$b->addChild(
    $doc->createTextNode($text)
);

I mean there is nothing wrong with:

echo  "<strong>" . htmlspecialchars($text,ENT_QUOTES,'UTF-8') . "</strong>";

It requires you to type less, and you can customize the charset as well as the other flag constants.

$query = http_build_query(array(
    'name' => $foo  ));
$url = 'index.php?' . $query

Wow, that is pretty dangerous you are doing right there if you rebuild a query where $foo is required dynamically and past through that function. It's advised to use hard coded switches that match the possible arguments and fail if they don't.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 13, 2009 02:41PM

Quote

While reading: I don't agree on that the used examples are insecure. You just presented them in a insecure manner, in such a way that htmspecialchars without flags is insecure, but you always have to set flags. So the function isn't insecure, it's insecure if you do not understand what it's meant for.

When I was giving the lecture, the examples made me double-take as well. They don't illustrate the core concepts I'm trying to express well, are completely wrong at some points, and need to be updating.

One thing that the lecture implies that is not true is that htmlspecialchars() is always insecure. It's insecure if applied incorrectly, without knowledge, but can be used securely.

Quote

I mean there is nothing wrong with:

echo "<strong>" . htmlspecialchars($text,ENT_QUOTES,'UTF-8') . "</strong>";

I disagree on several points:

* ENT_QUOTES is an implementation detail; the location of this string (regular text) is obvious and what I should be concerned with

* ENT_QUOTES is a little wasteful; ENT_NOQUOTES is, in this case, perfectly secure. One could argue that ENT_QUOTES is the defense-in-depth measure, but I would argue that this is necessary because string concatenation can do it wrong so easily.

* UTF-8? You shouldn't have to hard-code this string on every escape function you use. There is no reason this charset should change through the execution of a template.

text2html() or escapeHtml() as an alias to htmlspecialchars() would be a better choice; escapeHtmlQuotAttr(), escapeHtmlAposAttr(), escapeHtmlRegular() would make the context distinction clear (these names kind of suck, but hopefully you get the idea).

Quote

$query = http_build_query(array(
    'name' => $foo  ));
$url = 'index.php?' . $query


Wow, that is pretty dangerous you are doing right there if you rebuild a query where $foo is required dynamically and past through that function. It's advised to use hard coded switches that match the possible arguments and fail if they don't.

I don't quite understand you here.

HTML Purifier - Standards Compliant HTML filtering

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: rvdh
Date: January 14, 2009 01:33AM

Quote

I disagree on several points:

* ENT_QUOTES is an implementation detail; the location of this string (regular text) is obvious and what I should be concerned with

* ENT_QUOTES is a little wasteful; ENT_NOQUOTES is, in this case, perfectly
secure. One could argue that ENT_QUOTES is the defense-in-depth measure, but I
would argue that this is necessary because string concatenation can do it wrong so easily.

* UTF-8? You shouldn't have to hard-code this string on every escape function you
use. There is no reason this charset should change through the execution of a template.

* You made a big mistake right there. The ENT_NOQUOTES quote style leaves the quotes unaltered instead of translating them, that's the least thing you would
 want. All quotes are converted with ENT_QUOTES, this is useful because of this principle:

-------------------------------------------------------------------
INPUT: obtain data -> escape -> put in database.

OUTPUT: obtain data -> remove slashes (if any) ? -> translate with htmlspecialchars -> charset? -> display.
-------------------------------------------------------------------

I've been doing this since at least 2003, I worked on thousands of websites/applications, do
 you really think I don't got the jist of it by now? I see absolutely no reason
 to use anything else. Just a suggestion. :)

By the way on every site I worked on I never had to "purify" html, because that
 fails ad infinitum. In my opinion, data should be entered verbatim into a 
database, and only escaped. And then be converted on output at will.

* The charset can change if I want it too, it's just convenient in some 
situations I've worked in. Moreover, let's assume the charset is set through an
included file, what if the inclusion fails and the data is, for example UTF-7 or
some other exotic charset? What if one developer changes a charset defined in 
their httpd.conf or htaccess when I am away? No, I always set the charset and 
quote style to the charset and data type I expect. Moreover htmlspecialchars 
defaults to ISO-8859-1.



Quote


Quote

$query = http_build_query(array(
'name' => $foo ));
$url = 'index.php?' . $query

Wow, that is pretty dangerous you are doing right there if you rebuild a query where $foo is required dynamically and past through that function. It's advised to use hard coded switches that match the possible arguments and fail if they don't.

I don't quite understand you here.

Well, you seem to rebuild a query obtained through the querystring that is shown as $foo. So nothing happens here, you only rebuild it without sanitation. 
Hence, I said: use switches to match the page names or anything else you are trying to obtain through the querystring. If you don't use switches or cast it,
 you will be into trouble.

This is not criticism, I am just trying to help. Please understand, I got enough flight hours to know what I am doing.



Edited 7 time(s). Last edit at 01/14/2009 01:53AM by rvdh.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Date: January 14, 2009 03:25AM

This looks like it'll be a lively conversation.

First, the broad picture: your "principle" does work (barring a few minor details). But it doesn't work because it's a magic incant; there are very specific reasons why it works: each of your steps reflects a format-shift to the various mediums you need.

Internally, you probably understand this "format-shifting" business, to the point which you don't need to verbalize it. You know it works. But a beginner developer doesn't see the underlying connections: they just know "if I do this" and "do this", they'll be safe.

I was nitpicking when I told you what I thought was wrong with your piece of sample code. But that's exactly the problem: if you deal on the level of htmlspecialchars(), you're dealing on an abstraction that's too low to be safe. It's why you don't want to be worrying about garbage collection and buffer overflows on web applications. There are bigger fish to fry.

My reasons for this "javascript-esque" style are two-fold: one is that it is pedagogical: verbose or not, the DOM is what, in the end, a browser cares about (it's no surprise that you found it to be "javascript"-esque). When you write HTML tags, you're simply expressing, in a text-only shorthand, element and text nodes. If you don't believe me install Firebug and mouse around a webpage with the inspect elements feature on. (I don't talk about this much in the lecture notes)

The second is that it prevents error. Someone else hacking on your code can forget to apply your recipe; more likely, they'll forget to write well-formed HTML. (I do talk about this in the lecture notes).

Quote

You made a big mistake right there. The ENT_NOQUOTES quote style leaves the quotes unaltered instead of translating them, that's the least thing you would
want.

Your argument here assumes that you want the quotes translated, which is what you really need to be defending. Untranslated quotes are perfectly fine when not in attribute values; single quotes are perfectly fine in double quoted attributes and vice versa. And yes, I understand and acknowledge the defense-in-depth argument.

Quote

escape -> put in database.

If you're using bound variables you shouldn't need to explicitly escape your input data. While the prime-time readyness of DOM builders is debatable, bound queries are tried and true and should always be used.

Quote

remove slashes (if any) ?

This doesn't actually make sense. The fact that you need to do this means you double-escaped, which means you weren't paying close enough attention to the formats of your variables.

Quote

By the way on every site I worked on I never had to "purify" html, because that
fails ad infinitum.

Do you use BBCode? Textile? Markdown? If so, you are converting some markup to HTML, and then purifying against some subset of the HTML spec. (maybe in parallel)

Quote

No, I always set the charset and
quote style to the charset and data type I expect.

This was a misunderstanding; I wasn't very clear: you do have to set the encoding variable, but it should be done globally. And a good way to do that is to have a wrapper function around htmlspecialchars()

Quote

So nothing happens here, you only rebuild it without sanitation.

Sometimes I might not want the sanitation/validation. If I'm running an XSS database I need to allow arbitrary characters in my URLs.

Cheers,
Edward

HTML Purifier - Standards Compliant HTML filtering



Edited 1 time(s). Last edit at 01/14/2009 03:26AM by Ambush Commander.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: rvdh
Date: January 15, 2009 05:46AM

Not sure what you are trying here but your reasoning fails on almost any point, especially when dealing with real world situations. I'm the guy in the field, not some guy sitting in some academic racket theorizing what people "should" use and what not, because that is really pointless.

Quote

If you're using bound variables you shouldn't need to explicitly escape your
input data. While the prime-time readyness of DOM builders is debatable, bound
queries are tried and true and should always be used.

I'll even debate you on the fact that parameter binding isn't necessary as well, it's mere preference, and not any more secure than escaping data, and un-escape them on output. And why do you think that is secure? because someone said so? did you did research on it to conclusively say that parameter binding suffices? because it isn't. It's the same argument that stored procedures are safer, it's a non-point because it depends upon implementation.

Quote

When you write HTML tags, you're simply expressing, in a text-only shorthand,
element and text nodes.

God. are you serious? or pulling a leg here? but that doesn't mean I "have" to use DOM functions in PHP. There is no progress here, it's more aesthetics rather than a better example of security. Have you tested those DOM functions in PHP? how do you know it's safe? I know this is safe:

echo "<strong>" . htmlspecialchars($text,ENT_QUOTES,'UTF-8') . "</strong>";

I'm pretty sure it is, can you vouch for the DOM function? I know I won't given the bug/track-history of PHP.


Quote

This was a misunderstanding; I wasn't very clear: you do have to set the encoding variable, but it should be done globally. And a good way to do that is to have a wrapper function around htmlspecialchars()

Of course, but like I said, what if something changes along the way. Let's assume the charset is is defined in the httpd.conf or the php.ini. What happens when an administrator decides one day to upgrade to PHP5 from PHP4, or something similar?

Bingo!

Big chance that your "global" charset is overridden. So it IS advised to set it globally, BUT it's also advised to set it upon data retrieval and not in the backwards logic you are using here, because you have the assumption that the global charset will never change. (assumption is the mother of all fuckups) further more I've said that already, inclusions might fail as well, which also results in charset changes if they are set in an include. This isn't the biggest issue, but it's important as well.

So, my conclusion is that your examples are based upon the love for blackbox security and code-aesthetics, rather than an actual representation of in-secure methods and ways to code. You didn't explain anything, besides showing your preferences.



Edited 1 time(s). Last edit at 01/15/2009 06:11AM by rvdh.

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: nEUrOO
Date: January 23, 2009 09:19PM

rvdh Wrote:
-------------------------------------------------------
> I'll even debate you on the fact that parameter
> binding isn't necessary as well, it's mere
> preference, and not any more secure than escaping
> data, and un-escape them on output. And why do you
> think that is secure? because someone said so? did
> you did research on it to conclusively say that
> parameter binding suffices? because it isn't. It's
> the same argument that stored procedures are
> safer, it's a non-point because it depends upon
> implementation.

Well, at some point you have to assume that the implementation is okay, otherwise, it's a never-ending problem.
Concerning the stored procedure, how could a strong data binding be worse than escaping data. I mean, in SQL, your queries are actually compiled and data just replaced with, automatically, the "good" & strong typing. Of course it depends on the database implementation and, for instance, you may be able to exploit database software flaws, but this is misleading somehow since when you do escape SQL data, you only want to prevent from SQL injection/manipulation.

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Options: ReplyQuote
Re: Intro to Web App Security lecture
Posted by: rvdh
Date: January 27, 2009 09:23PM

Agreed, I don't postulate that binding parameters is unsafer than escaping data, but since I haven't seen any research on breaking -or attacking- those libs I cannot assume it's safe, can I? assumption is the key problem here and black-box security, if you don't process your data before you let the black box handle it for you.

So what I meant is that when you trust your PHP and MySQL, or libs without doing some research on it, you can be in for surprises. A good example is the article from Stefan Esser on the truncation attack on MySQL, everyone assumed that MySQL would be safe, whereas I (and others) said plenty of time to check the length and type of data your getting from input, BEFORE letting the blackbox (PHP,MySQL) process it for you. As it turns out, it's the safest bet.

Options: ReplyQuote


Sorry, only registered users may post in this forum.