If you have FF 2.0 by now, you should know about the news that they have an "anti-phishing" feature build in. I thought i'dd go examine this feature and found some interesting things about it, and some serious flaws IMO.

It seems they base the blacklist on a list maintained by Google:

http://sb.google.com/safebrowsing/update?version=goog-white-domain:1:17,goog-white-url:1:361,goog-black-url:1:6806,goog-black-enchash:1:12490

At least that is what i found out by looking at the incomming/outgoing tcp requests, if you look at the list (a very small one BTW)
you'll see sites that are being blacklisted. ok, very good.
But there is a problem.

If you copy such URI found in that list into your browser, and go to it.
FF shows you that nice warning, Ok superb! but, what happens when i change the ip, convert it to Hex Address? i can go to that site _whithout_ seeing that warning...

example:

http://200.119.135.99/ebay/login5878/

I converted to:

http://0xc8.0x77.0x87.0x63/ebay/login5878/

and no warning.

Seems not a very strong feature afterall, any ideas about it?

(EDIT: spellchecks ;)



Edited 2 time(s). Last edit at 10/30/2006 04:23PM by jungsonn.

Good find. I've heard that it wasn't so good but I haven't really looked into it myself.

That list surely is tiny, but more importantly black listing never works as RSnake has pointed out time and time again.

Don't forget our IRC: irc://irc.irchighway.net/#slackers
-WhiteAcid - your friendly, very lazy, web developer

Heh, this is really funny, I managed to get the red icon and a triangle stuck on my browser chrome. Buggy...

The URLs in the goog-black-url are a strict match, however, the values in goog-black-encash are regular expression values. The key is a hash of the normalized domain or ip address and the value is a list of regular expressions. The values in that table are stronger and harder to evade.

Thanks, tc, good to know... why are they any different though? What's the purpose of having two?

- RSnake
Gotta love it. http://ha.ckers.org

There are some data licensing reasons as to why there are two lists. We're working on having a single list, but it may require some changes to the existing protocol, so it's a bit more long term.

even assuming they were all hashes.. thats a seriously small list

i think this may do more harm than good, by giving a false sense of security. but they get an E for effort (better than an F)

-maluc

Hmm indeed.

with all due respect, if this feature is one of the main biggest features of the new flagship, i would expected more from it. Ok, i can imagine that a blacklist is faster to check on the half hour in the browser then to actively analyse a page. But overall improvements can be made, think about a triage on reverse DNS entries?, and a fix for that Hex Address conversion should be minimal i guess. :)

Anymore suggestions about improvements one could make/think of ?

Fuzzy matching seems like it would solve a lot of these issues... also looking at the resultant IP address instead of a direct string compare would solve the issue you mentioned. Both seem like a really good idea. tc, which lists are you using, can you say? This list seems way to small to be from APWG or PRN unless they were heavily pruned.

- RSnake
Gotta love it. http://ha.ckers.org

The list from the link above isn't complete (it's an update from what the client currently had). The lists are pruned.

Fuzzy matching is a harder problem. For example, comparing strings is hard because of unicode characters that look like regular ascii characters. Anyway, it's something that's being looked into.

Got me thinking again, i noticed that the list is imported/screened via a GET method, and not even on a secure line. So the datastream "could" be intercepted and modified? Just asking for more trouble, still i think that blacklisting based on submission won't work, maybe with some heuristics on the issue: If for instance Ebay sits on a narrow IP range, one could filter against the known range, (rev.DNS also) and a matching URI (like unique loginpage) which phishers copy, and render it "possibly black".

anywayz, those are a few ideas i have about it.

Unrelated to your comment, Jungsonn, but I just realized that if you open these phishing sites inside an IEtab they don't show up as phishing sites, even if IE7.0 does. So neither the Firefox nor the IE7.0 protection works inside of a Firefox IETab. Wonderful.

- RSnake
Gotta love it. http://ha.ckers.org

The data file is downloaded over http, but first a key is gotten via https. It's tricky stuff and you're welcome to poke holes, but it's not as simple as just pwning a GET request

http://lxr.mozilla.org/mozilla/source/toolkit/components/url-classifier/content/url-crypto-key-manager.js#38

I never said it would be easy, i noticed that the list gets updated multiple times in a session (http), I think it's reasonable to load the list just once a day, but when the list _IS_ loaded via http, one could tamper the stream. i don't understand why you need to match it after that, if it's already being loaded via https in a base64 encoded (this aint encryption BTW, as being stated in that Url-Crypto-class *sigh*) key? why not run 1 persistent https session to load the list? could be that i'm missing something here, but this spins in my head now :)

@RSnake,

u mean IEtab, the extension? if so, i haven't got the extension on my linux machine, but that's awful... hence, i think extensions shouldn't be allowed to tamper/manipulate the browse/chrome core functions, to big of a risk in all cases.

IETab uses the IE browser component the way other apps (TurboTax, Quicken, AIM, etc.) embed IE. It's not going to pick up anything that's part of the IE *application* built on that browser core -- there's no IE UI involved. I personally think IEView (launch separate IE window) is a safer way to go, though more cumbersome.

The reason Google uses http is to avoid melting their servers. Don't know if they were just being cautious or if it's a real problem, but that's the reason given in that source file. The current list is updated every half-hour which seems pretty aggressive. They could ease the load by backing off a bit, though since it's only sending diffs that's saving connections but wouldn't save any data.

I think the main concern about https is the active checking mode, where every URL is sent to Google for a server-side check. That's not the default in Firefox due to privacy concerns but it's the way the Google toolbar uses the anti-phishing server. Doing a full SSL handshake on every URL the user visits really would be prohibitive, as would the alternative of holding tens of millions of connections open.

base64 is used for sending the keys over already-encrypted https. The encryption is used for URL data sent over non-SSL connections and happens in a different file. They're using RC4 it looks like -- that's in url-crypto.js in the same directory I linked to earlier (calls the NSS crypto library for the actual encryption).

But now I'm wondering if this only applies to the URLs transmitted in active-checking mode.

I completely agree... in having seen IETab inside Firefox the security model for their anti-phishing software (both of them) is broken. It's definitely not safe (well, neither is the phishing software either, but you get the point).

Once every thirty minutes might seem like a lot, but really, I think it's probably just about right (maybe eventually it will be even not enough). The reason being the time between an email being sent and the list being updated is the name of the game. The major damage a phishing site does is within the first 24 hours. The faster you can get the list updated and propagated the better.

- RSnake
Gotta love it. http://ha.ckers.org

Hmmm link seems down now.

i'm interested in the crypto-class, did not saw it being called into the given script, maybe i overlooked something.

Thanks for explaining it more deeply dveditz!, still i believe that actively fetching the complete list (http) on the half hour is much, what happens when the list becomes huge? I would prefer that only the current site count would be fetched (http) and matched againt the local count. Like: 167 local, 170 remote, and then fetch the updates/updated list via https. Then you could make even more connections, like every 10 minutes and instead of the complete list you only fetch a few bytes.

I'd put some list checksum/hash value instead/added to site count, but generally it's an idea worth thinking of.

Really a diff, like lpilorz is talking about makes the most sense... only pull what you have to pull. But when the list becomes huge that's still going to be a problem if it changes regularly.

At that point we may have to take other measures like stopping it at the network level (not even routing the request if it's blacklisted) and taking it out of the client completely. Of course that will change the attack vector tremendously and people could use that as a DoS otherwise innocent domains so I don't recommend this unless you know the whole domain is bad rather than a URL on that domain.

- RSnake
Gotta love it. http://ha.ckers.org

Yeah, I really don't think there is any way to make this work 100%.

Also, with the "features" not working properly, people will be lead into a false sense of security regarding phishing. "Oh. The filter never told me it's phishy and it LOOKS like paypal." Ugh. I think when it comes to phishing, the only thing we can do is hope for SOME user education. :\

- Kyran

maybe i confused something.. but every half hour does FF download the full blacklist..

Quote
jungsonn
...still i believe that actively fetching the complete list (http) on the half hour is much, what happens when the list becomes huge?

or an update to what it already has..

Quote
tc
The list from the link above isn't complete (it's an update from what the client currently had). The lists are pruned.

?

-maluc

User education is basically proven not to work 100% (in small groups yes, but en masse, no). I've even seen examples where people who were IN the phishing industry get phished. Real conversation:

Director of anti-phishing: Hey, [RSnake] go here []

Ugh. This forum makes me say Ugh too much.
Web app security and all things that are related to it seems like an impossible battle. There are now over 100 million websites on the internet and I bet 90% of them have some sort of easily exploitable vulnerability....

Ugh.

- Kyran

Lets vote the dumb people off the planet.
.
.
.
.
.
.
.
.
Hey, where did everyone go?

-id

/me disappeared.

Don't forget our IRC: irc://irc.irchighway.net/#slackers
-WhiteAcid - your friendly, very lazy, web developer

Sl.ackers gone wild!

Pfft... sla.ckers! Sheesh! who in their right mind would register ackers.org? Lamerz. ;)

- RSnake
Gotta love it. http://ha.ckers.org

That should be made classic.

maluc: it downloads a diff. The client sends a url that includes arguments like "version=goog-white-domain:1:16" ("I have version 1.16") and the host responds with something like

[goog-white-domain 1.17 update]
+myspace.com 1

The next time the client sends version "1.17" for that list and gets back nothing, until the list is updated on the server. You can play with the link in jungsonn's original post in this thread. That returns quite a bit bigger list now, and if you modify the versions to match what you get back you get nothing (or you can back off a version or two and get a small list).

In my profile "urlclassifier2.sqlite" is over 5Mb and growing -- Ugh. The diff does include "-" lines so hopefully they keep a reasonable balance between protection and filling up user's disk.