Cenzic 232 Patent
Paid Advertising
sla.ckers.org is
ha.ckers sla.cking
Sla.ckers.org
Whether this is about ha.ckers.org, sla.ckers.org or some other project you are interested in or want to talk about, throw it in here to get feedback. 
Go to Topic: PreviousNext
Go to: Forum ListMessage ListNew TopicSearchLog In
Stopping Robots
Posted by: jungsonn
Date: February 06, 2007 12:03PM

Okay this is an idea that is in it's infancy and I'm not sure what to think about it yet and I ask your ideas about it.

So there are robots around which can SPAM, harvest, do al sorts of malicious stuff and can act like a real user. Fine, we know this. But I think that some methods require real user activity, and so I though about some basic JavaScript.

Let's say someone wants to comment on a blog:

They input their comment, and when he/she clicks the submit button, I throw a javascript Alert(are you a robot?) clicking yes/no, or Prompt to type in a phrase or anything like it. Most robots (if not all) tend to hang on such things because they can't click on the alert or prompt.

But, that is what I think.

now I want to know, is this really the case? can't robots click away alerts? and moreover can they fill in javascript prompts?



Edited 1 time(s). Last edit at 02/06/2007 12:03PM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Date: February 06, 2007 12:14PM

What I'd do is have some hidden form fields which appear like they need to be filled in. The robots will probably automatically populate all the form fields, whereas to a human it would remain hidden and unchanged. Js would probably work pretty well, too, but that would mean the users would have to be able to execute js to access your form.

Options: ReplyQuote
Re: Stopping Robots
Posted by: Kyran
Date: February 06, 2007 12:24PM

it would be better to style them hidden, instead of using the hidden type attribute, some robots might be smart enough to leave those alone.

- Kyran

Options: ReplyQuote
Re: Stopping Robots
Posted by: WhiteAcid
Date: February 06, 2007 12:35PM

Jungsonn: The bots simply wouldn't execute the javascript.
Following the idea though, you could not have a submit button but instead have a normal button which submits the data via AJAX. This would break the functionality for non-JS people though and to automate the process someone could still make a program to just send the packet the AJAX object creates.

SirNotAppearingOnThisForum: That's not always the case. If I were to make a bot which for instance posts comments on wordpress blogs then I'l make it so that it only fills in the fields it has to leaving the rest along. If you rename the default name attribute of the critical fields though, that'll stop the automated bots.

Other than that CAPTCHA is pretty much all there is.

I've thought about having something like this in JS on the page:
a = ranomd number
b = random number
c = a+b
submit c as part of form. The script then checks if c is valid (it has copies of a and b in the sessian variables)
That way only people with JS capable browsers can submit the form. The problem is that again it kills the functionality for non-JSers. Furthermore it's actually quite trivial for a bot to implement a JS engine.

If we could use the results from http://images.google.com/imagelabeler/ we could perhaps make quite a good CAPTCHA thing.

Don't forget our IRC: irc://irc.irchighway.net/#slackers
-WhiteAcid - your friendly, very lazy, web developer

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 06, 2007 12:35PM

Yeah I know about those things, but some can analys your page and really if it's a huge page or forum they first audit it's security manually and then feed the vector into the robot to handle it further..

What I really want to know if a robot can click away an alert or fill in a prompt instance asking you to fill in some char, to my knowledge they can't do that.(or the gross of them) maybe they would be able to click an alert away, but not the prompt. anyway I really don't know, so if anyone got some insights on this, I'dd like to hear.

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 06, 2007 12:37PM

Posted at the same time :)

Yeah whiteacid, my idea is to move away from captcha images which are sometimes way to hard to read.

Options: ReplyQuote
Re: Stopping Robots
Posted by: nEUrOO
Date: February 06, 2007 08:42PM

I pretty much had the same thought as you some days ago: http://rgaucher.info/b/index.php/post/2007/01/31/How-to-prevent-spammers-bot
But this is strongly linked to the fact that robots don't handle JavaScript and I'm really not sure about it (since it's not that much difficult to plug spidermonkey etc.).

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 07, 2007 02:44AM

Yes looks nice, but that's adding fields to the form, and not exactly what I want.

I'll go make some examples later this day, now I think it would be best to leave the whole comment form alone, but to launch a user interaction after submitting, so it's only stored with a single click on an alert or prompt. I'll see, thought it would be cool. The main reason is portability of this function, I want to be able to use it in multiple existing and new applications but without edditing form data and stuff. because with it, you also have to protect the actual processing file, most bots go directly to that file with stored form variables as vector, and skip the whole form.



Edited 3 time(s). Last edit at 02/07/2007 02:47AM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: _sniff
Date: February 07, 2007 05:50AM

@jungsonn
i m working on robots from last cpl of yrs and wud really like to test your example pages, cud create/write robot to test whther i can crawl them or not.
waiting for your example pages...

- sn|ff

Options: ReplyQuote
Re: Stopping Robots
Posted by: nEUrOO
Date: February 07, 2007 12:02PM

Do you guys know a website or something with description or state of the art of the spam robots?

nEUrOO -- http://rgaucher.info -- http://twitter.com/rgaucher

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 07, 2007 04:04PM

Okay, I did have tons of work today, so I'm a bit late I made a quicky:

http://www.redsonn.com/hi/bla.php

haven't tested it fully, but you get the point I guess.

I edited the link, was the wrong one :)



Edited 1 time(s). Last edit at 02/07/2007 04:49PM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 08, 2007 10:42AM

Okay, so I had a second option/idea with cookies:
http://www.jungsonnstudios.com/blog/?i=115&bin=1110011

Maybe I'll combine the two, i'm not sure yet.



Edited 1 time(s). Last edit at 02/09/2007 09:19AM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 08, 2007 06:40PM

I just post it here to discuss it, I thought about some variable that must be accesable by javaScript & PHP together (that's the hard part), while one cannot call PHP functions in JavaScript, I can echo stuff into JavaScript. So I first generate a session and echo it into a JavaScript who builds a cookie around it, next page; the cookie is pulled read<->matched against session, then the comment is stored and cookie deleted and session destroyed.

But think I don't have to explain what's happening:

Post.php
<?
session_start();
$_SESSION['sentinel'] = md5(rand(1,9999999));
$send = $_SESSION['sentinel'];
?>

<script>
function createCookie(name,value) {
var date = new Date();
date.setTime(date.getTime()+(1*24*60*60*1000));
var expires = "; expires="+date.toGMTString();
document.cookie = name+"="+value+expires+"; path=/";
}
createCookie('postcomment','<?=$send;?>');
</script>
<noscript>
This action requires JavaScript.
</noscript>

Send.php - This is the script which handles the comment.
<?
# V2.0 code fixes.
error_reporting(0);
session_start();
# strlen fix
if(isset($_COOKIE['postcomment']) && strlen($_SESSION['sentinel']) > 0) {
# Added a strict not equal to
if($_COOKIE['postcomment'] !== $_SESSION['sentinel']) {
session_destroy();
setcookie("postcomment", "", time() - 8600);
die('Not equal!');
}
echo "Okay! I'll insert your comment!";
# request form variables & call the db query here.
session_destroy();
setcookie("postcomment", "", time() - 8600);
die('Ok');
} else {
session_destroy();
setcookie("postcomment", "", time() - 8600);
die('No cookie! and/or wrong session');
}

?>

Now my only fear was that some human start to load the cookiename into its bot by hand, and then sends the bot to the first page, and quickly skips to the second one. But I thought about it some more; This can't be done because it needs to go to the first page to get the correct session, if it's feeded into the bot previously, it will rehash the cookie and the bot can't do a thing. At least this is my theory around it.

Actually it's a CAPTCHA but then stored into a cookie, with no user interferance.

Like to hear the comments/suggestions, thanks for listening :)



Edited 2 time(s). Last edit at 02/09/2007 09:21AM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: rsnake
Date: February 09, 2007 08:39PM

Not exactly accessible. How do people with text readers use this?

- RSnake
Gotta love it. http://ha.ckers.org

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 10, 2007 08:11AM

They still can read. maybe not write :)

But it's a flimsy requirement to accept cookies and to have javascript.
I've been into discussions a few years on the use of javascript,
but fact remains that the gross of surfers have it turned on.

I could make it 2-way.
In the noscript tag I could paste a link which goes to an original captcha
image version, solving this issue for those who have turned of javascript
and won't accept cookies.

My intention is to move away from CAPTCHA images as much as possible.

As for the accessible; there's always a gordian knot somewhere.



Edited 2 time(s). Last edit at 02/10/2007 08:14AM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: _sniff
Date: February 12, 2007 11:01PM

@jungsonn
i m back! i am able to crawl your site (atlst the example pages you posted), now cud you pls tell me if you are keeping them in any db?
i dont want to fill all your db with my comments :)

edit: corrcted typo (btw all my words are similer to typo) lol !

- sn|ff



Edited 1 time(s). Last edit at 02/12/2007 11:03PM by _sniff.

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 13, 2007 05:01AM

No they are not stored :)
But that example on my test server, isn't that great I realized a few days ago.

But can you bypass the above script?

Options: ReplyQuote
Re: Stopping Robots
Posted by: _sniff
Date: February 13, 2007 07:13AM

yeah jungsonn, i am able to bypass your functionality!
nd successfully crawled 'Right' pages.

bt nyway.. it is very very nice use of scripts :)

- sn|ff

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 13, 2007 11:36AM

erm.. you mean the online examples on my test server?
or the posted script above in the forum? your comment is a little vague.

makes a difference because the are not the same :)

If you can go around it, then you must read the javascript source I guess?
otherwise it's impossible to go around it. And if you do can you tell me how you did it? Also, can you tell me how much work it took? and did you needed to create a custom vector based on a fixed cookiename?

I need some real data you know, to be sure.

Hope to hear from you.

Then I can modify it, by adding some random entropy for the cookie name



Edited 1 time(s). Last edit at 02/13/2007 11:52AM by jungsonn.

Options: ReplyQuote
Re: Stopping Robots
Posted by: _sniff
Date: February 15, 2007 01:44AM

ummm,... i was talking abt example running on your server (i didnt know they are not same, nvr tried to know) :( --- lazy me !

cud you pls paste those scripts on your server, will try to bypass tht!
nd let you know wateve i get from your server (right or wrong) :)

- sn|ff

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 15, 2007 09:47AM

I will do that soon, along with a better example i'm working on. :) I'll post it here hwen i find the time, thanks for your time to test it for me. I appreciate that.

Options: ReplyQuote
Re: Stopping Robots
Posted by: psifertex
Date: February 15, 2007 11:54AM

I did something similar to this a couple of years ago to stop comment spam on my blog. It was much more basic, but built around the same concept -- I just used document.write to write out the actual comment form. The url was in a couple of pieces that were then just concatenated together. Really simple, really basic, only worked if javascript was turned on, but it was for my personal blog and not that many folks commented, so I didn't have to worry about the accessibility issues. An old post showing the code is at:

http://wantingseed.com/sprout/2005/11/17/killing-comment-spam-dead/

It really could be done much simpler too -- any human who actually looks at the code can figure it out right away, but most of the spam bots don't bother to have javascript parsers, they're just looking for form submits that look familiar and going from there.

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: February 15, 2007 03:07PM

Yeah that looks nice only one big problem with your example though: a human can figure out all variables, make a large URI from it, and feed it to the bots.

that's why it's not the same as my script. because you first have to aquire a session first on the comment form which only can be set through javascript. This renders spamming next to impossible, because a bot will never figure this out.

Options: ReplyQuote
Re: Stopping Robots
Posted by: backbone
Date: March 08, 2007 02:34AM

also somebody could make a bot that strips out the javascript from the page... so the prompt wouldn't appear anymore...

but i don't think people create such bots... also I don't know if there are still people who use ancient browsers with no js suport...

Options: ReplyQuote
Re: Stopping Robots
Posted by: jungsonn
Date: March 08, 2007 08:12AM

The version I use on my blog does actually work, all the bots can't figure it out. And boy do they try it... ^^

But there is a weakness in this version also, a reader of my blog posted me a complete PHP script to bypass my script. It works, granted. But it requires a custombuild regex to make it work.

I'm now working on it to throw in a little more randomness whereas it would be impossible to write a regex for. Until now, I think I'm very safe with my anti robotscript.

Options: ReplyQuote
Re: Stopping Robots
Posted by: rsnake
Date: March 09, 2007 04:20PM

Actually I regularly use browsers with no JS support (this one actually). It's a much faster way to surf the web and gets around all that annoying spyware (Adwords/Google Analytics).

- RSnake
Gotta love it. http://ha.ckers.org

Options: ReplyQuote
Re: Stopping Robots
Date: March 10, 2007 05:37AM

you could also give this method a shot

Options: ReplyQuote


Sorry, only registered users may post in this forum.