Camen Design Forum
It seems a particular kind of spam bot is becoming increasingly common here. The current anti-spam system is unbelievably effective for how stupid it is, avoiding hundreds of spams a day, however this one particular bot is doing the right thing to get through. Please ignore it for now and I will do what I can to stop it.
Test, test, test.
The forum was tend to be hit by a bot too smart to fall for the honeypot sometime. What’s the idea for a possible better solution?
One thing I read about not too long ago (though I can’t seem to find where) was a delay measuring system. You simply check how long it took someone to write a reply and go from there. This method assumes a bot is just filling as much information as possible in the least time needed.
I needed at least 10 seconds to write the above, and I would only be that fast if I would be copying the text off of something. You can assume even the shortest replies would take 20 seconds to formulate and write out by a human. Since they would probably need to read 1 reply earlier in the thread they are going to lose time there a bot would never lose.
This can be used for spam prevention. Take the timestamp the page was loaded in the browser and compare it to the timestamp on submit. Did the whole thing get written in under 10 seconds do not let it through. You could do a simple word count and add 10 seconds to it for every 30 words: ``floor(preg_match_all('#\w+#',$_POST['text'],$m)/30)*10``
The only problem with this is communicating the first timestamp to the submit page.
Any other options you are thinking of? I’m just hoping you’re not going to get us stuck to a CAPTCHA of some sorts.
(Further reading: smashingmagazine.com/…)
Firstly, there's no session and no client information is stored (like, in a db) server-side, so a huge amount of control most forums take for granted is not possible here. I would have to write the current time into a hidden field, which of course could be faked easily. It would at least be new and unique enough that automated bots would be fooled by it, but considering most forums have flood control, bots are already programmed to wait between posting.
Secondly, I don't want to impact any user's ability to post without unexpected things happening. Bolstering discussion is about removing all barriers, which is why this forum has no CAPTCHA. I didn't want anything to impede users, and forcing them to wait in order to make a reply is annoying and ineffective. I’m fighting spam, not users :)
The fix I’ve made is quite ingenious and involves a magical property of web browsers that few realise is present. I’ve never seen any forum ever use this feature. I won’t describe it, I will leave that up to people browsing the source (this change is not on GitHub yet).
I can confirm my little trick has worked and it's killed off that spam-bot that has been getting through since day 1. I now need to work on better delete functionality to clean things up in an easier manner.
Most forums have flood control, yes, that’s why I wasn’t talking about time between posts (which is, as you said, impossible due to not having a real user database) but only about time used to write a post.
“I would have to write the current time into a hidden field, which of course could be faked easily.” Yep, exactly my thoughts, which is why I said “communicating the first timestamp to the submit page” would be problematic. Good to see you understood my thoughts there :)
The only way to send along a timestamp would be by obscuring the data. Something I’m not a big fan of but which should stop spammers anyway. With spam it’s about as many posts in as short time as possible. For example, you could actually dress up the timestamp as an email address inside your honeypot after appending a control number. Any value in the honeypot that can’t be turned back into a timestamp with a matching control can be assumed coming from a bot and will be stopped. If the bot was too smart to fall for the honeypot you will move on to checking the timestamp. If the bot just pasted a lot of text but submitted within 10 seconds it can be stopped again.
This would be unique enough to stop a lot of spam. Only two types of bots will now go through. The first are those perfectly emulating human behaviour (reply times) while also being smart enough not to fill in all fields. The second are those that will reverse engineer the timestamp and spoof it, these are less than 1% of all spam bots because it involves developing spamming software specifically for targeting these forums.
Both these cases will be almost impossible to stop with your (as of yet) undisclosed new trick too. The first would include running in virtual browsers to fake human behaviour and the second would simply be updated again.
(It’s fun to think about ways to trick computers. It’s a good thing we’re not up against Watson here! :D)
Looking forward to see this new trick of yours pop-up on GitHub.
Kroc, I just finished dinner and saw your new way of testing for spam bots. I can probably tell you why not everyone is testing for it this way: not all browsers will be sending that information.
Just now I tried to comment on this thread using Lynx, my reply was blocked.
A way to fix this would be to add a name attribute to the input. This will at least make Lynx, emacs and the likes send the right information.
Annoying :P Adding a name attribute won't work as Lynx won't send X&Y. Any suggestions?
First of all, note that it is the HTML5 spec that defines the following:
“If the user activates the control without explicitly selecting a coordinate, then the coordinate (0,0) must be assumed.”
Previously it was only stated:
“When a pointing device is used to click on the image, the form is submitted and the click coordinates passed to the server.”
This is probably why text browsers are allowed (in a loose sense of the word) to not send any X or Y information. I would guess this goes for mobile browsers as well, but have none to test.
htmlhelp.com/… has a compatibility table with old browsers and the coordinate system.
I myself ran a test with the following code:
~~~ HTML ~~~>
<form method="post">
<label for="submit">Submit
<input name="a" type="image" src="submit.png" id="submit">
</label>
</form>
<~~~
With this code I was able to get an array in ´´$_POST´´ containing ´´a_x´´ and ´´a_y´´ items from Lynx so it should work. You will just have to test for ´´name_[xy]´´ and possible ´´name.[xy]´´ as mentioned by the WDG although I have seen no evidence of and values with a dot being send by the browser.
I do applaud you for the creative method, and it would be really interesting to see this actually block spam!
You are right, and I have to applaud you for finding the workaround in Lynx, that is awesome. I think this is going to be pretty darn effective for now.
Yeah, I was about to say, delay monitoring would be easy if a hidden input value of the timestamp was used. It can be easily spoofed, but if you wrote your own very simple function to encrypt/obfuscate the original timestamp like Martijn suggested, most spam bots probably wouldn't be able to figure that out. But maybe I shouldn't underestimate spam bots.
Spam bots may request a page first, in order to be assigned a session ID, and then wait a minute before posting content in order to get around common minimum wait times. -- If an anti-spam feature is common in other forums, there will be bots programmed against it. Using new techniques such as I’ve added will be the most effective option. It’s blocking hundreds of spams a day to this forum and not a CAPTCHA or login in sight.
Translating Kroc’s reply: if it ain't broke, don't fix it. ;-)
It could be cool to have some kind of counter keeping track of how many posts are killed due to the honeypot and how many due to this new method. A simple TXT file of sorts.
I’m logging POSTs in the background in order to test the feature and to look out for attacks. I could rig up a fail-counter, but it's not exactly vital :P
I use the timestamp method Martijn is suggestion for all my contact forms. But what I do is check that the timestamp isn't to far in the past. I assume some spam bots crawl the form one time and then repeatedly send different body's with the same hidden values attached. That way the bot would have to re-crawl the form every 5 minutes, or be unable to send any more spam after that period. The prevent really smart, or human guided bot from detecting the timestamp and changing it to the current time, I (also) add a salted hash of the timestamp. When the form is posted I check that the salted hash of the send timestamp matches the send hash. If it matches I check if the timestamp is recent. If so I accept the message.
I hope someone finds this method useful, because I hate people just throwing captcha's around. Like captcha's are the only solution!
I like the idea of sending the timestamp twice, with one being hashed. The only direct problem with that would be NoNonsense Forum being completely OpenSource, the hashing algorithm would be available. It means a hidden SALT will have to be used, which in Kroc’s case means he has to work on 2 different versions of the forum code: one for use here with a secret SALT and one for sharing with a public SALT. The latter might then be used wrong by others which makes them vulnerable to spam bots using the public SALT.
As for a way coders could protect their timestamp I find it absolutely brilliant! Much better a solution than my obscuring the data.
Of course the forum should never check the timestamp for being too old, unless this is set to an hour. Some threads might take long to read and some replies even longer to write.
You are always trading off of probabilities when it comes to fighting spam. How likely is it that someone will specifically modify a bot to work with my forum? Almost certainly unlikely unless NNF sees extremely wide deployment, which is also extremely unlikely since it doesn't have the features (or protection) most people would want.
Whilst I could use some piece of server-specific information to salt hashes (thus making reading the code ineffective), it wouldn't then prevent anybody from coding around the whole principle once they know what the keys are for. They could just request a fresh page each time, before submitting the page, and I expect many bots do already do this as some forums must block posts that don't have a session assigned to a logged-in user.
What about:
$salt=implode(posix_uname());
I wouldn't use it to secure banking information, but for fighting spam, I think it suffices.
If the timestamp is to old I just notify the user that I suspect it is spam, and say they should repost the message. Of course already refilling the field with the user generated content. This would however clash with the requirement of the timestamp not being to recent.
Your friendly neighbourhood moderators: Kroc, Impressed, Martijn