SpamBlock Plugin: IP Validation

[blog.brockha.us]

Hi Folks, I need some input.

Last year I added the "IP Validation" to the spamblock plugin of s9y for trackbacks/pingbacks. This method is one of the most effective ways for rejecting spam inside my blog. It successfully rejects 400-700 SPAM Trackbacks per day(!) in my blog.

How does this work in short: Track/Pingbacks are pointing to an URL of the originator of the TB/PB. The method looks up the IP of the TB/PB call and matches it with the IP of the handed URL. If they don't match, the tb/pb is rejected/moderated.

Bots are sending URLs for sites, they are trying to advertise. But the bot itself is not located on the advertised site, so the IP mismatch shows up here. Blogs on the other hand are sending tb/pb from the blog and are "advertising" the blog, so here the IPs match. Normally..

And here is the problem:
I found the first blog, that is sending normal trackbacks but fails the IP validation test. Another pingback, that fails the IP validation test, is the pingback produced by the microblogging service identi.ca.

So I updated the spamblock plugin to be able to add exclusions for ip validation. This works nice.

Here I need input:
As the IP validation is extremely effective in my blog and it is filtering out so many spam tb's, switching the method to "moderate" is no alternative for me. What I need is some automatically filled whitelist, as only one-tenth of a percent are false alerts.

Does someone has a good idea, how to get an automatically filled whitelist? Perhaps some kind of call back "confirmation" like the one we have for the comment subscription?

[onli]

We could easily add a event_hook which fires if an trackback is allowed manually and add this url to a whitelist. But I don't see the advantage, you would still have to moderate all trackbacks marked via IP-Validation, wouldn't you? You could only bypass all following valid trackpacks which would otherwise get caught via IP-Validation, which are only 1% as you say.

Ain't Akismet a better method to prevent trackback-spam?

PS: We are working at a bayes-filter for comments. If it works, maybe one could extent this to trackbacks. That's probably the only method to autonomously create automatic a whitelist (and blacklist).

[blog.brockha.us]

Akismet is good, no question. But it is relying on an external server and so for performance and other reasons ip validation is the more charming way in my eyes.

I'm searching for an automated solution for filling ip validation whitelists, without the need to moderate the rest. This means: There should be a new tb/pb state like "awaiting confirmation" or something. The blog would inform the trackback sender, that the trackback was rejected because of a suspicios IP situation. The sender should be able then, to verify he is no bot and the trackback should change then to accepted.

Something like this. Or some central server holding a whitelist of blogs that are configured in that imho strange way..

[onli]

A collision. Hope you notice the PS.

A central server is kind of a duplication of the akismet-method. Having to manually confirm your own trackback could be possible - send the correct error-code and include in the description a link to a formular. You probably don't even need an additional state, simply use "moderating" and approve it via the formular. But as I'm quite surprised the sender ain't noticed by now that his trackback ain't added I can't know for sure whether that's a good idea or not

[blog.brockha.us]

No, I didn't notice your P.S. Who is "we" in your P.S. btw? Are you involved in Akismet?

Yes, you are right, that a "whitelist server" for ip validation is some kind of duplication. I don't realy like this idea, too. It was just some kind of "brainstorming".

Hmm.. Perhaps there should be some global mechanism, allowing senders to fill that form for trackbacks set to "moderated". The mechanism could delete moderated comments then after X days.. But how do we get a secure way to seperate bots from blogs in that form..

And the blog owner noticed, that the trackback was rejected. But I think, he didn't see a possibility to change this, so he ignored the problem. If he would be informed via email (where to get the email from?) perhaps thinks would change there..

Hmm.. Not easy.. I really would like to stay with ip validation but I see, it is somewhat problematic. In my case it is not that problematic. It was the first blog having this problems, so I prefer denying all that massive spam automatically and loosing that one trackback instead of switching to moderation. But this is my personal blog situation..

[onli]

The "we" referred to the cooperation with "kleinerChemiker" in http://board.s9y.org/viewtopic.php?f=10&t=14824.

Didn't Dirk tell in the comments that he got a "added succesful" message? Am I misunderstanding http://www.deimeke.net/dirk/blog/index. ... html#c4484 ?

If Dirks Blog got problems with that, I'm on his server. So you probably know two Blogs with that configuration

>But how do we get a secure way to seperate bots from blogs in that form..
If necessary, we could add a captcha. As all that is about proving you're a human that ain't too far away.

[Don Chambers]

We also have the ability to use the TypePad antispam server: http://board.s9y.org/viewtopic.php?f=2&t=14504

I realize it too relies on an external server, but I thought I would point it out.

[rowi]

I don't use the IP validation at all. Just the URL check which validates whether there's a backlink from the trackbacking site. This catches approx. 300 spamback per day and as far as I know without any false positives.
But theres at least one site (IIRC Wordpress powered) from where I received trackbacks in the past which sends the trackback after every edit of the trackbacking article and they are not recognised as duplicates as they originate from different IP adresses.
The hosting site seems to be a cluster which is configured in a way that the nodes of the cluster do have valid (non-FC1918) IP adresses and don't perform NAT to the 'official Cluster-IP'.
Although I consider this setup broken I assume it's not uncommon in a world with *IX admins who use the mouse to administer their servers

I know this doesn't help with the problem of a whitelist server (which I consider a very ugly solution) but it's an aspect of the whole trackback/IP subject.

[blog.brockha.us]

At the moment I am near to abandoning from IP validation..

One of the main argument against it is: Mass Providers with multiple blog installations may fall into that trap. That doesn't seem to happen at my blog, but I guess only because I never get a trackback from such a blog installation.

Hmm. I will give Akismet a try and see, how that works..

That's sad, as I still think ip validation is a very cheap but powerfull technique.. Well.. Let's see.

Thanks for all the input, folks!

P.S.: Akismet is marking identi.ca pingbacks as spam, too..

[garvinhicking]

Hi!

How about letting the plugins that need whitelisting for API servers hook into the same spamblock routine and override the spamblocks "this is spam" message, so "whitelisting" the comment again?

I really dislike that a user needs to configure his spamblock plugin manually, I believe the plugin should take care of that?

Regards,
Garvin

[onli]

Regarding the bots who send spammy trackacks to other sites, won't the option "check for existing url" help most? Or do the spammers add a link from them to you?

Garvin, sorry, i wonder if someone else understands what you want to say, I don't Which plugins?

[blog.brockha.us]

Regarding the bots who send spammy trackacks to other sites, won't the option "check for existing url" help most? Or do the spammers add a link from them to you?

Some do, some don't.. (most indeed don't..)

I really dislike that a user needs to configure his spamblock plugin manually, I believe the plugin should take care of that?

Me too! This was the reason for this post. But the same as onli: Sorry, I don't really understood, what you wanted to point out. Could you clarify please?

[garvinhicking]

Hi onli!

Plugisn like the twitter plugin should add the IPs of identica.ca's trackback servers to its whitelist.

Regards,
Garvin