One article causes enormous load

Flominator · Post by **Flominator** » Tue Jan 13, 2009 9:53 pm

Hi there,

some days ago my hoster automatically blocked my whole webspace because the article http://flominator.ramselehof.de/index.p ... -Done.html was causing a lot of load.

I've been watching it for some months: this article receives unrealistic hits compared to the others and to GoogleAnalytics.

I thought, it was some bot, but the IPs are from a German cable provider and some random dialup providers. At least my hoster told me that.

Now they demand that I should solve this, but I don't know how.

What is different on this article?
Is there any way to log the accesses via s9y?
Would moving the article and using .htaccess to block the redirect help?
Any other suggestions?

Thanks in advance,

Flo

Post by **Don Chambers** » Wed Jan 14, 2009 3:10 am

I'm personally not sure what you want to fix here. It seems to me that you have an interesting article - perhaps it got bookmarked at some of the social bookmarking sites like technorati or digg. 3,000 hits does not seem excessive to me....

Post by **garvinhicking** » Wed Jan 14, 2009 10:46 am

Hi!

Ask the provider to give you proof: You need access logs of what gets requested how exactly, to back up what they expect from you.

It might not be the article itself, but trackback/comment spammers. Ask them if it's about TRAFFIC that is causing them trouble, or if it's simply Connection flooding (like DOS/DDOS).

In that case, they would need to deploy firewall or iptable rules to prevent DOS/DDOS - with PHP you can do nothing about this, this is a matter of "higher level" server setup.

Regards,
Garvin

ameo · Post by **ameo** » Wed Jan 14, 2009 6:18 pm

Don Chambers wrote:I'm personally not sure what you want to fix here. It seems to me that you have an interesting article - perhaps it got bookmarked at some of the social bookmarking sites like technorati or digg. 3,000 hits does not seem excessive to me....

Just as Don mentioned it's all about your host .. apparently your site started to get some attention on any social network and that caused some stress on your host servers

as for s9y installation, It can handle the DIGG effect with no problems.
i've been through the same issue with my OLD host so i know what you mean. now by deleting that post you've just lost many possible future readers

Flominator · Post by **Flominator** » Wed Jan 14, 2009 9:29 pm

Thanks for your hints. I asked the provider for further details and am waiting for there response.

Flominator · Post by **Flominator** » Thu Jan 15, 2009 12:28 pm

They give me an ultimatum until Saturday to fix it

They say that during yesterday evening there were 449 connections that looked like this one:

217.50.11.138 - - [14/Jan/2009:18:17:18 +0100] "GET
/index.php?/archives/75-Getting-Things-Done.html HTTP/1.1" 200 13935 "-"
"Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.0.5) Gecko/2008120122
(CK-PC-WELT) Firefox/3.0.5"

* is there anything not normal with these requests?
* is it possible to filter such requests via htaccess?

Post by **garvinhicking** » Thu Jan 15, 2009 12:41 pm

Hi!

449 connections during which timespan exactly? 449 connections for a whole evening is VERY LITTLE.

Which provider is this? I don't think they should be calling themselves a provider, if they think that 449 connections are much.

The listed request is a perfectly normal and valid request which looks like a human being. The IP is one of a dialup network. I am quite sure, that this is a request of a person who actually read your entry like a human is supposed to.

Regards,
Garvin

konus · Post by **konus** » Thu Jan 15, 2009 1:00 pm

449 Readers in a whole evening are nothing. 449 Readers per secound could cause some trouble.

Shutting down the whole webspace and giving your an ulitmatum to solve the problem, but not giving your information about it does not sound very professional.

You might be thinking about to move to a more professional webhoster. I am very satisfied with the support of my (german) webhoster and can recommend it to you.

Flominator · Post by **Flominator** » Tue Jan 20, 2009 12:55 pm

The thing is: I don't want to switch my host, because I'm really happy with it.

I talked to them today and they told that these 449 requests came from the same ip during 1:30 minutes. In GoogleAnalytics the post has about 20 views per month, so there is something strange going on.

They told me, that they think it might be some kind of link or speed check script addon within Firefox or something. The advice they gave me, was to mark the entry as draft for a month and then republish it.

Could that work? Does s9y send 404 headers in this case? Any better suggestions?

Post by **garvinhicking** » Tue Jan 20, 2009 1:18 pm

Hi!

A provider has technical means to reduce single IPs, if they have too many follow-up requests on port 80 through iptables or firewalls. This is a good means.

Changing anything in your blog is totally stupid, because that IP could simply request another article and check it multiple times. So your provider would actually recommend you to turn off all your articles?

Anyhow, even if you put down the article, the 449 hits would then be made to your blog startpage (with a 404 header). So that would cause the EXACT SAME traffic and CPU load.

I wouldn't be happy with that provider, if I were you.

Regards,
Garvin

judebert · Post by **judebert** » Tue Jan 20, 2009 4:34 pm

First off, I agree with Garvin: I'd be investigating new hosting services, and the provider is the one with the means to block specific IPs.

That said, you can block specific IPs in your .htaccess. If it's a choice between losing your blog and blocking one asshat, block his IP in your .htaccess.

Post by **garvinhicking** » Wed Jan 21, 2009 11:45 am

Hi!

Note that the mentioned IP is a dial-up IP. After ~12 hours, somenoe else gets this IP, and it might mean that you block visitors who behaved properly. Plus, the visitor with the previous IP gets a new IP after 12 hours, and then can easily hammer your site again.

Regards,
Garvin

Flominator · Post by **Flominator** » Thu Jan 22, 2009 7:56 pm

In the meantime it turned out that Garvin was right about the thing with the same traffic and the same load. (I expected that to happen, as well).

My provider told me that they've never had such a problem. They can't generally block all frequent requests, since some of their customers do have so much traffic (without load problems).

While writing this it occurred to me, that these other customers might have that much traffic and 449 hits per minute, but even their hits won't come from the same ip. Maybe that's a point for them to start from.

I have to stress again, that I really like my provider. They have a great support (well until now) and they let me do a lot of stuff. I'm running some tools there that really use a lot of resources, but nobody complained about it.

An idea I had after reading judebert's post:
What about some code @ index.php that lists the ips that visited the site during the last 5 mins. After n visits in n seconds let index.php write the IP into the .htaccess file ...

Post by **garvinhicking** » Fri Jan 23, 2009 9:32 am

Hi!

When s9y' index.php is called, it's already too late and the performance impact has already beendealt. DDOS must be prohibited on IP level,not PHP-applicationlevel.

Regards,
Garvin

Flominator · Post by **Flominator** » Sat Jan 24, 2009 9:09 am

I'm aware of that, but after 3 tries the htaccess file is written and then the ip is blocked on ip level