Comments on: Rate limiting http traffic (mod_evasive and iptables)

By: Bob

Bob — Mon, 27 Jun 2011 17:45:54 +0000

hitcount can’t be higher than 20 using the default iptables settings. if you set it higher, the rule never gets triggered.

also, i think you want to reverse the order of the rules. otherwise, the set gets triggered, but the update doesn’t.

By: Mike

Mike — Sat, 05 Feb 2011 16:29:43 +0000

Did you ever do a write up on your PHP solution? I’m having a similar issue with scrapers crawling my site too quickly. I’m looking at mod_evasive, but have the same concerns that you do. Thanks!

By: David Goodwin

David Goodwin — Thu, 04 Feb 2010 22:31:04 +0000

Obviously – but there’s still the problem that some pages need rate limiting and others don’t. I don’t care if someone requests the home page a zillion times in one minute – it’s effectively static (well, PHP, but no DB calls). I do care if someone starts going through each business one after the other leeching their details.

Yes – I could split images off onto another domain – unfortunately the code base is horrible, and it would have to be enforced via e.g. mod_rewrite. I don’t think we’re yet at the position of needing to do this.

I’ve written a PHP solution, which i’ll soon post here, which does at least allow for a friendly ‘error’ page and a captcha to fill in which makes it more user friendly, and less likely to fsck up if there are a lot of users behind one proxy (for example).

By: Alex Howells

Alex Howells — Thu, 04 Feb 2010 20:54:53 +0000

Dish up the images from another subdomain, for example images.client.com – they are static content and you should then be able to build mod_evasive settings so that they only kick in on requess to the main application.

This sort of architecture would help you anyway in future, when you look to putting relatively static assets like images and Javascript into a CDN or something?

Finally keeping them on a different VirtualHost (or subdomain) would allow a fairly simple load balancer to distinguish between them: thus you could build $solution to employ 5-6 beefy ‘application’ servers and just a couple of other boxes for ‘static’ content, and potentially deploy different caching, software (Apache vs. nginx) and tuning to make it all funky and fast.

Then you can apply mod_evasive settings

By: Jonkarra

Jonkarra — Tue, 02 Feb 2010 15:43:19 +0000

Well checkpoint has a lot of this functionality built in, although it can be a pain to get it working just the way you want, as some of the application security features are a little buggy(or plain just dont work). Although for what you want its pretty solid. Although it is proprietry and damn expensive so doubt its your sort of thing