'How is this access getting through without getting a 403

I have a standard Apache 2 setup on an Ubuntu Server. I basically block almost everything from the outside world, but it is still open to friends and family for some basic stuff. Anyway, I frequently notice entries like these (generalizations for common sense) in my access logs:

157.245.70.127 - - [every-day] "GET /ab2g HTTP/1.1" 400 497 "-" "-"
157.245.70.127 - - [every-day] "GET /ab2h HTTP/1.1" 400 497 "-" "-"
xxx.xxx.xxx.xxx - - [sometimes] "POST /cgi-bin/.%2e/.%2e/.%2e/.%2e/bin/sh HTTP/1.1" 400 487 "-" "Mozilla/5.0 the rest or the user-agent..."

Nothing scary, but, I wanted to just force them to a 403. Here is a generalized excerpt from the config:

<Directory ~ "/var/www/.*">
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
    # A pure global block of anything not supported in my server
    Include /etc/apache2/IPBlackList.conf
    RewriteEngine on
    RewriteCond %{REQUEST_URI} ^.*?(ab2[gh]|and other things).*$ [OR]
    RewriteCond %{HTTP_USER_AGENT} (^-?$|and other things) [OR]
    RewriteCond %{REQUEST_METHOD} CONNECT
    RewriteRule "" "-" [F]
</Directory>

This works for every other case. That is, where ever you see "and other things", all those things result in a 403, regardless if the IP is blocked or not. This is the goal, but for some entries, I get pretty much a 400. Not bad, but, it's getting on my nerves.

I have expressly put the 157.245.70.127 in my IP block list. For all other IP's in the block list, it works just fine.

For the blank user agent, works virtually every time, but that one gets through every single time.

In other words... that 157 IP is getting through the IP block, the request URI block, and the blank user agent block.

The "cgi-bin" ones come from different IPs and have varying URIs, and, sometimes they get a 403, but other times not. Generally speaking, when I block the IP it works, but... why isn't the HTTP_POST not working in some cases?

What am I missing??? How can I resolve???

mod-rewrite


Solution 1:[1]

On my apache' servers, these "400" error responses are usually sent in response to HTTP/1.1 requests that have no HOST header. This error check precludes other processing, so REWRITE and SETENVIF have no effect.

FWIW - The only way I've found to intercept them is from the LOG output stream, by sending to an external program which looks for and processes 400 error marked packets. But they are already processed by that time.

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 CER

Related Questions