Hacks and Gadgets by HJL

Here’s a handy way to check out what sites are associated with an IP address:

A2B has recently implemented ping blocking to reduce the volume of spam hitting their geotagged search engine. They publish a current list of blocked IP addresses, and you can also submit queries to see what sites have been seen recently at a given IP address.

Here’s the list of sites from an identified spam IP address (70.86.36.194). They also list the blogs associated with the IP address.

Many of the blocked sites are submitting 6000 to 8000 pings per day. A2B reduced their ping traffic from 27GB to 6GB daily by blocking the 112 top spam blog IP addresses turning up in their server logs.

The lookup service is a little similar to some of the reverse DNS services that are available (IP to domain name), but it appears to have a shorter time horizon. A quick check on some sites I know to be on shared hosting services only shows some of the sites that are at their respective IP addresses.

via SpamHuntress, see also comments there from Sam Critchely of A2B.

After a few weeks of relative quiet on the web spam front over the holidays, I see that there’s been a huge uptick in new spam referrers turning up in the server logs here. I suppose the spam operators have come back from the holidays as well.

Referrer spam is fairly pointless on my site, since I don’t publish automatic lists of site referrers. However, it can chew up a lot of bandwidth and CPU cycles.

The first line of defense here is using .htaccess to completely block the top level domains and IP addresses that are known to be problematic. Additional filtering is done by individual applications such as Wordpress (Akismet et al) but if you block unwanted traffic with .htaccess you don’t even generate the web page. This is of particular interest to Dreamhost users, where they seem to have started tracking CPU use more closely.

The method shown here will work on Dreamhost and most other hosted services that allow user defined .htaccess files. Here’s the general approach. In .htaccess, we define the “bad referrer” patterns, which will match the incoming HTTP Referer field. If the BadReferrer variable is set, then we block that request, sending back an HTTP 403 response.

SetEnvIfNoCase Referer ".*.baddomain.com" BadReferrer
SetEnvIfNoCase Referer ".*anotherbaddomain.com" BadReferrer

order deny,allow
deny from env=BadReferrer

The patterns used in .htaccess are regular expressions, which can be a little hard to read. It’s important to precede the “.” in domain names with a “\”. It’s a good idea to save a working copy of .htaccess before you start editing it, since you can make your entire site inaccessible if you accidentally create a pattern that blocks all referrers rather than the unwanted ones. The first case shown above will block “www.baddomain.com”, while the second will block “anotherbaddomain.com”. See the Apache documentation for more complete information on using .htaccess.

Here’s the htaccess blocklist rules I’m currently using to keep out spam domain referrals. The patterns don’t exactly match real domains in all cases, as wildcards are used occasionally to catch variations of the same domain name. (Warning — many of the domain names are predictably offensive.)

If have specific IP addresses that you want to block, you can also define that here:

deny from 10.0.0.1

Blocking individual IP addresses is less useful, since most spam traffic seems to originate from networks of hijacked PCs, with IP addresses all over the world that change frequently. That said, there also seem to be individual spammers who are running spamming applications on their personal computers, which are pretty easy to block in .htaccess.

Here my previous notes on referrer spam:

• Temporary Fix for Referrer Spam
• Referrer Spam IP Blocklist
• Blocking Referrer Spam

I have to look this up from time to time, usually just after installing a new Linux system intended for use as a test server. There are GUI tools for configuring the firewall, but I usually leave the entire windowing system uninstalled.

By default, many Linux distributions will start up sshd and httpd, but you won’t be able to connect to them over the network, since the ports will be blocked by iptables. The examples below are for Red Hat or CentOS (which is basically Red Hat), and may vary slightly betweenl distributions.

To see the current iptables firewall rules, use iptables –list (that’s two dashes):

[root]# iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
RH-Firewall-1-INPUT  all  --  anywhere             anywhere

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere            icmp any
ACCEPT     ipv6-crypt--  anywhere             anywhere
ACCEPT     ipv6-auth--  anywhere             anywhere
ACCEPT     udp  --  anywhere             224.0.0.251         udp dpt:5353
ACCEPT     udp  --  anywhere             anywhere            udp dpt:ipp
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited

The part we’re interested in is the “RH-Firewall-1-INPUT” list. We need to add two rules, which will accept incoming TCP connections to port 22 (sshd) and port 80 (httpd).

[root]# iptables -I RH-Firewall-1-INPUT 3 -p tcp -m tcp --dport 22 --tcp-flags SYN,RST,ACK SYN -j ACCEPT

[root]# iptables -I RH-Firewall-1-INPUT 3 -p tcp -m tcp --dport 80 --tcp-flags SYN,RST,ACK SYN -j ACCEPT

The additions will now appear in the output of iptables –list:

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination
ACCEPT     all  --  anywhere             anywhere
ACCEPT     icmp --  anywhere             anywhere            icmp any
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:http flags:SYN,RST,ACK/SYN
ACCEPT     tcp  --  anywhere             anywhere            tcp dpt:ssh flags:SYN,RST,ACK/SYN
ACCEPT     ipv6-crypt--  anywhere             anywhere
ACCEPT     ipv6-auth--  anywhere             anywhere
ACCEPT     udp  --  anywhere             224.0.0.251         udp dpt:5353
ACCEPT     udp  --  anywhere             anywhere            udp dpt:ipp
ACCEPT     all  --  anywhere             anywhere            state RELATED,ESTABLISHED
REJECT     all  --  anywhere             anywhere            reject-with icmp-host-prohibited

You can now connect to the server over the network. Unfortunately, this will only last until it’s rebooted. To save the iptables configuration, do something like:

/etc/init.d/iptables save

This will update the saved configuration in /etc/sysconfig/iptables, which will be used to initialize iptables at boot time.