Fighting SPAM: Identifying compromised email accounts

A compromised email account is one where spammers have determined someone's email password, and they're using the email account to send out spam email.

Various email servers have better and worse logging.  Depending on the server (qmail, postfix, sendmail) the logs may or may not let you directly correlate an outgoing spam email with the actual account that sent the email.

So, the following can be pretty useful.  It collects up all the IP addresses ($13 - the thirteenth field in the logfile, in this particular case) that each user has connected from, and prints out the accounts that are connecting from more than one IP.

awk '/LOGIN,/ {if (index(i[$12], $13) == 0) i[$12]=i[$12] " " $13} END {for(p in i) {print split(i[p], a, " ") " " p " " i[p]}}' maillog|sort -n|grep -v '^1 '

If you see an account for an individual, which is getting connections from dozens or hundreds of IP addresses, that's very possibly a compromised email account.

Note that an end-user with a smartphone will end up with a big bank of IPs connecting to check email.  They'll all have similar IP addresses in most cases.

Who's attacking my server?

Ssh rocks.  It's an awesome protocol.  But, it has an "achilles heel" - a default port.  Why?  Because services *should* have default ports according to some people.  Not me.

So, since ssh runs by default on port 22, people attack that port left and right, to see if they can bust into your server.

The program "lastb" reports the bad login attempts.  That's kind of useful but kind of crazy to work through.

The following summarizes the matter, reporting the IP addresses that have made over 1000 failed login attempts via ssh.

lastb -ai|awk '{print $10}'|sort|uniq -c|sort -n|egrep [0-9]{4}

If the output from the above doesn't convince you to move ssh to a custom port, such as 123456, nothing will.

For Plesk servers - which IPs are tearing up the server?

Plesk servers put all of the apache websites in /var/www/vhosts/websitename

Inside of those websitename directories there are directories for the website content (httpdocs) for the configuration files (conf) and for the error logs and access logs (statistics/logs)

So, whereas a "netstat" based check looks only at the CURRENT connections to the server, the following looks at the situation HISTORICALLY.

The following reports IP addresses that have made over 1000 requests today or yesterday.

cd /var/www/vhosts/; awk '{c[$1]++} END {for (i in c) {print c[i] " " i}}' */statistics/logs/access_log{,.processed} |sort -n|egrep [0-9]{4}

That returns counts of the number of hits by the worst offenders.

Who are they?  To find out, copy/paste the output of the above command as input into the following command, then hit "control-D".

for i in `awk '{print $2}'`; do echo "------------------ " $i " ---------------------"; whois $i|egrep -i netname\|orgname\|descr; done

How many web hits are coming from which IP addresses?

Sometimes apache's getting beat up, either by accident or on purpose.  Who's doing it?  What IP addresses are hitting web ports, and how heavily?

netstat -pant | sed -e 's/::ffff://g' | awk -F' *|:' ' $5 ~ /80|443/ {print  $6  } ' | sort | uniq -c | sort -n

The resulting list will show which IPs are hitting the websites the hardest at the bottom of the list.

The Dilemma

Security sucks.

There, I said it.  Security issues are the bane of our existence.

I keep running into scenarios where the root cause of a problem is, someone doesn't trust someone else.

That someone else is someone the first person fundamentally MUST trust, because without that trust, they might as well pack it in and go home.

"I don't trust my developer, so don't give him privileges on the server - just let him upload his code." (...code that takes people's credit card information and DOES WHATEVER HE WANTS WITH IT!)

"Help us be PCI compliant" (...what matters is that we pass these tests, not that our server is secure.)

I just got an email with some dude's entire firewall configuration.  He's trying to figure out if it's secure or not... and he just mailed it to an ENTIRE LIST of people - thousands, possibly, any number of whom might inadvertently be saving their email in some sort of insecure manner such that anyone on the planet could read it.

The dilemma?  How to DISCUSS security with the community, to help determine and implement best practices, without revealing your weaknesses.

Welcome to the 21st century.  If you don't want something to be known to the entire universe, don't decrypt it.

ipv6 day

The best word I can think of to describe today is frankly  ANTICLIMACTIC.

There was a lot of concern about what would happen on "International IPV6 Day".

Much of the world brought up their ipv6 interfaces today.

Were they prepared?  Were they aware of the issues? 

For example, had they considered what this guy's been up to:

http://www.youtube.com/watch?v=c7hq2q4jQYw

These next few weeks, like never before, bear close scrutiny.  Question EVERYTHING out of place.

It's possible the hackers aren't yet motivated to play in the ipv6 playground.  It's also possible they're waiting 'till people think everything is OK before launching assaults.

And, of course, it's possible they're quite busy, right now.