Analysing nefarious ssh access attempts
Maintaining my own server has taught me a lot of things over the past years—among other things, I increased my respect for those brave system administrators that have to deal with more than one server, always making sure that no bad stuff happens to its contents. In this article, I want to offer you a brief look behind the veil and show you the strange things my poor server has to deal with on a daily basis.
Analysing IP addresses
More precisely, we are looking at potentially nefarious
attempts. I consider a connection attempt to be nefarious when an
invalid password is used to access the server. Since all my users employ
public keys to access the server most of the time, this is a pretty
solid criterion and easy to evaluate. First, let’s gather some
evidence. All the authorization attempts are logged in
/var/log/auth.log (or its archived versions). This file has
a very simple structure:
Feb 25 00:07:21 myrddin sshd: Failed password for root from X.X.X.X port 493 ssh2 Feb 25 00:07:28 myrddin sshd: Invalid user ftpadmin from X.X.X.X port 567
As you can see, I redacted the IP addresses to protect the, well,
probably innocent. Moreover, we observe that the error message is
slightly different, depending on whether someone tried to access the
server using an existing user, or an invalid one. Some Python
magic (see the link at the end of the article) results in an
enumeration of all connection requests, sorted by IP addresses. Out of
983892 total log entries (spanning roughly one month), 167911,
i.e. roughly 17% of them, concern a potentially nefarious
connection attempt. In this calculation, the subsequent messages of
sshd complaining about an invalid password or an invalid user have
been ignored, so the actual percentage is even higher. This is somewhat
My surprise was even larger when I checked which IP addresses were
responsible for these requests. It turns out that three unique IP
addresses are responsible for more than 25% of all invalid password
requests. I will not list them here for privacy reasons—all of
them belong to ISPs from China. Two of them are under the auspices
of China Telecom, while
the third one is registered to belong to China Unicom.
While both ISPs list a contact e-mail address for reporting abuse,
I doubt that contacting them will prove to be effective. If these
access attempts do not stop, I will try writing them a nice letter.
In the meantime, why not collect more data?
has some nice OS detection capabilities:
sudo nmap -O -Pn X.X.X.X
For the two China Telecom IP addresses, only a single port is open to
25. Despite the port number, this appears to contain an
ssh server, to which I probably will not have access.
wildly that this might be a
FreeBSD 6.2-RELEASE system, but it
acknowledges that the OS results might be somewhat off. The China Unicom
IP address, by contrast, proves to be more interesting:
Nmap scan report for X.X.X.X Host is up (0.18s latency). Not shown: 977 closed ports PORT STATE SERVICE 22/tcp open ssh 25/tcp open smtp 80/tcp filtered http 111/tcp filtered rpcbind 135/tcp filtered msrpc 139/tcp filtered netbios-ssn 199/tcp filtered smux 445/tcp filtered microsoft-ds 593/tcp filtered http-rpc-epmap 901/tcp filtered samba-swat 1025/tcp filtered NFS-or-IIS 1034/tcp filtered zincite-a 1068/tcp filtered instl_bootc 1434/tcp filtered ms-sql-m 3128/tcp filtered squid-http 4444/tcp filtered krb524 5800/tcp filtered vnc-http 5900/tcp filtered vnc 6006/tcp open X11:6 6129/tcp filtered unknown 6667/tcp filtered irc 6669/tcp filtered irc 8080/tcp filtered http-proxy Device type: general purpose|WAP|storage-misc|broadband router Running (JUST GUESSING): Linux 3.X|4.X|2.6.X|2.4.X (95%), Asus embedded (92%), HP embedded (91%) OS CPE: cpe:/o:linux:linux_kernel:3 cpe:/o:linux:linux_kernel:4 cpe:/o:linux:linux_kernel cpe:/h:asus:rt-ac66u cpe:/h:hp:p2000_g3 cpe:/o:linux:linux_kernel:3.4 cpe:/o:linux:linux_kernel:2.6.22 cpe:/o:linux:linux_kernel:2.4 Aggressive OS guesses: Linux 3.10 - 4.11 (95%), Linux 3.13 (95%), Linux 3.13 or 4.2 (95%), Linux 4.2 (95%), Linux 4.4 (95%), Linux 3.16 (94%), Linux 3.16 - 4.6 (94%), Linux 3.12 (93%), Linux 3.2 - 4.9 (93%), Linux 3.8 - 3.11 (93%) No exact OS matches for host (test conditions non-ideal). Network Distance: 16 hops OS detection performed. Please report any incorrect results at https://nmap.org/submit/ . Nmap done: 1 IP address (1 host up) scanned in 58.88 seconds
Again, no exact matches there, but the number of open and filtered ports
is interesting. The SSH server responds with
OpenSSH_6.9p1 Ubuntu-2 pat
and apparently permits logins via certificate and passwords. Hence, in
theory, I could now play the same game and try out user and
password combinations, but I do not want to stoop to that level.
Analysing user names
Let us rather do a more interesting analysis, namely tabulating user name combinations over all failed nefarious requests. The top five valid user name requests are:
It is pretty clear that most scripts target
root specifically. Of
course, none of these users is allowed to login via
ssh anyway on
my server, but an attacker does not know that. My main take-away of
this list is that automated hacking attempts have become relatively
fine-tuned these days. I only experimented with ghost
for about a month, so it was interesting to see that some scripts
already include this platform.
As for the invalid user name requests, the distribution is quite strange. The top ten requests are:
- admin (4.57%)
- test (3.81%)
- user (3.07%)
- ubuntu (2.64%)
- ftpuser (2.40%)
- postgres (1.19%)
- oracle (1.10%)
- nagios (1.08%)
- git (0.92%)
- teamspeak (0.89%)
The distribution is much more uniform, in some sense—some access attempts really go through a lot of interesting two-character user names, maybe because they are targeting a specific market. Woe to those who install one of these services on their server and have it facing the outside world.
Finally, I want to visualize where most of the nefarious connection attempts are coming from. This is a prime occasion to bring out a Choropleth map! Numerous services and packages exist for accomplishing this task, but in the end, I went with plotly because their code interfaces nicely with Python. I only had to create a table in which I collected countries, their ISO-3166 code, and the number of requests. The former two pieces of information can be easily obtained from a GeoIP database. To use this with plotly, I merely had to convert the two-character country code to a three-character one. This resulted in the following map, for which the shading corresponds to the number of failed access attempts.
To make small-scale differences visible, I used logarithmic scaling. This means that there are roughly ten times more failed password attempts occurring from Chinese IP addresses than from U.S. ones, for example. This is pretty sobering to see.
Code & coda
It is certainly a humbling experience to see what a typical server has
to deal with every day. While none of these access attempts are probably
targeting myself directly, it still feels somewhat odd to be subjected
to these things. I will have to think carefully about how to handle
this; probably, it is now time for
ssh access ports.
Alternatively, I could try to be nice and send these users some messages
in which I express my desire not to attacked. If you are one of these
persons, please stop doing what you are doing. If you are interested in
performing the same analysis for your own server, have a look at
Auceps, the collection of
scripts I wrote for analysing logs.
Stay safe, until next time!