[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-relays] My VPS relay has just been hacked
First, many thanks for all the advice. I will do some serious homework.
To the question What kind of virtualisation? It's OpenVZ. In fact it's
the OVZ Starter option from here:
http://www.edis.at/en/server/linux-vps-solusvm-openvz/zurich
and the OS option I chose is what Edis describe as "Debian 7 amd64
minimal (64bit)"
I've restarted the VPS for 5-10 minutes at a time, long enough to watch
the malware at work but not long enough for it to ramp up its flood of
output. It seems to live here, in /boot:
root@xxxxx:/boot# ls -al
total 128
drwxr-xr-x 2 root root 4096 Oct 26 14:19 .
drwxr-xr-x 23 root root 4096 Oct 26 13:58 ..
-rwxr-xr-x 1 root root 122698 Oct 24 17:19 hxyqbutesc
and it's started in a straightforward way from /etc and /etc/init.d ;
rc1.d, rc2.d and so on have links to /etc/init.d/hxyqbutesc :
root@xxxxx:/etc# ls -l rc1.d
total 4
1 root root 18 Sep 13 20:21 K01quotarpc -> ../init.d/quotarpc
1 root root 13 Sep 14 22:41 K01tor -> ../init.d/tor
1 root root 17 Sep 13 20:21 K04rsyslog -> ../init.d/rsyslog
1 root root 369 Aug 5 07:47 README
1 root root 18 Sep 13 20:21 S01bootlogs -> ../init.d/bootlogs
1 root root 20 Oct 24 17:19 S01hxyqbutesc -> ../init.d/hxyqbutesc
...
root@xxxxx:/etc# ls -l rc2.d
total 4
1 root root 677 Aug 5 07:47 README
1 root root 18 Sep 13 20:21 S01bootlogs -> ../init.d/bootlogs
1 root root 20 Oct 24 17:19 S01hxyqbutesc -> ../init.d/hxyqbutesc
...
and in /etc/init.d is a short script:
root@xxxxx:/etc/init.d# ls -l
total 196
-rw-r--r-- 1 root root 2427 Aug 5 07:47 README
-rwxr-xr-x 1 root root 1276 Aug 5 07:47 bootlogs
-rwxr-xr-x 1 root root 1281 Aug 5 07:47 bootmisc.sh
-rwxr-xr-x 1 root root 3816 Aug 5 07:47 checkfs.sh
-rwxr-xr-x 1 root root 1099 Aug 5 07:47 checkroot-bootclean.sh
-rwxr-xr-x 1 root root 9673 Aug 5 07:47 checkroot.sh
-rwxr-xr-x 1 root root 3033 Aug 5 07:47 cron
-rwxr-xr-x 1 root root 1329 Aug 5 07:47 halt
-rwxr-xr-x 1 root root 1423 Aug 5 07:47 hostname.sh
-rwxr-xr-x 1 root root 3880 Aug 5 07:47 hwclock.sh
-rwxr-xr-x 1 root root 317 Oct 26 13:58 hxyqbutesc
...
which looks like this:
root@xxxxx:/etc/init.d# cat hxyqbutesc
#!/bin/sh
# chkconfig: 12345 90 90
# description: hxyqbutesc
### BEGIN INIT INFO
# Provides: hxyqbutesc
# Required-Start:
# Required-Stop:
# Default-Start: 1 2 3 4 5
# Default-Stop:
# Short-Description: hxyqbutesc
### END INIT INFO
case $1 in
start)
/boot/hxyqbutesc
;;
stop)
;;
*)
/boot/hxyqbutesc
;;
esac
and runs the main file /boot/hxyqbutesc , which is mainly hex numbers
with many embedded filenames and paths. The timestamp Oct 24 17:19 is
an hour or two before the VPS was suspended on Friday evening (can't be
more precise yet because I'm not sure what timezone the suspension email
was sent from).
Thank you to the people who suggested using ps as well as top. ps -x
gives eg this when the malware is working:
PID TTY STAT TIME COMMAND
1 ? Ss 0:00 init [2]
2 ? S 0:00 [kthreadd/3277]
3 ? S 0:00 [khelper/3277]
1373 ? Ssl 0:00 who
1406 ? Sl 0:00 /usr/sbin/rsyslogd -c5
1451 ? Ss 0:00 /usr/sbin/cron
1478 ? Ss 0:00 /usr/sbin/sshd
1857 ? Ss 0:00 vzctl: pts/0
1858 pts/0 Ss 0:00 -bash
3474 ? Ss 0:00 ls -la
3477 ? Ss 0:00 cat resolv.conf
3478 ? Ss 0:00 route -n
3479 ? Ss 0:00 ifconfig eth0
3481 ? Ss 0:00 su
3489 ? Ss 0:00 sleep 1
3492 ? Ss 0:00 ifconfig
3493 ? Ss 0:00 sh
3494 ? Ss 0:00 grep "A"
3496 ? Ss 0:00 pwd
3497 pts/0 R+ 0:00 ps -x
For process 1373 top would give "hxyqbutesc" instead of "who" (this line
doesn't change), and the ten lines for 3474 - 3496 would be the two
changing blocks of ten-letter random names. Assuming the glimpses of
command names here can be taken at face value, something very sinister
is clearly going on ... other commands that flash into view are "who,
whoami, uptime, bash, id, cd /etc, netstat -antop, top, echo "find",
gnome-terminal".
As for how it got in, most people seem to suspect an attack through ssh.
But when I ssh'd in yesterday to start investigating, I was careful to
note the "Last Login" message, and it gave my own genuine last login,
three weeks ago from my home IP. Can I take this at face value to mean
that ssh has not been used between my own two logins? Can "Last Login"
be falsified?
Some suggest an exploit using out-of-date software. Well, I last did an
apt-get dist-upgrade 29 days ago, (after the bash fix), and tor and
obfsproxy (included in the dist-upgrade) were the only things I
installed over the "minimal" Debian 7.6 - so I was reasonably up-to-date.
And some suggested that the malware might have come through the Solus
control panel (which of course has root access - I ssh in by requesting
a temporary (time-limited) console from Solus, and it gives me a
one-time strong password to use for that). Is this plausible? perhaps
the next step is to send all this to Edis support and see if they
noticed anything unusual around 17:19 last Friday?
Thanks again for everybody's help,
Nick Sheppard
_______________________________________________
tor-relays mailing list
tor-relays@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays