vps is compromised
Hello,
i have a vps which i am not able to ssh, when i use vnc and change ssh config and login with password...
i see that ~/.ssh/authorized_keys
doesn't have my keys but someone else's key (Misha@asus-laptop) ....so i realize someone hacked into vps.
i have some data in it so i don't want to wipe it off and start from scratch again and also this is an opportunity for me to learn about security etc things. (if needed i can wipe it off and restore from backup)
It had some staging wordpress site which i deleted just now
and looking at cat \etc\passwd
shows
root:x:0:0:root:/root:/usr/bin/zsh
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys:x:3:3:sys:/dev:/usr/sbin/nologin
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/usr/sbin/nologin
man:x:6:12:man:/var/cache/man:/usr/sbin/nologin
lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin
mail:x:8:8:mail:/var/mail:/usr/sbin/nologin
news:x:9:9:news:/var/spool/news:/usr/sbin/nologin
uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin
proxy:x:13:13:proxy:/bin:/usr/sbin/nologin
www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin
backup:x:34:34:backup:/var/backups:/usr/sbin/nologin
list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin
irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin
gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin
nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin
systemd-network:x:100:102:systemd Network Management,,,:/run/systemd:/usr/sbin/nologin
systemd-resolve:x:101:103:systemd Resolver,,,:/run/systemd:/usr/sbin/nologin
systemd-timesync:x:102:104:systemd Time Synchronization,,,:/run/systemd:/usr/sbin/nologin
messagebus:x:103:106::/nonexistent:/usr/sbin/nologin
syslog:x:104:110::/home/syslog:/usr/sbin/nologin
_apt:x:105:65534::/nonexistent:/usr/sbin/nologin
tss:x:106:111:TPM software stack,,,:/var/lib/tpm:/bin/false
uuidd:x:107:112::/run/uuidd:/usr/sbin/nologin
tcpdump:x:108:113::/nonexistent:/usr/sbin/nologin
landscape:x:109:115::/var/lib/landscape:/usr/sbin/nologin
pollinate:x:110:1::/var/cache/pollinate:/bin/false
sshd:x:111:65534::/run/sshd:/usr/sbin/nologin
systemd-coredump:x:999:999:systemd Core Dumper:/:/usr/sbin/nologin
lxd:x:998:100::/var/snap/lxd/common/lxd:/bin/false
gitlab-runner:x:997:997:GitLab Runner:/home/gitlab-runner:/bin/bash
mysql:x:112:120:MySQL Server,,,:/nonexistent:/bin/false
lsadm:x:996:996:lsadm:/:/sbin/nologin
redis:x:113:121::/var/lib/redis:/usr/sbin/nologin
and i don't feel all those users list belongs to me.
Also, what other folders/files i should check to confirm vps is compromised?
thank you so much.
This discussion has been closed.
Comments
try using
last -a
command, if the users and ip address different from yours then you could say it compromised.⭕ A simple uptime dashboard using UptimeRobot API https://upy.duo.ovh
⭕ Currently using VPS from BuyVM, GreenCloudVPS, Gullo's, Hetzner, HostHatch, InceptionHosting, LetBox, MaxKVM, MrVM, VirMach.
alternatively, assuming it was a low end hacker (that did not care to cover up the traces left behind), you could try taking a look at /var/log/syslog and/or /var/log/secure (grep for ssh or sshd) to see if there were any suspicious ssh connections
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
Also check
/etc/shadow
to see if there are accounts with passwords set.For domain registrations, create an account at Dynadot (ref) and spend $9.99 within 48 hours to receive $5 DynaDollars!
Looking for cost-effective Managed/Anycast/DDoS-Protected/Geo DNS Services? Try ClouDNS (aff).
Thanks everyone for your response...highly appreciated
it shows ips mostly from my isp range.
first command shows full system log and second file doesn't exists
show this
thanks.
ThemeForest Profile
history | less
It wisnae me! A big boy done it and ran away.
NVMe2G for life! until death (the end is nigh)
Thanks for that command, learned something new
i have gone through whole history...kinda interesting to see what i have done
i don't see anything suspicious.
i doubt if that key is from provider (wishosting) they moved vps from one DC to another one last time.
so they might put their key.
ThemeForest Profile
Hmmm..
So if there was no /var/log/secure I guess it's not a CentOS - how about /var/log/auth.log? (assuming it's Debian or Ubuntu)
Also, were there no traces of ssh connections inside the syslog? (I was expecting something like this: https://www.ibm.com/support/pages/sshd-can-use-unix-syslog-facilities-logging )
You could also try finding files (using the last modified/accessed filters) - this perhaps could also show some suspicious activity (assuming you're aware of what's usually happening on the server) - command examples: https://www.cyberciti.biz/faq/howto-finding-files-by-date/
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
Misha = Michael so that's probably wishosting's keys
oops sorry, i should have mentioned it is ubuntu 20
/var/log/auth.log
has too much logs even at this minute...someone trying with random ports with failed passwordthe contents are too big, i am just pasting recent log
ThemeForest Profile
if thats true then i am bit relieved
ThemeForest Profile
You're not supposed to just read it
Looks like they (people bruteforcing boxes with password auth) got you - this is definitely too much to go line by line, the last idea I'm able to come up with ATM is to grep the shit out of that ssh log to pin point the user / IP combination that successfully logged in (assuming this indeed was a hack and not the VPS provider as suggested above, lol :P), and then perhaps take a look at /var/log/messages (everything that is in there shortly after the first successful attempt) - although I'm not sure whether you'd be able to find anything meaningful there ><'
Also maybe try finding files accessed/modified shortly after the attacker's ssh session successfully started (and rinse and repeat for all the times after subsequent logins as root, that were made no longer using the password, but the seemingly newly set ssh key)
For future reference:
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
.... When you copy and pasted the log, you should have inspected it?
♻ Amitz day is October 21.
♻ Join Nigh sect by adopting my avatar. Let us spread the joys of the end.
Thanks for great explanation.
i have created a ticket to know whether it is really himself/team.
if its not himself, then i will dig deeper into it,
definitely learned few things with this incident.
thanks.
ThemeForest Profile
@seenu
A few random thoughts:
You say that your key was removed from .ssh/authorized_keys and that an unknown Misha key is present.
You do not mention anything else which seems really broken or strange. I am not at all an expert, but the files you posted, /etc/passwd and /etc/shadow seem basically sane to me.
Does the authorized_keys file in your most recent backup have your key?
Does the authorized_keys file in your most recent backup have the Misha key?
What is the backup authorized_keys file's modification date?
What is the modification date on the presently in use .ssh/authorized_keys?
When is the most recent day and time that you logged in using your key?
Look at auth.log on the date and prior to the time authorized_keys was last modified.
I vote for having ssh access firewall-restricted to specific IPs. Make sure to have more than one allowed IP in case you lose access to an allowed IP.
Honeypots
Maybe you might be interested in this paper: https://github.com/security-union/going-fishing-with-my-raspberry-pi/blob/master/going_fishing_raspberry_pi.pdf
The guy who wrote the above paper also has a very cool Youtube video:
Intrusion Recovery via Selective Re-execution
From 2010, Intrusion Recovery Using Selective Re-execution (pdf)
From 2015, Intrusion Recovery Using Selective Re-execution
MetalVPS
thanks for your time @Not_Oles
unfortunately i don't have a backup of authorized_keys, when i am not able to login...i login through vnc and found that my key is not there so i just removed misha key and placed mine....after few moments...i realized...i should have seen last modified time
I want to wait till i get response from provider before i want to investigate this one further.
thanks
ThemeForest Profile
I am finally relieved.
Just got a mail from support saying it is their key
Mods... please close this thread.
thanks everyone, i learned something new in this process.
ThemeForest Profile
Closed per request.
https://clients.mrvm.net