Important commands to rescue hacked servers.

by Fady S. Ghatas | May 15, 2017 | Linux, Security, Web development | 0 comments

This post will you a boost in tracing the trails of your server intruder. You can use this post as a reference for your periodic server health checks as well. This is a very important article for server security specialists, white hats and developer in general.

Running processes

top

Use Shift-M to sort processes by Memory column (Very Important to detect DDoS Attacks)

Use Shift-P to sort processes by CPU column (Very Important to detect DDoS Attacks)

Use C to view full paths of running processes (Very Important to detect malicious shells upload location)

Please note top command will only show currently running processes.

ps

report a snapshot of the current processes.

ps aux

Report all processes from all users

ps ax -O psr, estime

Report all processes and add 2 columns (Core serving this process and Elapsed time)

For more columns check “man ps”

Files modified within x days / x minutes

find . -mmin -60 -type f

Detects files (-type f) modified within last hour (60 minutes -mmin -60). You can replace [.] with the desired folder.

find . -mtime -1 -type f

Same as above but uses days count instead (1 day).

This command is very important to detect uploaded shells.

Cumulative CPU usage

uptime

uptime gives a one line display of the following information. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.

What do these “load average” numbers mean?

For a single core processor :

0.8 : the processor was idle for 20% of the time.

1.2 : the processor was busy to serve a process while another process[es] waited for 20% of the time to get served, or, in other words, the processor was overloaded by 20%.

2.5 : the processor was overloaded by 150%. The processor was serving a process while other processes waited for 150% of the time (Must be more than a single process)

For example:

06:12:49 up 1:39, 3 users, load average: 1.08, 1.44, 1.31

Current time, up for x:xx, number of users, 1 minute load average, 5 minutes load average, 15 minutes load average.

Who has recently logged in + uptime

w (Yes, that’s the command)

You can use this command to view who is logged into the system using TTY, SSH, etc.

You can specify the username, for example:

w root

Searching for something?

Grep mails

grep -E -o “\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,6}\b” filename.txt

Logs Paths

Service	Log	Path
Apache	AccessLog [General paths]	/usr/local/apache/logs/access_log
Apache	ErrorLog [General paths]	/usr/local/apache/logs/error_log
Apache/cPanel sites	ErrorLog [cPanel per website error log]	/home/[server]/logs/
Apache/cPanel sites	AccessLog [cPanel per website access log]	/usr/local/apache/domlogs/[server] -OR- /home/[server]/access-logs/
Apache/cPanel itself	ErrorLog [cPanel itself, logging all cPanel errors]	/usr/local/cpanel/logs/error_log
Apache/cPanel itself	AccessLog [cPanel itself, logging all cPanel accesses]	/usr/local/cpanel/logs/access_log
PHP	Log	/var/log/messages
nginx	Access log	/var/log/nginx/access.log
nginx	Error Log	/etc/nginx/logs/error_log -or- find it in nginx.conf file
SSH	Log	/var/log/messages -OR- /var/log/secure find it in /etc/rsyslog.conf -OR- find it in /etc/rsyslog.d/50-default.conf
FTP	Log	/var/log/messages -OR- /var/log/vsftpd.log [for vsftpd] -OR- find it in /etc/vsftpd.conf -OR find it in /etc/vsftpd/vsftpd.conf
Exim	Log	/var/log/exim_mainlog
CSF/LFD	Log	/var/log/lfd.log