/// article

How to Investigate Suspicious Processes and Files on a Linux Server

By lordfrancs3

A strange process name on a Linux server does not automatically mean the server has been compromised. Neither does finding an unfamiliar file under /tmp or /dev/shm. The real question is whether the process, file, account, location, timestamps, command line, network activity, and persistence mechanisms make sense together. We had to answer exactly that question while investigating a compromised Zimbra mail server running on Ubuntu. During the incident, we discovered processes named: javab idle We also found suspicious files with names such as: .khp .khp_ts .rguard javab idle Some of the artifacts appeared in locations including: /dev/shm /tmp /home/SSL The filenames alone did not prove anything. What made them significant was the relationship between the files, running processes, scheduled execution, user ownership, and system logs. This guide uses that incident as a real-world case study and explains how Linux administrators can investigate suspicious processes and files without immediately deleting the evidence. Customer names, domains, IP addresses, SSH keys, credentials, remote destinations, and other identifying information have been removed. Important: If the server may be involved in a serious security incident, preserve evidence before making changes. Commands that kill processes, change permissions, or delete files can destroy information that may be useful later. What We Actually Found on the Zimbra Server Before discussing general investigation techniques, it is important to separate our confirmed findings from the additional checks recommended later in this guide. During our actual incident, we confirmed that suspicious processes named javab and idle were running under the zimbra account. One process used a command resembling: ./javab -o <REMOTE_HOST>:<REMOTE_PORT> The actual destination has been removed. We found suspicious artifacts under /dev/shm and /home/SSL, including files named: javab idle .khp .khp_ts .rguard During the continued investigation, we also encountered suspicious lock-related files under /tmp, including names associated with the same activity. The most important discovery came later when we found this line in the zimbra user’s crontab: * * * * * /home/SSL/.khp The cron job executed /home/SSL/.khp every minute. System logs also showed cron executing that command as the zimbra user. This connected the filesystem artifacts with the recurring suspicious processes. Zimbra’s own security guidance recommends checking running processes, recently modified files, temporary directories, SSH access, and the zimbra crontab when investigating a compromised system. Start With the Running Processes A good starting point is determining what is currently running. The standard Linux ps command reports a snapshot of active processes. One of the commands we used during the incident was: ps aux | grep -E 'javab|idle' A broader view is: ps auxf The f option provides a process hierarchy, which can help identify parent and child relationships. For more command-line detail, I usually prefer: ps auxwwf The additional w options prevent output from being shortened to the terminal width. When examining the output, pay attention to fields such as: USER PID %CPU %MEM START TIME COMMAND Do not focus only on %CPU. A malicious process may consume significant CPU, but it can also remain nearly idle while maintaining persistence or waiting for instructions. The USER column was particularly important in our case because the suspicious processes were associated with the zimbra account. That gave us another direction to investigate. Use pgrep When You Already Know the Process Name Once we identified javab and idle, repeatedly searching the entire process table was unnecessary. We could use: pgrep -a javab and: pgrep -a idle The -a option displays the PID and command line of each matching process. The Linux pgrep documentation notes that process-name matching has limitations. The -f option can be used when you need to match against the complete command line instead. For example: pgrep -af '/home/SSL' This can be useful when the executable name changes but the suspicious path remains visible in its command line. Another example is: pgrep -af '\.khp' These broader searches are additional investigation techniques. Our confirmed indicators during the incident were already known process and file names. Do Not Trust the Process Name A Linux process can call itself something that looks harmless. For example: backup java system-update monitor worker A name is only an indicator. Once you have the PID, determine what executable is actually running. Suppose the suspicious PID is: 12345 You can inspect: readlink -f /proc/12345/exe Linux exposes /proc/<PID>/exe as a symbolic link to the pathname of the program being executed. If the executable has already been deleted from the filesystem while the process continues running, Linux can append (deleted) to the pathname. For example, you might see: /home/SSL/javab That is far more useful than seeing only: javab in a process list. Check the Process Command Line Through /proc Linux exposes process information through the /proc virtual filesystem. To inspect the command line: tr '\0' ' ' < /proc/12345/cmdline echo The arguments in /proc/<PID>/cmdline are separated by null characters, which is why tr is used to replace them with spaces. Linux documents /proc/<PID>/cmdline as containing the process command line. However, a process can modify the information exposed there, so administrators should not treat it as unquestionable forensic evidence. This is an important limitation. You should correlate /proc information with: filesystem paths logs open files network connections user ownership timestamps scheduled tasks A single command rarely tells the entire story. Check the Process Working Directory Another useful /proc entry is: readlink -f /proc/12345/cwd This reveals the process’s current working directory. For example: /home/SSL would deserve attention if you were already investigating suspicious files under that directory. You can also inspect the process root: readlink -f /proc/12345/root and its open file descriptors: ls -l /proc/12345/fd These are additional general investigation techniques. They were not the primary commands that originally exposed our Zimbra compromise. Use lsof to See What the Process Has Open The name lsof means “list open files.” On Linux, many resources appear as files or file descriptors. lsof can therefore help identify what files, sockets, libraries, and other resources a process has open. For a particular PID: lsof -p 12345 You may see entries pointing to: /home/SSL/javab /tmp/something /dev/shm/something network sockets shared libraries log files If you want to concentrate on network activity: lsof -Pan -p 12345 -i Here: -P prevents port numbers from being converted to service names. -n prevents hostname resolution. -a combines the selection conditions. This can make the output faster and easier to interpret during an incident. Depending on permissions and Linux security restrictions, you may need root privileges to inspect another user’s processes fully. Search for Known Indicators With find Once we knew the suspicious filenames, we searched the server for additional copies. One of our actual searches resembled: find \ /home /root /tmp /var/tmp /dev/shm /run /opt/zimbra \ -xdev \ \( \ -name 'javab' -o \ -name 'idle' -o \ -name '.khp' -o \ -name '.khp_ts' -o \ -name '.rguard' -o \ -name 'ksmd' \ \) \ -ls 2>/dev/null GNU find searches directory trees from the specified starting paths and evaluates the supplied expression against each object it encounters. Several parts of this command are worth explaining. The starting directories were: /home /root /tmp /var/tmp /dev/shm /run /opt/zimbra These were selected because they were relevant to our incident. The expression: \( -name 'javab' -o -name 'idle' -o -name '.khp' \) means: find files named javab OR idle OR .khp The -ls action displays detailed information about matching files. The option: -xdev prevents find from descending into directories on another filesystem from each starting point. Finally: 2>/dev/null hides error output such as permission-denied messages. That can make results easier to read, but remember that it can also hide errors that may matter. Remove the redirection if you need to see everything. Search by Time, Not Only by Filename Attackers can rename files easily. If you know roughly when an incident began, timestamps can reveal other files created or modified around the same period. For example: find /home/SSL /tmp /dev/shm \ -xdev \ -type f \ -mmin -120 \ -ls This searches for regular files whose contents were modified within approximately the last 120 minutes. If the incident occurred yesterday, you might instead use: find /home/SSL /tmp /dev/shm \ -xdev \ -type f \ -mtime -2 \ -ls These searches can generate many legitimate results. Time-based searches should therefore be treated as a way to identify files requiring inspection, not as a malware detector. Inspect File Ownership and Permissions Once you find a suspicious file, start with: ls -la /home/SSL The -a option matters because it displays hidden names beginning with a dot. Without it, files such as: .khp .khp_ts .rguard may not appear in the normal directory listing. For a particular file: ls -l /home/SSL/.khp Typical output might resemble: -rwxr-xr-x 1 zimbra zimbra 4780 Aug 25 21:13 /home/SSL/.khp From this one line you can examine: file type permissions owner group size modification time pathname The Linux ls documentation describes options for showing hidden files and selecting modification, access, metadata-change, and birth timestamps where available. Ownership is particularly important. If a suspicious executable under an unusual directory is owned by: zimbra:zimbra and suspicious processes are also running as zimbra, those findings deserve to be correlated. Ownership alone still does not prove maliciousness. Use stat for Better File Metadata ls -l gives a useful summary, but stat provides more detail. For example: stat /home/SSL/.khp The stat command displays file or filesystem status information. Typical information includes: Size Blocks File type Permissions UID GID Access time Modify time Change time Birth time, if supported Three timestamps are particularly useful. mtime, or modification time, normally changes when file contents change. ctime, or status-change time, changes when metadata such as ownership or permissions changes and also when file contents change. atime, or access time, may change when a file is read, depending on filesystem and mount settings. Some filesystems can also expose a creation or birth timestamp. Do not interpret these timestamps as an unquestionable attacker timeline. Filesystem settings, restoration processes, copying, administrator actions, and deliberate manipulation can affect them. They are evidence to correlate with other evidence. Why /dev/shm Deserves Attention During our incident, suspicious artifacts appeared under: /dev/shm That does not mean /dev/shm is inherently suspicious. On Linux, POSIX shared-memory objects are normally backed by a tmpfs filesystem mounted at /dev/shm. Legitimate applications use /dev/shm. You should therefore never use a rule such as: Everything in /dev/shm is malware. That would be wrong. What made /dev/shm relevant in our incident was that files there matched other known suspicious indicators and were related to unauthorized activity already under investigation. A useful inspection command is: ls -lah /dev/shm Then inspect individual objects with: stat /dev/shm/<FILE> and: file /dev/shm/<FILE> You can search for recently modified regular files: find /dev/shm \ -xdev \ -type f \ -mmin -120 \ -ls Again, a match means “investigate this file,” not “delete this file.” Why /tmp and /var/tmp Also Deserve Attention Temporary directories are another useful place to inspect during an incident because many applications legitimately write temporary data there. That also means strange-looking files can be perfectly normal. During our Zimbra investigation, /tmp became relevant because we found lock-related artifacts associated with names we were already tracking. The correct approach is correlation. For example, suppose you find: /tmp/.idle_lockd at approximately the same time that: idle is running under the compromised application account. That relationship is more interesting than either item alone. Start with: ls -lah /tmp For targeted searching: find /tmp /var/tmp \ -xdev \ \( \ -name '*idle*' -o \ -name '*javab*' -o \ -name '*.khp*' \ \) \ -ls 2>/dev/null Do not recursively delete unfamiliar files from /tmp on a production server. Active applications may depend on sockets, lock files, temporary databases, or other objects stored there. Investigating /home/SSL Another significant location in our incident was: /home/SSL The directory name itself looked plausible. A directory named SSL could easily be assumed to contain certificates or security-related files. Instead, our investigation found suspicious files there, including: javab idle .khp .khp_ts This is an important operational lesson. Do not judge a directory by its name. Check its contents: ls -lah /home/SSL Check ownership: stat /home/SSL Then inspect individual files: stat /home/SSL/.khp file /home/SSL/.khp If the file is executable: ls -l /home/SSL/.khp Do not execute it simply to find out what it does. Identify the File Type The file utility can provide useful information without directly executing the suspicious program. For example: file /home/SSL/javab You may discover that something is: ELF executable shell script Python script compressed data text shared object File extensions on Linux are not authoritative. A file called: report.txt can still contain executable binary data. Likewise: javab tells you almost nothing about the real format. Calculate a Cryptographic Hash Before modifying or deleting a suspicious file, calculate a hash: sha256sum /home/SSL/.khp The result resembles: <64-character-SHA256-value> /home/SSL/.khp A SHA-256 hash gives you a stable identifier for the file contents. You can use it to: compare copies correlate the same file on other servers document evidence search internal threat-intelligence records verify that your preserved copy matches the source Do not upload potentially sensitive corporate files to a public malware-analysis service without considering company policy, privacy, confidentiality, and legal requirements. Preserve Evidence Before Deleting Anything The natural reaction when finding malware is: rm -f suspicious-file That may destroy information you still need. NIST’s Guide to Integrating Forensic Techniques into Incident Response recommends properly gathering and handling evidence, maintaining records of major actions, preserving integrity, and maintaining appropriate custody of collected evidence. For operational troubleshooting, a basic evidence directory might begin with: mkdir -p /root/incident-evidence chmod 700 /root/incident-evidence Record the current time: date -Is > /root/incident-evidence/collection-time.txt Capture the process list: ps auxwwf > /root/incident-evidence/processes.txt Record metadata before copying: stat /home/SSL/.khp \ > /root/incident-evidence/khp-stat.txt Calculate the source hash: sha256sum /home/SSL/.khp \ > /root/incident-evidence/khp-source.sha256 Then preserve a copy: cp -a /home/SSL/.khp \ /root/incident-evidence/ Hash the copy: sha256sum /root/incident-evidence/.khp \ > /root/incident-evidence/khp-copy.sha256 This is a lightweight operational approach, not a replacement for formal forensic acquisition. Simply reading files can affect certain metadata depending on filesystem and mount behavior. If the incident may involve legal action, regulatory requirements, or a serious breach investigation, use your organization’s forensic procedures and qualified incident-response personnel. Be Careful Before Killing the Process A suspicious process may contain useful volatile evidence that disappears when it terminates. Before killing it, consider capturing: ps -fp <PID> readlink -f /proc/<PID>/exe readlink -f /proc/<PID>/cwd tr '\0' ' ' < /proc/<PID>/cmdline echo lsof -p <PID> If appropriate: lsof -Pan -p <PID> -i Once you have collected what you need and confirmed the process is unauthorized, containment may require terminating it. For example: kill <PID> or: pkill javab The decision depends on the incident. If a process is actively damaging data or attacking other systems, containment may take priority over collecting every possible artifact. Check Whether the Executable Has Already Been Deleted One useful Linux-specific clue is: readlink /proc/<PID>/exe You might see: /home/SSL/javab (deleted) Linux documents that /proc/<PID>/exe can show (deleted) if the pathname has been unlinked while the process continues to execute. That situation deserves investigation. It can occur legitimately during software upgrades, but it can also appear when a suspicious executable has been deleted while the process remains alive. Again, context matters. Look for Parent and Child Relationships A suspicious executable does not launch itself. Try: ps -ef --forest or: ps auxf Find the suspicious PID and look at its parent process. You can also display specific fields: ps -o pid,ppid,user,lstart,etime,args -p <PID> This shows: PID parent PID user start time elapsed time command If the parent is still running, investigate it too. A recurring process whose parent changes every minute may suggest a scheduler or another launcher. That became relevant in our Zimbra incident because killing the visible processes did not stop the activity permanently. We later discovered the persistence mechanism: * * * * * /home/SSL/.khp The file investigation therefore became a persistence investigation. Compare Process Start Time With File Timestamps One useful technique is to compare: process start time file modification time file metadata-change time cron execution time authentication time log entries For example: ps -o pid,lstart,args -p <PID> and: stat /home/SSL/.khp can be viewed together. Suppose the file appeared at approximately 21:13 and the associated process started soon afterward. That does not prove causation, but it gives you a timeline to investigate. Then search the logs around the same period. This is much more useful than looking at every log entry from an entire week. Check Permissions Carefully Linux permissions can provide useful clues. For example: -rwxrwxr-x can be interpreted as: owner: rwx group: rwx others: r-x The x bit means a regular file can be executed when other requirements are met. Linux chmod manages these file mode bits. If you discover an executable owned by an application account in an unexpected directory, ask: Should this account own this file? Should this file be executable? Should this directory contain executables? Who could have written this file? Is the group writable? When did the mode change? Do not immediately run: chmod -R 000 <DIRECTORY> on a production server. Changing permissions recursively can break legitimate services and modify evidence. Containment During Our Incident After identifying /home/SSL as part of the suspicious activity, one containment action we used was: chmod -R a-x /home/SSL This removed execute permission recursively from the contents of the suspicious directory. We also terminated confirmed unauthorized processes. This was an incident-specific containment decision. It should not be copied blindly. Before using a recursive permission command, verify that the target directory does not contain legitimate executables required by production services. A formal forensic response may also preserve the system state before changing permissions. What readlink Adds to the Investigation The readlink command resolves symbolic links and can return canonical paths with -f. This makes it particularly useful with /proc. For example: readlink -f /proc/<PID>/exe answers: What executable is this process actually using? While: readlink -f /proc/<PID>/cwd answers: From which directory is the process currently operating? Those answers can connect something you saw in ps with something you found on disk. What lsof Adds to the Investigation ps tells you about the process. find tells you about files. lsof helps connect the two. For example: lsof -p <PID> can show whether the suspicious process currently has a configuration file, log, shared library, device, socket, or deleted file open. If the process opens: /home/SSL/.khp or another known suspicious artifact, that relationship becomes another part of your evidence. The same applies to network sockets. A process connecting to an unexpected remote destination deserves further investigation, especially when combined with other indicators. Do Not Rely on One Indicator A useful Linux investigation is built from correlated evidence. In our Zimbra incident, the important relationships eventually looked roughly like this: zimbra account | +---- suspicious process: javab | +---- suspicious process: idle | +---- suspicious files | | | +---- /dev/shm | +---- /home/SSL | +---- /tmp | +---- /home/SSL/.khp | +---- referenced by zimbra crontab | +---- executed every minute One unusual file would have been weak evidence. One unusual process name would have been weak evidence. The combination made the situation clear. Actual Incident Findings Versus Additional Investigation Techniques For clarity, these were confirmed during our incident: suspicious javab and idle processes, suspicious files under /dev/shm and /home/SSL, related artifacts under /tmp, ownership connected to the zimbra account, a suspicious .khp file, recurring activity after process termination, and a cron entry executing /home/SSL/.khp every minute. The following techniques are broader recommendations that can improve a future investigation: inspecting /proc/<PID>/cwd, collecting complete lsof output, hashing every artifact before containment, searching files by modification time, comparing process start times with filesystem metadata, and capturing formal forensic evidence according to an incident-response procedure. Those recommended checks should not be interpreted as additional findings from our original Zimbra incident. A Practical Investigation Workflow When you encounter an unfamiliar process on a Linux production server, a safe initial workflow is to identify it without changing anything, capture its PID and command line, inspect /proc, inspect open files, identify the executable, record file metadata, search for related files, calculate hashes, correlate timestamps with logs, investigate persistence, preserve the important artifacts, and only then perform containment when circumstances allow. For example: # Identify processes ps auxwwf pgrep -af '<SUSPICIOUS_NAME>' # Inspect a specific process ps -o pid,ppid,user,lstart,etime,args -p <PID> readlink -f /proc/<PID>/exe readlink -f /proc/<PID>/cwd tr '\0' ' ' < /proc/<PID>/cmdline echo # Inspect open resources lsof -p <PID> lsof -Pan -p <PID> -i # Inspect a suspicious file ls -l <FILE> stat <FILE> file <FILE> sha256sum <FILE> # Search for known indicators find /tmp /var/tmp /dev/shm /home \ -xdev \ \( \ -name '<INDICATOR_1>' -o \ -name '<INDICATOR_2>' \ \) \ -ls 2>/dev/null # Search recently modified files in targeted locations find /tmp /var/tmp /dev/shm \ -xdev \ -type f \ -mmin -120 \ -ls This is an investigation workflow, not an automated malware-removal script. Review every result before taking action. Lessons From the Zimbra Incident The biggest lesson was that suspicious processes should be investigated as relationships, not isolated objects. javab became more significant when we discovered where it was stored and which user was running it. .khp became more significant when we discovered it under /home/SSL. /home/SSL/.khp became critical when we found it referenced by the zimbra user’s crontab. The recurring processes made sense once we connected them to persistence. Temporary directories also require context. /tmp and /dev/shm contain legitimate files on healthy Linux servers. Their contents become interesting when they match other indicators, have unexpected ownership, execute as application accounts, appear at suspicious times, or connect to other evidence. The most important operational rule is simple: Do not delete the first suspicious file you find. Investigate what created it, who owns it, what executes it, what it opens, when it appeared, and whether something can recreate it. That information can tell you much more than the filename itself. References Zimbra, Investigating and Securing Systems Zimbra’s security investigation documentation covers suspicious processes, recently modified files, temporary directories, SSH activity, and compromise investigation procedures. Zimbra: Investigating and Securing Systems Linux ps(1) Manual Official Linux manual page for inspecting active processes. Linux ps(1) manual Linux pgrep(1) and pkill(1) Manual Documents process matching, full-command-line searches, and process signalling. Linux pgrep and pkill manual GNU Findutils Official GNU documentation for the find utility and its search expressions. GNU Findutils: Invoking find Linux lsof(8) Manual Documents lsof and the inspection of open files and process resources. Linux lsof(8) manual Linux /proc/<PID>/exe Documents the /proc/<PID>/exe symbolic link and how Linux exposes the executable associated with a process. Linux proc_pid_exe(5) manual Linux /proc/<PID>/cmdline Documents process command-line information and its limitations. Linux proc_pid_cmdline(5) manual Linux readlink(1) Manual Documents resolving symbolic links and canonical paths. Linux readlink(1) manual Linux Shared Memory and /dev/shm Documents POSIX shared memory and the normal use of the tmpfs filesystem mounted at /dev/shm. Linux shm_overview(7) manual Linux tmpfs(5) manual Linux stat(1) Manual Documents displaying file and filesystem metadata. Linux stat(1) manual NIST SP 800-86 NIST’s Guide to Integrating Forensic Techniques into Incident Response provides guidance for collecting, preserving, documenting, and handling digital evidence. NIST SP 800-86: Guide to Integrating Forensic Techniques into Incident Response