Study guide

11.2 Linux and macOS Endpoint Incident Investigation

EC-Council Certified Incident Handler ECIH

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction to Incident Handling and Response

1.1 Information Security Fundamentals, Threat Landscapes, and Attack Vectors 1.2 Incident Management Principles, CSIRT Team Structures, and SOAR Automation 1.3 Cyber Defense Frameworks: NIST SP 800-61, ISO 27035, MITRE ATT&CK, and Cyber Kill Chain 1.4 Risk Management, Security Policies, and Security Auditing

2 Chapter 2: Incident Handling Process: Preparation, Detection, and Triage

2.1 Incident Response Preparation, Policies, and Playbook Design 2.2 Incident Recording, Categorization, and Triage Methodologies 2.3 Incident Notification, Escalation Paths, and Stakeholder Communication

3 Chapter 3: Incident Handling Process: Containment, Eradication, and Post-Incident

3.1 Incident Containment Strategies, Isolation, and Evidence Protection 3.2 Eradication Procedures, Remediation, and Secure System Recovery 3.3 Post-Incident Review, Lessons Learned, and Threat Intelligence Sharing

4 Chapter 4: Forensic Readiness and First Response

4.1 Forensic Readiness Planning, Legal Mandates, and Regulatory Compliance 4.2 First Response Principles, Scene Securing, and Order of Volatility 4.3 Digital Evidence Collection, Packaging, and Chain of Custody 4.4 Anti-Forensics Detection, Evidence Gaps, and Countermeasures

5 Chapter 5: Handling and Responding to Malware Incidents

5.1 Malware Classification, Attack Vectors, and Modern Threat Profiles 5.2 Static Analysis, Dynamic Analysis, and Sandboxing Workflows 5.3 Malware Detection, Memory Forensics, and Eradication Strategies

6 Chapter 6: Handling and Responding to Email Security Incidents

6.1 Email Attack Vectors: Phishing, Spear Phishing, BEC, and Spoofing 6.2 Email Header Forensics, Link Triage, and Malicious Attachment Analysis 6.3 Email Authentication Protocols (SPF, DKIM, DMARC) and Incident Containment

7 Chapter 7: Handling and Responding to Network Security Incidents

7.1 Network Reconnaissance, Unauthorized Access, and Port Scanning 7.2 Denial of Service (DoS/DDoS) Attack Mechanisms and Traffic Mitigation 7.3 Wireless Network Incidents, Session Hijacking, and Packet Analysis 7.4 Inappropriate Usage Incidents: Validation, Evidence, and Response

8 Chapter 8: Handling and Responding to Web Application Security Incidents

8.1 Web Application Threat Vectors: Injection, XSS, and CSRF Incidents 8.2 Web Server and Application Log Analysis (Apache, Nginx, IIS) 8.3 Web Application Incident Containment, Patching, and Defense-in-Depth

9 Chapter 9: Handling and Responding to Cloud Security Incidents

9.1 Cloud Shared Responsibility, Threat Landscape, and IAM Attacks 9.2 Cloud Telemetry and Audit Logging (AWS CloudTrail, Azure Activity, GCP Audit) 9.3 Cloud Incident Containment, Snapshot Forensics, and Container Security

10 Chapter 10: Handling and Responding to Insider Threats

10.1 Insider Threat Typologies, Behavioral Indicators, and Risk Profiles 10.2 Insider Threat Detection: UEBA, Data Loss Prevention, and Audit Trails 10.3 Insider Incident Investigation, Evidence Handling, and Legal Reporting

11 Chapter 11: Handling and Responding to Endpoint, IoT, and OT Incidents

11.1 Windows Endpoint Forensics and Volatile Artifact Analysis 11.2 Linux and macOS Endpoint Incident Investigation 11.3 Mobile Device Isolation, Acquisition, and Incident Handling 11.4 IoT Incident Handling and Firmware Investigation 11.5 OT and ICS Incident Handling: Safety, Visibility, and Recovery

11.2 Linux and macOS Endpoint Incident Investigation

Key Takeaways Explain This Topic AI

Linux authentication and system event telemetry is divided across distribution-specific logs (/var/log/auth.log in Debian/Ubuntu vs /var/log/secure in RHEL/CentOS) and granular kernel auditing via auditd (/var/log/audit/audit.log).

User sessions and logins are recorded in binary accounting structures (utmp for active sessions, wtmp for historical logons/reboots, btmp for failed authentications), which require specialized utilities (w, last, lastb) to query.

Linux persistence mechanisms frequently abuse Cron jobs (/etc/cron*, /var/spool/cron/), systemd services and timers (/etc/systemd/system/), rc.local, and SSH authorized_keys.

The /proc virtual filesystem exposes volatile process data, allowing handlers to recover deleted malicious binaries directly from /proc/[PID]/exe and inspect open network connections via lsof -i and ss -tulpen.

macOS endpoint investigations require parsing the Apple Unified Logging System (log show), analyzing persistence in LaunchDaemons and LaunchAgents, inspecting quarantine extended attributes (com.apple.quarantine), and reviewing TCC database permissions (TCC.db).

Last updated: October 2026

Linux and macOS Endpoint Incident Investigation

Investigating incidents across Unix-like systems—enterprise Linux hosting servers and macOS utilized on corporate workstations—demands specialized knowledge of system auditing architectures, process subsystems, and persistence mechanisms. Linux dominates cloud infrastructure and containers, making it a high-value target for web shells, cryptominers, ransomware, and rootkits. Concurrently, macOS endpoints present distinct targets for corporate espionage. Incident handlers must understand the logging subsystems, volatile memory artifacts, and configuration frameworks unique to Linux and macOS to identify, contain, and remediate intrusions.

Linux Log Repositories and Audit Subsystems

Linux systems store authentication and operational logs within /var/log/:

System Event Logs: In Red Hat, CentOS, and Fedora, general system messages are captured in /var/log/messages. In Debian and Ubuntu, equivalent events are stored in /var/log/syslog. These logs record kernel errors, hardware faults, service startups, and daemon status changes.

Authentication Logs: Authentication attempts, privilege shifts, and account management operations are logged to /var/log/secure on Red Hat/CentOS, and /var/log/auth.log on Debian/Ubuntu. Handlers examine these files for SSH authentications, PAM errors, sudo executions, and user additions.

Linux Audit Daemon (auditd): The kernel auditing subsystem records configured system calls, file watches, and privilege events in /var/log/audit/audit.log . Local audit records are not inherently tamper-proof: a root-level adversary may disable audit rules, stop logging, or alter files. Forward records to a protected remote collector and alert on audit configuration changes or gaps. Handlers configure persistent rules in /etc/audit/rules.d/ and query events with dedicated tools. Analysts query audit records using specialized utilities:

ausearch: Searches audit logs for specific message types, syscalls, or timestamps (e.g., ausearch -m EXECVE -ts today).

aureport: Produces structured summary reports on executable runs, login attempts, syscall failures, and accounts.

User Activity, Session Tracking, and Shell Histories

Reconstructing user actions and logon activity requires analyzing binary accounting databases and user environment configurations:

Binary Session Accounting Files: Unlike human-readable text logs, Linux maintains session records in binary structures:

utmp (/var/run/utmp): Tracks currently logged-in users, active terminal sessions, and system runlevels. Queried using w, who, and users.

wtmp (/var/log/wtmp): Binary historical database of logins, logouts, terminal sessions, and reboots. It is not inherently tamper-proof and may be rotated, truncated, or altered by privileged actors. Queried using the last command.

btmp (/var/log/btmp): Records failed login attempts. Queried using lastb, btmp provides critical telemetry for identifying brute-force attacks.

lastlog (/var/log/lastlog): Sparse database maintaining the most recent login timestamp, source terminal, and host IP for each user in /etc/passwd. Queried via the lastlog utility.

User Account Databases: Handlers inspect /etc/passwd to identify unauthorized accounts, anomalous shells (such as /bin/sh on service accounts), and UID 0 accounts. The /etc/shadow file stores salted password hashes, password aging parameters, and account disable flags (denoted by ! or *).

Shell Command Histories: User command histories are stored in home directories (e.g., ~/.bash_history, ~/.zsh_history). Handlers examine shell histories to reconstruct adversary commands. Adversaries deploy anti-forensic techniques: prepending commands with whitespace (when HISTCONTROL=ignorespace is set), running history -c, pointing HISTFILE to /dev/null, or terminating with kill -9 $$. Handlers treat missing or truncated histories as suspicious anomalies.

Persistence Mechanisms in Linux

Adversaries establish persistence across multiple configuration layers in Linux:

Cron Scheduled Tasks: System-wide cron configurations reside in /etc/crontab, /etc/cron.d/, and hourly/daily directories (/etc/cron.hourly/, /etc/cron.daily/). User-specific schedules reside in spool directories (/var/spool/cron/crontabs/ or /var/spool/cron/). Attackers insert curl or bash reverse shells to regain access.

systemd Services and Timers: Modern service management across Linux. Handlers inspect unit files in /etc/systemd/system/ (custom units) and /lib/systemd/system/ (system packages). Malicious services define ExecStart= commands pointing to backdoors with Restart=always. Attackers also configure .timer units to execute tasks on elapsed intervals.

Initialization Scripts: Legacy startup scripts in /etc/rc.local, /etc/init.d/, /etc/profile.d/, and shell configurations (/etc/bash.bashrc, ~/.bashrc) execute unauthorized commands during boot or user logon.

SSH Authorized Keys: A common backdoor involves appending an adversary public key into ~/.ssh/authorized_keys or /root/.ssh/authorized_keys, granting persistent, passwordless access.

Live Volatile Investigation and the /proc Filesystem

The /proc virtual filesystem presents real-time telemetry on running processes:

Inspecting Process Subdirectories (/proc/[PID]/): For any suspicious PID, handlers inspect:

/proc/[PID]/exe: Symlink to the executable on disk. When an attacker deletes a running binary to thwart file scans, the link shows (deleted). Handlers can recover the intact binary by copying it directly: cp /proc/[PID]/exe /evidence/recovered_malware.bin.

/proc/[PID]/cmdline: Complete null-byte-delimited command invocation string and arguments.

/proc/[PID]/cwd: Symlink to the working directory.

/proc/[PID]/environ: Process environment variables, revealing injected dynamic libraries (LD_PRELOAD) or tokens.

/proc/[PID]/fd/: Open file descriptors, displaying open files, pipes, and sockets.

Process and Network Commands: Handlers run ps auxef to display process hierarchies with full command lines, lsof -i to inspect open network sockets mapped to PIDs, and ss -tulpen to identify listening TCP/UDP sockets, process ownership, and inode numbers.

macOS Incident Investigation

macOS incorporates distinct Apple security architectures and logging subsystems:

Apple Unified Logging System (ULS): Introduced in macOS 10.12, ULS stores telemetry in compressed tracev3 files in /var/db/diagnostics/ and /var/db/uuidtext/. Handlers query ULS using the log utility:

  • Historical query: log show --predicate 'process == "sudo"' --style syslog
  • Real-time stream: log stream --predicate 'eventMessage contains "failed"'

LaunchDaemons and LaunchAgents: Apple launchd manages persistent processes via property lists (.plist):

LaunchDaemons (/Library/LaunchDaemons and /System/Library/LaunchDaemons): Run at boot as root, operating independently of user GUI logons. They represent primary targets for persistence.

LaunchAgents (/Library/LaunchAgents and ~/Library/LaunchAgents): Run only after a user logs into a graphical desktop session, executing within that user context.

Quarantine Events: When a file is downloaded via browsers, email, or AirDrop, macOS attaches the extended attribute com.apple.quarantine. Handlers inspect this via xattr -p com.apple.quarantine [file]. Furthermore, macOS logs downloaded files in an SQLite database at ~/Library/Preferences/com.apple.LaunchServices.QuarantineEventsV*, recording file names, timestamps, and download URLs.

Transparency, Consent, and Control (TCC): Regulates application access to sensitive resources (Camera, Microphone, Full Disk Access). Policies are stored in SQLite databases at /Library/Application Support/com.apple.TCC/TCC.db and per-user ~/Library/Application Support/com.apple.TCC/TCC.db. Handlers query the access table to identify unauthorized permission grants.

FSEvents: Located in /.fseventsd/ on root volumes, FSEvents captures kernel file system transactions in gzip-compressed binary files, enabling reconstruction of file creations, modifications, and deletions.

Loading diagram... Linux and macOS Endpoint Artifact Triage Flowchart