ExamFinch.com My Study

Study guide

5.7 Packet Analysis and Secure Protocols for Defenders

Practice this exam

EC-Council Certified Ethical Hacker CEH

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 CEH Exam Format, Cost & Eligibility 1.2 The Five Phases of Ethical Hacking 1.3 Information Security Fundamentals 1.4 Ethical Hacking Concepts, Scope & Legal Framework 1.5 The CEH Domains & Your Study Priority Map

2 Reconnaissance, Footprinting, Scanning & Enumeration

2.1 Footprinting & Reconnaissance Concepts 2.2 Footprinting Sources & Techniques 2.3 Network Scanning Concepts 2.4 Banner Grabbing, OS Fingerprinting & Scanning Beyond IDS/Firewall 2.5 Enumeration 2.6 Vulnerability Scanning & Countermeasures

3 Vulnerability Analysis & System Hacking Phases

3.1 Vulnerability Analysis Concepts 3.2 System Hacking Methodology Overview 3.3 Gaining Access: Password Attack Concepts 3.4 Privilege Escalation Concepts 3.5 Maintaining Access: Backdoors, Rootkits & Persistence 3.6 Covering Tracks, Anti-Forensics & Blue-Team Detection

4 Malware Threats, Sniffing & Social Engineering

4.1 Malware Concepts and Taxonomy 4.2 Malware Analysis and Detection 4.3 Sniffing Concepts and Layer 2 Attacks 4.4 Sniffing Countermeasures and Detection 4.5 Social Engineering Types and Human Psychology 4.6 Social Engineering and Insider Threat Defenses

5 Network & Perimeter Hacking

5.1 Denial of Service and DDoS Attacks 5.2 Session Hijacking 5.3 Evading IDS and IPS (Defensive Focus) 5.4 Evading Firewalls 5.5 Evading Honeypots and Defensive Deception 5.6 Network Security Controls and Architecture 5.7 Packet Analysis and Secure Protocols for Defenders

6 Web Server & Web Application Hacking

6.1 Web Server Attacks & Hardening 6.2 OWASP Top 10 Overview 6.3 Injection: SQL Injection & Command Injection 6.4 Cross-Site Scripting (XSS) & CSRF 6.5 Authentication, Session & Access-Control Flaws 6.6 API & Web App Security Testing Concepts

7 Wireless Network Hacking

7.1 Wireless Concepts and Standards 7.2 Wireless Encryption 7.3 Wireless Threats and Attack Concepts 7.4 Wireless Security Countermeasures

8 Mobile Platform, IoT & OT Hacking

8.1 Mobile Platform Threats & Security Models 8.2 Mobile Attack Vectors & Risks 8.3 Mobile Device Management & Enterprise Defense 8.4 IoT Hacking Concepts & Defense 8.5 OT, ICS & SCADA Security

9 Cloud Computing & Cryptography

9.1 Cloud Computing Concepts 9.2 Cloud Threats & Attack Surfaces 9.3 Cloud Security Controls 9.4 Cryptography Fundamentals 9.5 PKI & Cryptographic Applications 9.6 Cryptanalysis, Crypto Attacks & Best Practices

5.7 Packet Analysis and Secure Protocols for Defenders

Key Takeaways Explain This Topic AI

Wireshark and tcpdump capture and dissect packets; defenders use display filters (e.g., tcp.flags.syn==1, http.request) to triage floods, scans, and exfiltration.

Cleartext protocols (Telnet/23, FTP/21, HTTP/80, SNMPv1/2, POP3/110, IMAP/143) expose credentials to sniffing; their encrypted replacements are SSH/22, FTPS/SFTP, HTTPS/443, SNMPv3 with authPriv, and POP3S/IMAPS.

Correctly configured TLS provides confidentiality, integrity, and peer authentication. IPsec AH provides integrity/authentication without encryption; ESP can provide confidentiality and integrity according to the negotiated security association.

Attack signatures are visible in captures: many SYNs without ACKs (flood/scan), gratuitous ARP replies (ARP poisoning), and oversized/abnormal DNS replies (amplification/tunneling).

Defenders enforce encryption everywhere, monitor with span ports/SIEM, and disable insecure legacy protocols to deny attackers the cleartext they rely on.

Last updated: September 2026

Packet Capture and Analysis Tools

Defenders read attacks directly off the wire with packet analyzers . Wireshark is the dominant GUI tool: it captures live traffic (in promiscuous or monitor mode) and dissects every protocol layer, while tcpdump is its command-line counterpart for headless capture and scripting. Captures are stored in pcap/pcapng files for offline analysis or sharing with a SIEM.

The key skill is filtering . Wireshark distinguishes capture filters (BPF syntax, applied before capture, e.g. tcp port 80 ) from display filters (applied to an existing capture, e.g. http.request ). Useful defensive display filters:

  • Display Filter Surfaces
  • tcp.flags.syn==1 && tcp.flags.ack==0 SYN floods / port scans
  • http.request Web requests (and Layer-7 floods)
  • dns DNS queries (amplification/tunneling)
  • arp.duplicate-address-detected ARP poisoning / MITM
  • icmp Ping sweeps, ICMP tunneling
  • tcp.analysis.retransmission Network stress / desync

Reading the TCP flags , sequence numbers, and timing lets an analyst confirm whether traffic is a handshake, a flood, a hijack (the ACK storm of §5.2), or a scan.

Reading Attack Signatures in Captures

Many attacks in this chapter leave distinctive fingerprints a defender can recognize in Wireshark:

SYN flood / scan — a torrent of SYN packets with no corresponding ACK completing the handshake; many half-open SYN_RECV entries.

Smurf / amplification — a flood of ICMP echo replies or large UDP responses (DNS/NTP) arriving unsolicited at the victim.

ARP poisoning (MITM) — unsolicited or duplicate gratuitous ARP replies mapping the gateway IP to the attacker's MAC; Wireshark flags duplicate-address-detected .

DNS tunneling — abnormally long, high-entropy, or high-volume DNS queries to one domain (TXT/NULL records carrying encoded data).

Session hijacking — the ACK storm , out-of-sequence segments, and an injected RST.

Cleartext credential theft — Follow TCP Stream on a Telnet/FTP/HTTP session reveals usernames and passwords in plaintext.

This is precisely why ethical hackers demonstrate sniffing: to prove that unencrypted protocols hand credentials to anyone on the segment, justifying the move to encryption.

Cleartext vs. Secure Protocols

The single highest-impact defensive measure against sniffing and hijacking is to eliminate cleartext protocols . CEH expects you to pair each insecure protocol with its encrypted replacement and remember the ports:

  • Insecure (Cleartext) Port Secure Replacement Port
  • Telnet 23 SSH 22
  • FTP 21/20 SFTP / FTPS 22 / 990
  • HTTP 80 HTTPS (TLS) 443
  • SNMP v1/v2c 161 SNMPv3 (auth+priv) 161
  • POP3 / IMAP 110 / 143 POP3S / IMAPS 995 / 993
  • SMTP (open) 25 SMTPS / STARTTLS 465 / 587
  • LDAP 389 LDAPS 636
  • rlogin/rsh/rcp 513/514 SSH 22

Legacy protocols transmit credentials and data in the clear, so a single sniffer (Wireshark, Ettercap) on the segment harvests everything. Their encrypted replacements wrap the same function in TLS or SSH , defeating both passive sniffing and active injection.

TLS, IPsec, and Defensive Practice

TLS 1.3 protects application traffic and removes legacy cryptographic options while reducing handshake round trips in common cases. Correct peer authentication and configuration remain essential. IPsec secures traffic at the network layer , transparently to applications, and is the backbone of site-to-site and remote-access VPNs. Know its two protocols: AH (Authentication Header) provides integrity and authentication but no encryption , while ESP (Encapsulating Security Payload) provides confidentiality and/or integrity/authentication, according to the selected security association — ESP with encryption is the option that keeps payloads confidential. IPsec runs in transport mode (protects payload, host-to-host) or tunnel mode (protects the whole packet, gateway-to-gateway VPN).

Defensive practice for a network analyst:

Encrypt everything in transit — TLS for apps, SSH for admin, IPsec/VPN for site links.

Disable legacy/cleartext services and weak ciphers (SSLv3, TLS 1.0/1.1, SNMPv1).

Capture at choke points via SPAN/TAP and feed pcap/NetFlow to the SIEM for correlation.

Inspect TLS at the perimeter (NGFW decryption) so encryption does not blind the IDS.

Baseline normal traffic so floods, beaconing, and tunneling stand out.

Two caveats CEH expects you to hold together. First, encryption does not make traffic invisible — metadata (who talks to whom, when, how much) still leaks via NetFlow, and TLS inspection at the perimeter is needed so that encryption does not become an attacker's cover. Second, switched networks do not automatically prevent sniffing: an attacker on the LAN can use ARP poisoning or MAC flooding (forcing the switch to fail-open like a hub) to capture traffic that the switch would normally isolate — which is exactly why encryption, not network topology, is the real protection.

Port security, DHCP snooping, and Dynamic ARP Inspection harden the switch against those sniffing setups.

The exam synthesis: packet analysis lets defenders see the attacks taught in this chapter, and replacing cleartext protocols with TLS/SSH/IPsec removes the sniffable, injectable traffic that sniffing, MITM, and session hijacking all depend on. In short, the analyst's two jobs are to see (capture and filter intelligently) and to deny visibility to attackers (encrypt everything and disable legacy cleartext services).

For the exam, anchor the cleartext-to-secure protocol swaps cold: Telnet (23) → SSH (22) , FTP (21) → SFTP/FTPS , HTTP (80) → HTTPS (443) , POP3 (110)/IMAP (143) → POP3S (995)/IMAPS (993) , SNMPv1/v2 → SNMPv3 with authPriv , and LDAP (389) → LDAPS (636) . The lesson the CEH wants is that confidentiality comes from encryption, not obscurity: a sniffer on a switched network using ARP poisoning can still read any protocol that transmits credentials in plaintext, so encrypting the channel (TLS, SSH, IPsec) is the durable countermeasure rather than relying on the switch fabric alone.