Projects / Network DFIR

Lateral Movement Reconstructor

A bounded-memory DFIR pipeline that reconstructs lateral movement from network captures and summarizes findings locally.

Python Zeek MITRE ATT&CK NetworkX Qwen2.5 Local AI Lateral movement Active Directory Kerberoasting WMI WinRM RDP PsExec ChatML Streaming detection

Project abstract

Problem

Lateral movement is buried among large volumes of normal network traffic. Traditional capture analysis can also exhaust memory when telemetry is retained in large in-memory structures.

What I built

LMR is an asynchronous DFIR pipeline that streams Zeek-derived telemetry through bounded detection state, reconstructs a directed multigraph, and prepares evidence-backed findings for local incident summarization.

Why it matters

NetworkX graph metrics identify pivots, Patient Zero candidates, and target hubs while rolling eviction and context compression keep the pipeline usable on constrained hardware.

How it works

Capture
.pcap/.pcapng
Zeek decoding
Lazy TSV/JSON stream
Detection modules
Confidence scoring
NetworkX graph
Reports + local AI

Core architecture

Asynchronous ingestion and parsing

Zeek processes raw PCAP files into structured TSV/JSON logs. Custom Python generators, including itertools.chain, yield records sequentially instead of loading the full capture into memory.

Streaming detection

Pure Python detectors use lazy evaluation, type coercion, canonicalization, static signatures, and velocity thresholds.

Graph reconstruction

NetworkX models hosts and discrete attack paths in a directed multigraph, then calculates traversal and centrality metrics.

Reporting and visualization

D3.js and Chart.js render an interactive, client-side dashboard through Tailwind CSS; JSON, CSV, HTML, and Tshark filters preserve analyst artifacts.

Local AI inference

Qwen2.5-Coder-1.5B-Instruct runs locally for graph-grounded incident summarization after QLoRA adaptation. Only compressed topographical data is presented to the model; raw PCAP and raw JSON telemetry are not passed as the primary context.

Hardware-focused design

Intel Core i7Processor
16 GBSystem RAM
6 GBRTX 3060 VRAM
QLoRALocal model adaptation

Detection coverage and evasion resilience

TechniqueATT&CK IDEvidence source
SSH brute forceT1021.004 / T1110Zeek network telemetry and connection metadata
Linux persistenceT1053.003Zeek network telemetry and host context
Linux privilege escalationT1548.003Network evidence correlated with host context
PsExecT1021.002 / T1569.002Zeek SMB/RPC telemetry
SMB admin sharesT1021.002Zeek SMB telemetry
WMI/DCOMT1047Zeek DCE/RPC and network telemetry
WinRMT1021.006Zeek HTTP and network telemetry
RDPT1021.001TLS/SSL handshakes, native RDP cookies, and connection metadata
KerberoastingT1558.003Zeek Kerberos and network telemetry
NTLM spraying and pass-the-hashT1110.003 / T1550.002Zeek SMB, authentication and network telemetry
  • RDP: inspects raw TLS/SSL handshake data and native RDP cookies. The parser strips invisible Unicode characters, including U+200B and U+202E, and treats oversized values cautiously.
  • SMB, WMI, and WinRM: tracks administrative hidden-share mounting such as C$ and ADMIN$, remote execution UUIDs, recursive forward/backward slash variants, and null-byte path input.
  • Kerberos and NTLM: tracks high-volume TGS service sweeps associated with AES Kerberoasting and credential-spraying thresholds per source IP. Weak-cipher signatures include RC4 and DES.
  • Canonicalization: normalizes URL-encoded and Unicode-obfuscated text before matching, reducing the impact of Zalgo-style text, zero-width characters, and directional overrides.
  • Multi-stage correlation: records distinct tactics from each source IP over time. When the source shows a tactical pivot, confidence is escalated so isolated alerts are not treated like related compromise activity.

Mathematical foundations

Rolling FIFO state eviction
St+1 = { St∪{new state}if |St|≤10,000 (St−{1,000 oldest states})∪{new state}otherwise

The bounded window keeps active signatures stateful without retaining the full capture history.

Directed attack graph
G=(V,E) E⊆ { (IPu,IPv,t,w) }

Each vertex is a host IP; each edge records a source, destination, ATT&CK technique t, and observed frequency weight w.

Patient Zero
P0 = { v∈V ∣ deg⁻(v)=0 ∧ deg⁺(v)>0 }

These vertices initiate observed lateral movement without preceding incoming compromise telemetry in the analyzed graph.

Target hubs
Htarget = argmax v∈V deg⁻(v)

This identifies the vertex with the maximum incoming degree, highlighting frequently targeted assets such as domain controllers.

Multi-tactic confidence
C(v)= { Mediumif |T(v)|=1 Highif |T(v)|=2 Criticalif |T(v)|≥3

T(v) is the set of unique ATT&CK tactics observed for source v.

AI context compression

Graph edges are sorted by confidence weight and compressed into a dense string array. The engine then applies strict ChatML role tokens so the quantized local model receives a bounded topographical summary instead of raw JSON telemetry.

Design decisions

  • Zero-config input: only a capture path is required; protocols are identified by Zeek, not assumed from port numbers.
  • Proof for every finding: matched conditions, Zeek connection IDs, Wireshark filter, and ATT&CK mapping are retained.
  • Capture format detection: .pcap and .pcapng inputs are detected by file signature rather than extension.
  • Cross-platform use: Windows, macOS, and Linux are supported; Zeek runs in Docker when it is not installed locally.
Streaming deduplication

Alerts are deduplicated while findings are processed to prevent repeated evidence from creating alert floods. Measured deduplication results will be added only when available.

Engineering quality

  • Automated tests with pytest, static typing with mypy, and linting with ruff run on every push through GitHub Actions. View live CI.
  • Adversarial test cases cover obfuscated input, malformed data, and injection payloads.

Local AI context optimization

The local model runs with only 6 GB of VRAM. The reporting layer therefore converts the graph into a weighted, dense string representation rather than serializing raw network events.

  1. Rank paths: sort graph edges by confidence weight.
  2. Compress context: retain source, destination, technique, confidence, and relevant frequency.
  3. Format roles: wrap the summary in exact Qwen2.5 ChatML tokens.
  4. Generate locally: produce an executive incident summary without sending telemetry to a remote service.
<|im_start|>system
LMR incident-analysis instructions
<|im_end|>
<|im_start|>user
Ranked lateral-movement graph summary
<|im_end|>
<|im_start|>assistant

System execution flow

01Raw capture

PCAP or network traffic enters the pipeline.

02Zeek decoding

Protocol logs such as RDP and SMB are generated in TSV/JSON form.

03Lazy demultiplexer

Records are flattened and invalid schemas are dropped before detection.

04Detection engine

Static signatures and velocity thresholds evaluate canonicalized network events.

05Confidence scoring

Tactics per source are tracked and multi-stage pivoting escalates confidence.

06Concurrent outputs

NetworkX feeds D3.js, exports feed JSON/CSV/Tshark artifacts, and graph optimization feeds local Qwen2.5 inference.

What I found and fixed

Stable filter lookup

Problem: A filter lookup matched on a title string that no longer matched what a detector produced, so Kerberoasting findings silently lost their specific filter.

Fix: Matching now uses stable rule IDs, with a contract test to enforce the relationship.

Lesson: Display labels should not become internal identifiers.

Capture text escaping

Problem: The report inserted capture-derived text into the page unescaped, creating a self-XSS risk in a forensics tool.

Fix: Capture-derived values are escaped server-side before report rendering.

Lesson: Forensic evidence remains untrusted application input.

Honest limitations

  • Encrypted traffic, including SMB3 and WinRM, limits visibility; detection falls back to metadata.
  • Validation results in progress.
  • Detections are heuristic and need analyst review.
  • Rolling FIFO eviction bounds state memory but can discard older signatures during very long capture analysis.

How it was built

I designed the system around constrained-hardware execution: Python generators replace large buffered loads, Zeek handles protocol decoding, and a bounded signature tracker limits retained state. I separated detection from graph generation and rendering, then connected D3.js output to the command-line reports. I adapted Qwen2.5-Coder-1.5B-Instruct with QLoRA and built a ChatML compression layer so the graph summary fits local inference constraints.

Skills demonstrated
  • DFIR and network forensics
  • MITRE ATT&CK mapping
  • Detection engineering
  • Python packaging and CLI design
  • NetworkX graph analysis
  • Local LLM inference and ChatML formatting
  • CI and secure handling of untrusted data

Explore the implementation or review the case.