HeteroScan is a post-infection malware behavioral analysis system built for academic research. It combines three technologies: dynamic sandbox analysis via CAPE, heterogeneous graph neural networks (HAN), and large language model explanation via Claude. It addresses two key gaps in existing malware analysis research — the representation gap and the ground truth gap.
Analysis Pipeline
📁
Sample Upload
PE executable submitted
🧪
CAPE Sandbox
Isolated Windows 10 VM execution
🕸️
Graph Builder
Heterogeneous behavioral graph
🧠
HAN Model
Graph attention network inference
🤖
LLM Layer
Natural language explanation
Heterogeneous Graph Node Types
⚙️
Process Nodes
Each process observed during execution. Spawns edges connect parent to child processes.
📞
API Call Nodes
Windows API functions called by processes. The primary behavioral signal for detection.
📄
File Nodes
Files read or written during execution. Persistence mechanisms often involve file system changes.
🔑
Registry Nodes
Registry keys accessed. Malware commonly writes to Run keys for persistence.
Detected via NtProtectVirtualMemory + LdrLoadDll API call patterns in the behavioral graph
T1547
Boot/Logon Autostart
Detected via HKCU\Run and HKLM\Run registry write edges in the graph
T1059
Command & Scripting
Detected via cmd.exe, powershell.exe process spawn edges
T1071
C2 Communication
Detected via network connection edges to non-standard ports and unknown IPs
Research Context
This system is part of a master's thesis at the University of North Dakota titled "Malware Detection and Behavioral Analysis in IoT Environments: A Multi-Study Deep Learning Approach." Study 3 specifically addresses post-infection behavioral analysis using heterogeneous GNNs. The model is trained on 600+ samples across 20+ malware families including AsyncRAT, XWorm, njRAT, RedLine, AgentTesla, Vidar, Formbook, and more.
❓Help & Documentation
How to Analyze a Sample
1. Drag and drop a Windows executable (.exe, .dll) onto the upload zone, or click Browse File to select one from your computer.
2. Click Analyze Sample to submit it to the CAPE sandbox. The file executes in an isolated Windows 10 virtual machine.
3. Wait 3–5 minutes for sandbox execution and behavioral report generation.
4. HeteroScan automatically builds a heterogeneous behavioral graph, runs ensemble classification, and generates an LLM explanation.
5. Review the verdict, behavioral graph visualization, graph statistics, network connections, and AI-generated analysis report.
Pre-Analyzed Examples
Click any quick-load button to instantly view a pre-analyzed malware report:
njRAT — Remote access trojan with process injection and C2 beaconing on non-standard ports
AsyncRAT — .NET-based RAT with fileless execution, .NET framework enumeration, and port 777 C2
XWorm — Info-stealer with USB propagation, mutex creation, and encrypted C2 communication
RedLine — Credential stealer targeting browsers, crypto wallets, and system information
Understanding the Verdict
The MALICIOUS / BENIGN verdict is produced by a Voting Ensemble of Random Forest, Extra Trees, and XGBoost classifiers trained on 55 graph-derived structural features from 2,516 behavioral graphs across 37 malware families. The confidence score reflects ensemble agreement — scores above 90% indicate strong behavioral evidence.
The ensemble classifier operates on the structural properties of the behavioral graph, not on signatures or hashes. This means it can detect previously unseen malware variants that exhibit known behavioral patterns.
Graph Visualization Guide
The behavioral graph shows relationships between entities observed during sandbox execution:
🟠 Orange = Process nodes
🔵 Blue = API call nodes
🟢 Green = File nodes
🟣 Purple = Registry key nodes
🔴 Red = Network endpoint nodes
Node size reflects the count of that entity type. Edges between the central process node and surrounding nodes represent behavioral relationships.
Safety & Privacy
⚠️ HeteroScan executes uploaded files in an isolated sandbox environment with no internet access. Do not upload files containing sensitive personal information. All execution happens in a network-isolated Windows 10 VM. Uploaded samples are used solely for behavioral analysis and are not shared externally.