1、Automatically Generating Models for Botnet Detection,Presenter: 葉倚任Authors: Peter Wurzinger, Leyla Bilge, Thorsten Holz, Jan Goebel, Christopher Kruegel, Engin KirdaEuropean Symposium on Research in Computer Security (ESORICS09),Outline,Introduction System Overview Model Generation Data Generating D
2、etection Models Evaluation Conclusion,Introduction,Two main kinds of network-based detection system Vertical correlation technique Detection of individual bots Checking traffic patterns, content of C&C traffic, and bot related activities. Require prior knowledge of C&C channels and propagation vecto
3、rs of bot Horizontal correlation technique Detection of a group of bots Based on network traffic Require that at two bots in the monitor networks,Introduction (contd),Characteristic behavior of a bot Receive commands from botmater Carry out some actions in response to these commands This paper propo
4、sed a two-stage detection model to leverage these two characteristics In the experiments, the authors generated 18 different bot families. 16 controlled via IRC, One via HTTP (Kraken) One via a peer-to-peer network (Storm Worm).,System Overview,Input of the system A collection of bot binaries Launch
5、 a bot in a controlled environment and record its network activities (traces) Identify the commands that this bot receives as well as its corresponding responses Translate observations into detection models Output of the system Detection models for different bot families,Detecting Procedure,Stateful
6、 model (two-stage detection) Checking if a bot command is sent If yes in stage 1, checking if the responses is above a threshold or not (e.g., the number of new connections opened by a host) Use content-based specifications to model commands (comparable to intrusion detection signatures) Use network
7、-based specifications to model responses (comparable to anomaly detection),Model Generation Data,Run each bot binary for a period of several days Locating bot responses Finding commands Extracting model generation data,Locating bot responses,Assumption: bot responses that lead to a change in network
8、 behavior Partition network traffic into consecutive time intervals of equal length For each time interval, define 8 normalized features (called traffic profile):,Locating bot responses (contd),Convert the traffic profiles (vectors) into time series data d(t) as follows:where is the sliding window s
9、ize Locate bot responses by using CUSUM algorithm = 5 and an interval of 50 seconds delivered the best results in the tests,Finding bot commands,After locating bot responses, a small section of network traffic (snippet) is extracted for each response Cluster those traffic snippets that lead to simil
10、ar responses,Extracting model generation data,Extract two pieces of information the subsequent model generation step A snippet Contains 90 seconds of traffic Plus last 30 seconds of the previous one and first 10 seconds of the following one A snippet Average of the traffic profile vectors This perio
11、d is the time from the start of the current response to the next change in behavior,Generating Detection Models,Command model generation Response model generation,Command model generation,The goal is to identify common elements in a particular behavior cluster First, apply a second clustering refine
12、ment step that groups similar network packet payloads within each behavior cluster The longest common subsequence algorithm is applied to each set of similar payloads Generate one token sequence per set,Response model generation,Compute the element-wise average of the individual behavior profiles fo
13、r a behavior cluster Give minimal bounds for certain network features 1,000 for UDP packets 100 for HTTP packets 10 for SMTP packets 20 for different IPs A detection model is not generated if a response profile exceeds none of these thresholds,Evaluation,Collected a set of 416 different (based on MD
14、5 hash) bot samples From Anubis The collection period was more than 8 months Each bot produce a traffic trace with a length of five days Divided into families of bots 16 different IRC bot families (with 356 traffic traces) One HTTP bot family (with 60 traffic traces) One p2p bot family (Storm Worm,
15、with 30 traffic traces),Detection Capability,Split our set of 446 network traces into training sets and test sets Each training set contained 25% of one bot familys traces This procedure was performed four times per family (four-fold cross validation),Real-World Deployment,Deployed a sensor In front
16、 of the residential homes of RWTH Aachen University At a Greek university network The total traffic is in the order of 94 billion network packets over a period of over three months at two different sites in Europe,Real-World Deployment,In the Greek network, most cases were false positives. BotHunter
17、 w/o Blacklist means BotHunter without blacklists of known DNS names and IP addresses The detection rate of BotHunter w/o Blacklist in the detection capacity experiment drops to 39%,Conclusion,This paper proposed a two-stage detection method which included a command model and a response model Automatically derives signatures for the bot commands and network-level specifications for the bot responses Can generate models for IRC bots, HTTP bots, and even P2P bots such as Storm,
