Anomaly detection for Intrusion Detection

Opportunity

• Unlike Signature based techniques, Unsupervised Data Mining techniques can detect new/novel attacks
• Data Mining techniques can be used in conjunction with Signature based Systems such as SNORT to handle its blindspot
  
• SNORT is rules-based. If its a new attack???

Feasibility

• Unsupervised Data Mining techniques are in their infancy
• Cannot be effectively used in real-time (only offline analysis)
• Large number of False Alerts
  
• Some require attack-free data for training
• Still opportunity exists
• Clustering reduces the scale of data - from millions of records to a few hundred clusters. For each Cluster, you have a prototypical instance or a centroid that other elements are similar to. Can this be used to detect a Distributed DoS attack?
  
• Outlier detection
  
• Can be a pre-processing step used by Security companies to analyzing attack data in order to right rules in SNORT

Whacky ideas

• Mine CERT alerts to determine to automatically build SNORT rules?
• Build a search engine for Security alerts? (Like a Lexis Nexus but tailored for the domain of Security)