Outside the Closed World: On Using Machine Learning For Network Intrusion Detection by Robin Sommer and Vern Paxson.
Abstract:
In network intrusion detection research, one popular strategy for finding attacks is monitoring a network’s activity for anomalies: deviations from profiles of normality previously learned from benign traffic, typically identified using tools borrowed from the machine learning community. However, despite extensive academic research one finds a striking gap in terms of actual deployments of such systems: compared with other intrusion detection approaches, machine learning is rarely employed in operational “real world ” settings. We examine the differences between the network intrusion detection problem and other areas where machine learning regularly finds much more success. Our main claim is that the task of finding attacks is fundamentally different from these other applications, making it significantly harder for the intrusion detection community to employ machine learning effectively. We support this claim by identifying challenges particular to network intrusion detection, and provide a set of guidelines meant to strengthen future research on anomaly detection. Keywords-anomaly detection; machine learning; intrusion detection; network security.
From the introduction:
In this paper we set out to examine the differences between the intrusion detection domain and other areas where machine learning is used with more success. Our main claim is that the task of finding attacks is fundamentally different from other applications, making it significantly harder for the intrusion detection community to employ machine learning effectively. We believe that a significant part of the problem already originates in the premise, found in virtually any relevant textbook, that anomaly detection is suitable for finding novel attacks; we argue that this premise does not hold with the generality commonly implied. Rather, the strength of machine-learning tools is finding activity that is similar to something previously seen, without the need however to precisely describe that activity up front (as misuse detection must).
Between data breaches at firms that should know better and the ongoing dribble of Snowden revelations, cybersecurity will be a hot topic for years.
Beyond security concerns, the author’s isolation of machine learning as detecting something “similar to something previously seen” sets a limit on the usefulness of machine learning for detecting “new” subjects/concepts in a data stream.
A news account I saw earlier this week described machine processing of text as “objective.” A better term would have been “unimaginative.” Machine learning can return the meanings it has been taught but it will not offer a new meaning. Something to bear in mind when mining large bodies of texts.