Metasploit for Machine Learning: Deep-Pwning
From the post:
Deep-pwning is a lightweight framework for experimenting with machine learning models with the goal of evaluating their robustness against a motivated adversary.
Note that deep-pwning in its current state is no where close to maturity or completion. It is meant to be experimented with, expanded upon, and extended by you. Only then can we help it truly become the goto penetration testing toolkit for statistical machine learning models.
Metasploit for Machine Learning: Background
Researchers have found that it is surprisingly trivial to trick a machine learning model (classifier, clusterer, regressor etc.) into making an objectively wrong decisions. This field of research is called Adversarial Machine Learning. It is not hyperbole to claim that any motivated attacker can bypass any machine learning system, given enough information and time. However, this issue is often overlooked when architects and engineers design and build machine learning systems. The consequences are worrying when these systems are put into use in critical scenarios, such as in the medical, transportation, financial, or security-related fields.
Hence, when one is evaluating the efficacy of applications using machine learning, their malleability in an adversarial setting should be measured alongside the system’s precision and recall.
…
(emphasis in original)
As motivation for a deep dive into machine learning, looming reliance on machine learning to compensate for a shortage of cybersecurity defender talent is hard to beat. (Why Machine Learning will Boost Cyber Security Defenses amid Talent Shortfall)
Reducing cybersecurity to the level of machine learning is nearly as inviting as use of an older, less secure version of MINIX by Intel. If you are going to take advantage of a Berkeley software license, at least get the best stuff. Yes?
Machine learning is of growing importance, but since classifiers can be fooled into identifying a 3-D turtle as a rifle, it hasn’t reached human levels of robustness.
Or to put that differently, when was the last time you identified a turtle as a rifle?
Turtle vs. rifle is a distinction few of us would miss in language, even without additional properties, as in a topic map. But thinking of their properties or characteristics, maybe a fruitful way to understand why they can be confused.
Or even planning for their confusion and communicating that plan to others.