The Inductive Biases of Various Machine Learning Algorithms

The Inductive Biases of Various Machine Learning Algorithms by Laura Diane Hamilton.

From the post:

Every machine learning algorithm with any ability to generalize beyond the training data that it sees has, by definition, some type of inductive bias.

That is, there is some fundamental assumption or set of assumptions that the learner makes about the target function that enables it to generalize beyond the training data.

Below is a chart that shows the inductive biases for various machine learning algorithms:

Inductive reasoning has a checkered history (Hume) but is widely relied upon in machine learning.

Consider this a starter set of biases for classes of machine learning algorithms.

There may be entire monographs on the subject but I haven’t seen a treatment at length on how to manipulate data sets so they take advantage of known biases in the better known machine learning algorithms.

You could take the position that misleading data sets test the robustness of machine learning algorithms and so the principles of their generation and use have the potential to improve machine learning.

That may well be the case but I would be interested in such a treatment so that detection of such manipulation of data could be detected.

Either way, it would be an interesting effort, assuming it doesn’t exist already.

Pointers anyone?

I first saw this in a tweet by Alex Hall.

Comments are closed.