Inductive learning is essentially learning by example. The process itself ideally implies some method for drawing conclusions about previously unseen examples once learning is complete. More formally, one might state: Given a set of training examples, develop a hypothesis that is as consistent as possible with the provided data [1]. It is worthy of note that this is an imperfect technique. As Chalmers points out, "an inductive inference with true premises [can] lead to false conclusions" [2]. The example set may be an incomplete representation of the true population, or correct but inappropriate rules may be derived which apply only to the example set.
A simple demonstration of this type of learning is to consider the following set of bit-strings (each digit can only take on the value 0 or 1), each noted as either a positive or negative example of some concept. The task is to infer from this data (or "induce") a rule to account for the given classification:
- |
1000101 |
|
|
- |
1110100 |
|
|
+ |
0101 |
+ |
1111 |
|
|
+ |
10010 |
|
|
+ |
1100110 |
- |
100 |
|
|
+ |
111111 |
|
|
- |
00010 |
- |
1 |
|
|
- |
1101 |
|
|
+ |
101101 |
+ |
1010011 |
|
|
- |
11111 |
|
|
- |
001011 |
A rule one could induce from this data is that strings with an even number of 1's are "+", those with an odd number of 1's are "-". Note that this rule would indeed allow us to classify previously unseen strings (i.e. 1001 is "+").
Techniques for modeling the inductive learning process include: Quinlan's decision trees (results from information theory are used to partition data based on maximizing "information content" of a given sub-classification) [3], connectionism (most neural network models rely on training techniques that seek to infer a relationship from examples) and decision list techniques [4], among others.
References
- Adapted from lectures in a graduate course in representation & reasoning given by Dr. Peter van Beek, Department of Computing Science, University of Alberta.
- A.F. Chalmers. What is this thing called science?. University of Queensland Press, Australia, 1976.
- J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, San Mateo, 1993.
- R.L. Rivest. Learning decision lists. Machine Learning. 2(3):229-246, 1987.