Learning nonlinear monotone classifiers using the Choquet Integral

The learning of predictive models that guarantee a monotonic relationship between the output (response) and input (predictor) variables has received increasing attention in machine learning in recent years. While being less problematic for linear models, the difficulty of ensuring monotonicity increases with the flexibility of the underlying model class. This thesis advocates the so-called Choquet integral as a mathematical tool for learning monotone nonlinear models for classification. While being widely used as a flexible aggregation function in fields such as multiple criteria decision making, the Choquet integral is much less known in machine learning so far. Apart from combining monotonicity and flexibility in a mathematically sound and elegant manner, the Choquet integral has additional features making it attractive from a machine learning point of view. For example, it offers measures for quantifying the importance of individual predictor variables and the interaction between groups of variables, thereby supporting the interpretability of a model. Concrete methods for learning with the Choquet integral are developed on the basis of two different approaches, namely maximum likelihood estimation and structural risk minimization. While the first approach leads to a generalization of logistic regression, the second one is put into practice by means of support vector machines. In both cases, the learning problem essentially comes down to identifying the fuzzy measure on which the Choquet integral is defined. Since this measure has a large number of degrees of freedom, learning the Choquet integral is critical not only from a complexity point of view but also with regard to proper generalization. Therefore, both methods are analyzed theoretically, and different approaches to regularization and complexity reduction are proposed. Experimental results conducted on a set of suitable benchmark data are quite promising and suggest that the combination of monotonicity and flexibility offered by the Choquet integral facilitates strong performance in practical applications.