Diversity-based selection of learning algorithms: a bagging approach

Nowadays, classification problems are becoming increasingly important in many real-world applications. As the problems become more complex and the consequences of a bad decision are more serious, more advanced techniques, as the combination of classifiers, need to be applied. When combining classifiers, it is important to ensure diversity between them as it does not make sense to combine classifiers whose classification is the same. There are several techniques to ensure diversity in systems like these and generally it consider modify the data set, use different learning algorithms or make a process of improvement or learning on the individual classification. Although the relationship between diversity and system accuracy has not been fully established, it is clear that diversity remains a factor to be taken into account in the construction of ensembles of classifiers. In this paper we present a modification to the bagging algorithm to consider different learning algorithms during the training process and optimize the classifiers built to obtain diverse systems and as accurate as possible. Executed simulations suggest the use of the Double Failure pairwise measure to quantify the diversity of the system. With respect to the number of classifiers used, it was observed that the systems built had approximately half of the total classifiers they should have. After, the superiority of the proposed method with respect to five state-of-the-art ensembles of classifiers was verified and it is suggested the incorporation of a learning process like the one executed in Stacking. Finally, are shown results in biochemical real applications and the general conclusions are exposed.

Recommended citation: Cabrera-Hernández, L., Morales-Hernández, A., Meneses Gómez, M., Meneses Marcel, A., Casas Cardoso, G. M., García Lorenzo, M. M. (2021). "Diversity-based selection of learning algorithms: a bagging approach." Investigación Operacional. 42(4).
Download Paper | Download Bibtex