Comparing the Fault Diagnosis Performances of Single Neural Networks and Two Ensemble Neural Networks Based on the Boosting Methods

Pooria Karimi^1, and Hooshang Jazayeri-Rad¹

¹Department of Instrumentation and Automation, Petroleum University of Technology, Ahwaz, Iran

Cite this paper:
Pooria Karimi and Hooshang Jazayeri-Rad. Comparing the Fault Diagnosis Performances of Single Neural Networks and Two Ensemble Neural Networks Based on the Boosting Methods. Journal of Automation and Control. 2014; 2(1):21-32. doi: 10.12691/automation-2-1-4

Abstract

This work employs potential and distinct features of artificial neural networks, in particular the feed-forward multilayer structure, to achieve fault diagnosis in chemical plants. Artificial neural networks can automatically store information by learning from historical fault data without using any qualitative or quantitative model of the system. However, different limitations are encountered in utilizing a single neural network classifier in fault diagnosis of complex large-scale chemical processes. These limitations are mostly originated from the low reliability and also high generalization error of a single neural network classifier which may be faced in real applications. An attractive remedy for these issues is to use an ensemble of different neural network models instead of relying on a single one. In this work, boosting as a general well-known approach to improve the learning performance of learning algorithms by building ensemble models is investigated. In particular, an efficient boosting method called the Stage-wise Additive Modeling using a Multi-class Exponential Loss Function (SAMMELF) is utilized. This technique can effectively overcome the main limitations of the early boosting methods for fault diagnosis applications. The potential of the SAMMELF algorithm is proved through its application to the challenging diagnosis problem of the Tennessee-Eastman Process (TEP) where the whole sets of the predefined TEP faults are considered in the classification problem.

Keywords:
artificial neural network process fault diagnosis adaptive boosting Tennessee-Eastman Process

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

References:

[1]	Himmelblau, D.M., Fault detection and diagnosis in chemical and petrochemical processes, Elsevier Scientific Publishing Company, Amsterdam, 1978.
[2]	Chiang, L.H., Braatz, R.D. and Russell, E.L., Fault detection and diagnosis in industrial systems, Springer, New York, 2001.
[3]	Venkatasubramanian, V., Rengaswamy, Yin, R.K. and Kavuri, S.N., “A review of process fault detection and diagnosis: Part I: Quantitative model-based methods,” Computers & chemical engineering, 27 (3). 293-311. 2003.
[4]	Isermann, R., “Process fault detection based on modeling and estimation methods-a survey,” Automatica, 20 (4). 387-404. 1984.
[5]	Dash, S. and Venkatasubramanian, V., “Challenges in the industrial applications of fault diagnostic systems,” Computers & chemical engineering, 24 (2). 785-791. 2000.
[6]	Kresta, J.V., Macgregor, J.F. and Marlin, T.E., “Multivariate statistical monitoring of process operating performance,” The Canadian journal of chemical engineering, 69 (1). 35-47. 1991.
[7]	He, Q.P. and Wang, J., “Fault detection using the k-nearest neighbor rule for semiconductor manufacturing processes,” Semiconductor manufacturing, IEEE transactions on, 20 (4). 345-354. 2007.
[8]	Chiang, L.H., Kotanchek, M.E. and Kordon, A.K., “Fault diagnosis based on Fisher discriminant analysis and support vector machines,” Computers & chemical engineering, 28 (8). 1389-1401. 2004.
[9]	Verron, S., Tiplica, T. and Kobi, A., “Bayesian networks and mutual information for fault diagnosis of industrial systems,” in Workshop on Advanced Control and Diagnosis (ACD’06), 2006.
[10]	Verron, S., Tiplica, T. and Kobi, A., “Fault diagnosis of industrial systems by conditional Gaussian network including a distance rejection criterion,” Engineering applications of artificial intelligence, 23 (7). 1229-1235. 2010.
[11]	Chen, K.Y., Chen, L.S., Chen, M.C. and Lee, C.L., “Using SVM based method for equipment fault detection in a thermal power plant,” Computers in industry, 62 (1). 42-50. 2011.
[12]	Yélamos, I., Escudero, G., Graells, M. and Puigjaner, L., “Performance assessment of a novel fault diagnosis system based on support vector machines,” Computers & chemical engineering, 33 (1). 244-255. 2009.
[13]	Kulkarni, A., Jayaraman, V.K. and Kulkarni, B.D., “Knowledge incorporated support vector machines to detect faults in Tennessee Eastman process,” Computers & chemical engineering, 29 (10). 2128-2133. 2005.
[14]	Xu, J., Zhao, J., Ma, B. and Hu, S., “Fault diagnosis of complex industrial process using KICA and sparse SVM,” Mathematical problems in engineering, 2013. 1-6. 2013.
[15]	Hussain, M. A., Hassan, C.R.C., Loh, K.S. and Mah, K.W., “Application of artificial intelligence technique in process fault diagnosis,” Journal of engineering science and technology, 2 (3). 260-270. 2007.
[16]	Watanabe, K., Matsuura, I., Abe, M., Kubota, M. and Himmelblau, D.M., “Incipient fault diagnosis of chemical processes via artificial neural networks,” AIChE journal, 35 (11). 1803-1812. 1989.
[17]	Koivo, H.N., “Artificial neural networks in fault diagnosis and control,” Control engineering practice, 2 (1). 89-101. 1994.
[18]	Himmelblau, D.M., “Applications of artificial neural networks in chemical engineering,” Korean journal of chemical engineering, 17 (4). 373-392. 2000.
[19]	Sharma, R., Singh, K., Singhal, D. and Ghosh, R., “Neural network applications for detecting process faults in packed towers,” Chemical engineering and processing: process intensification, 43 (7). 841-847. 2004.
[20]	Eslamloueyan, R., “Designing a hierarchical neural network based on fuzzy clustering for fault diagnosis of the Tennessee–Eastman process,” Applied soft computing, 11 (1). 1407-1415. 2011.
[21]	Behbahani, M.R., Jazayeri-Rad, H. and Hajmirzaee, S., “Fault detection and diagnosis in a sour gas absorption column using neural networks,” Chemical engineering & technology, 32 (5). 840-845. 2009.
[22]	Hansen, L.K. and Salamon, P., “Neural network ensembles,” Pattern analysis and machine intelligence, IEEE transactions on, 12 (10). 993-1001. 1990.
[23]	Polikar, R., “Ensemble based systems in decision making,” Circuits and systems magazine, IEEE, 6 (3). 21-45. 2006.
[24]	Baxt, W.G., “Improving the accuracy of an artificial neural network using multiple differently trained networks,” Neural computation, 4 (5). 772-780. 1992.
[25]	Ali, K.M. and Pazzani, M.J., “Error reduction through learning multiple descriptions,” Machine learning, 24 (3). 173-202. 1996.
[26]	Valdovinos, R.M. and Sanchez, J. S., “Ensembles of multilayer perceptron and modular neural networks for fast and accurate learning,” in Artificial Intelligence, MICAI’06. Fifth Mexican International Conference on, 2006, 229-236.
[27]	Zhang, J., “Improved on-line process fault diagnosis through information fusion in multiple neural networks,” Computers & chemical engineering, 30 (3). 558-571. 2006.
[28]	Bauer, E. and Kohavi, R., “An empirical comparison of voting classification algorithms: bagging, boosting, and variants,” Machine learning, 36 (1). 105-139. 1999.
[29]	Zio, E., Baraldi, P. and Gola, G., “Feature-based classifier ensembles for diagnosing multiple faults in rotating machinery,” Applied soft computing, 8 (4). 1365-1380. 2008.
[30]	Xu, L., Krzyzak, A. and Suen, C.Y., “Methods of combining multiple classifiers and their applications to handwriting recognition,” Systems, man and cybernetics, IEEE transactions on, 22 (3). 418-435. 1992.
[31]	Kuncheva, L.I., Bezdek, J.C. and Duin, R.P.W., “Decision templates for multiple classifier fusion: an experimental comparison,” Pattern recognition, 34 (2), 299-314, 2001.
[32]	Ruta, D. and Gabrys, B., “An overview of classifier fusion methods,” Computing and Information systems, 7 (1). 1-10. 2000.
[33]	Freund, Y. and Schapire, R.E., “Experiments with a new boosting algorithm,” in ICML, 1996, 148-156.
[34]	Freund, Y. and Schapire, R.E., “A desicion-theoretic generalization of on-line learning and an application to boosting,” in Computational learning theory, 1995, 23-37.
[35]	Schwenk, H., and Bengio, Y., “Boosting neural networks,” Neural Computation, 12 (8). 1869-1887. 2000.
[36]	Schapire, R.E., Freund, Y., Bartlett, P. and Lee, W.S., “Boosting the margin: A new explanation for the effectiveness of voting methods,” The annals of statistics, 26 (5). 1651-1686. 1998.
[37]	Samanta, B., Bandopadhyay, S., Ganguli, R. and Dutta, S., “A comparative study of the performance of single neural network vs. adaboost algorithm based combination of multiple neural networks for mineral resource estimation,” Journal of South African institute of mining and metallurgy, 105 (4). 237-246. 2005.
[38]	Schwenk, H. and Bengio, Y., “Adaboosting neural networks: Application to on-line character recognition,” in Artificial Neural Networks-ICANN’97, Springer, 1997, 967-972.
[39]	Canty, M.J., “Boosting a fast neural network for supervised land cover classification,” Computers & geosciences, 35 (6). 1280-1295. 2009.
[40]	Zhu, J., Zou, H., Rosset, S. and Hastie, T., “Multi-class adaboost,” Statistics and Its Interface, 2. 349-360. 2009.
[41]	Downs, J.J. and Vogel, E.F., “A plant-wide industrial process control problem,” Computers & chemical engineering, 17 (3). 245-255. 1993.
[42]	Lyman, P.R. and Georgakis, C., “Plant-wide control of the Tennessee Eastman problem,” Computers & chemical engineering, 19 (3). 321-331. 1995.
[43]	Møller, M.F., “A scaled conjugate gradient algorithm for fast supervised learning,” Neural networks, 6 (4). 525-533. 1993.
[44]	Mahadevan, S. and Shah, S.L., “Fault detection and diagnosis in process data using one-class support vector machines,” Journal of process control, 19 (10). 1627-1639. 2009.