A Multilayer Perceptron Artificial Neural Networks Based a Preprocessing and Hybrid Optimization Task for Data Mining and Classification

Kais Ncibi¹, Tarek Sadraoui^2, , Mili Faycel² and Amor Djenina³

¹University of Economics and Management of Sfax, MODILIS Lab

²University of Economics and Management of Mahdia, MODILIS Lab

³University of Tebessa Algerie

Cite this paper:
Kais Ncibi, Tarek Sadraoui, Mili Faycel and Amor Djenina. A Multilayer Perceptron Artificial Neural Networks Based a Preprocessing and Hybrid Optimization Task for Data Mining and Classification. International Journal of Econometrics and Financial Management. 2017; 5(1):12-21. doi: 10.12691/ijefm-5-1-3

Abstract

Artificial neural networks (ANNs) optimization represent an attractive area that attract many researchers in different disciplines, this in the aim to improve the performance of this model. In literature, there is no fix theory that illustrates how to construct this non linear model. Thus, all proposed construction was based on empirical illustration. Multilayer perceptron (MLP) is one of the most used models in ANNs area. It was described as a good non linear approximator with a power ability to lean well non linear system, and most of research was limited to a 3 layers MLP, by describing that 3 layers are sufficient to have good approximation. In this context we are interested to this model construction for solving supervised classification tasks in data mining. This construction requires a preprocessing phase that seems to scribe be important for the final performance. This paper present a process of MLP construction based on two phases: a preparation phase and an optimization phase. The first one describes a process of data cleaning, discretization, normalization, expansion, reduction and features selection. The second phase aims to optimize the set of weights based on some combination of hybrid algorithms such back-propagation algorithm, a local search and different evolution. An empirical illustration will be done to in order to validate the proposed model. At the end, a comparison with others known classifiers will be done to justify the validity of the proposed model.

Keywords:
multilayer perceptron data mining mutual information function expansion data transforation

This work is licensed under a Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/

References:

[1]	R. Agrawal, T. Imielinski and A. Swami. “Database mining: A performance perspective”. IEEE Trans. Knowledge Data Eng., 5, pp. 914-925 (1993).
[2]	U. M. Fayyad, Piatetsky-Shapiro, G., and Smyth, P.. “From data mining to knowledge discovery: An overview”. In U. M. Fayyad, G. Piatetsky-Shapiro, & P. Smyth (Eds.), Advances in knowledge discovery and data mining ,pp. 1–34,. Menlo Park, CA: AAAI Press, 1996.
[3]	H.P. Kriegel, et al.. “Future trends in data mining”. Data Mining and Knowledge Discovery, 15(1), 87–97. Netherlands: Springer (2007).
[4]	M., James. “Classification Algorithms”. Wiley, 1985.
[5]	L. Breiman,, J.H. Friedman, R.A. Olshen and C.J. Stone. “Classification and Regression Learning”. Morgan Kaufman, 1984.
[6]	H.P. Kriegel, et al.. “Future trends in data mining”. Data Mining and Knowledge Discovery, 15(1), 87-97. Netherlands: Springer (2007).
[7]	R.O. Duda, Hart PE and Stork D.G.. “Pattern classification”. Wiley, New York, 2001.
[8]	D.E. Goldberg. “Genetic algorithms in search, optimization and machine learning”. Morgan Kaufmann, 1989.
[9]	G. P., Zhang. ”Neural networks for classification”, A survey. IEEE Transactions on Systems, Man, Cybernetics-Part C: Application and Reviews, 30(4), pp. 451-461, 2000.
[10]	G. P Zhang. “Avoiding pitfalls in neural network research”. IEEE Transactions on Systems, Man and Cybernetics-Part C: Applications and Reviews, 37(1), pp. 3-16, 2007.
[11]	Feraud, R. and CLEROT, F.. A Methodology to Explain Neural NetworkClassification. Neural Networks, 15, 2002, 237-246.
[12]	Sarle, W.S.: How to measure the importance of inputs?. Technical report, SAS Institute Ins, Cary, NC, USA. ftp://ftp.sas.com/pub/neural/FAQ.html,1998.
[13]	Dreo, J., Petrowski, A., Siarry, P., Tailard, E.. Metaheuristic for Hard Optimization: Methods and Case Studies. Springer, Heidelberg, 2005.
[14]	Craven M. W. and Shavlik J.W.. Extracting Tree structured representation of trained neural network, Neural information processing systems, (8), 24-30 (1996).
[15]	Kohavi R., and G. John (1997). Wrappers for feature subset selection, Artificial Intelligence, vol. 97, no. 1-2, pp. 273-324.
[16]	L., Fogel, J. Owens and J. Walsh, “Artificial Intelligence through Simulated Evolution”. John Wiley, Chichester, 1966.
[17]	D.E. Goldberg. “Genetic algorithms in search, optimization and machine learning”. Morgan Kaufmann, 1989.
[18]	J., Holland. “Adaptation in natural and artificial systems”. Univ. of Michigan Press, Ann Arbor, 1975.
[19]	J. Koza,. “Genetic programming on the programming of computers by means of natural selection”, Cambridge MA: MIT Press, 1992.
[20]	M. Dorigo,, Stutzle, T., ”Ant Colony Optimization”. MIT Press, Cambridge., 2004.
[21]	J., Kennedy Eberhart R., “Particle Swarm Optimization”, In Proceedings of IEEE International Conference on Neural Networks, pp. 1942-1948, 1995.
[22]	R. Storn Kenneth P. “Differential evolution A simple and efficient adaptive scheme for global optimization over continuous spaces”. Technical Report TR, pp. 95-012, International Computer Science Institute, Berkeley, CA, 1995.
[23]	F. Hui-Y. and J. Lampinen,“A trigonometric mutation approach to differential evolution”. In K. C. Giannakoglou, D. T. Tsahalis, J. Periaux, K. D. Papailiou, & T. Fogarty, editors, Evolutionary Methods for Design Optimization and Control with Applications to Industrial Problems, pp. 65–70, Athens, Greece. International Center for Numerical Methods in Engineering (Cmine), 2001.
[24]	Y.-H. Pao, S.M. Phillips and D.J. Sobajic, “Neural-net computing and intelligent control systems”. Int. J. Contr., 56, pp. 263-289, 1992.
[25]	R. Majhi, G. Panda and G. Sahoo, “Development and performance evaluation of FLANN based model for forecasting of stock markets”, Expert Systems with Applications 36, pp. 6800-6808, 2009.
[26]	C.L. Blake,. and Merz, C.J., ‘UCI Repository of machine learning databases’, em Irvine, CA: University of California, department of information and Computer Science. Available on-line at: http//www.ics.uci.edu/~mlearn/ML.Repository.html., 1998.