Abstract
The ease of collection and the increasing availability of large data stores has led to demands for improved methods for analyzing these data and deriving significant knowledge that maybe latent in these data stores. In particular there is hope that the use of new analytical techniques in connection with "data trawling", or data mining operations may reveal hidden relationships that lie buried within these data sets. This research investigates various techniques for the task of discovering relevant features and inference rules from data sets. Following the three steps of a knowledge extraction process, namely pre-processing (feature selection), rule discovery process, and post-processing (rule refinement), the research attempts to address some current difficulties in these three steps and introduces and integrates a °market trading' technique with existing techniques from the field of knowledge discovery and refinement with respect to data mining.
In connection with the pre-processing, a feature selection approach that employs neural networks is presented, and three associated pruning schemes that make automatic selection of the pruning threshold are proposed. The proposed neural network techniques are evaluated and compared with the x2-statistic-based discretization algorithm, called Chi2, by experimenting with six practical applications.
The Chi2 algorithm is investigated as a technique for solving problems in intelligent spatial information systems and fuzzy systems. The case studies show that the Chi2-based spatial data filtering can successfully reduce the number of spatial data items and the number of features, and therefore neural network computation can be efficiently performed. A novel approach of employing the Chi2 algorithm to select membership functions for fuzzy systems is proposed. In connection with the applications of fuzzy neural networks (FuNN models), three experimental examinations are demonstrated that an automatic selection of the number and widths of the membership functions by the Chi2-based membership function selection method can lead to the improvement of the generalization ability of FuNN fuzzy neural networks.
In connection with the rule discovery and refinement process, a novel market-based rule learning (MBRL) system is developed and its capability of evolving and refining rules is investigated. a classifier system-inspired model, it introduces a novel element by importing existing rule sets generated by other rule extraction techniques into the system. This basic change not only makes the RL system begin with pre-established rule sets with a relatively limited complexity, rather than a random set, but also enhances the likelihood of being able to interpret the evolved rules.
Moreover, the MBRL system produces various modifications in each of the layers of the structure. With the modifications introduced by the MBRL system, the problems existing in current classifier systems can be solved or lessened.
In this research, the MBRL system is proposed as a post-processing tool to be used with fuzzy neural networks (FuNN models) and the fuzzy neural network rule extraction technique, ReFuNN, in order to provide a general framework for fuzzy inference-based rule discovery. Similarly, as a post-processing tool, the MBRL system is also proposed to be used with feedforward neural networks, and the feed-forward neural network rule extraction technique, NeuroLinear, in order to improve the quality of extracted rules from feed-forward neural networks. The experimental results show that the MBRL system is a potentially useful additional tool that can be used to refine (fuzzy) neural network extracted rules and possibly discover and add some new, better performance rules. As a result, it can lead to improved performance by increasing the accuracy of the rule inference performance and/or improving the comprehensibility of the rules.
By illustrating how the MBRL system succeeded in finding solutions for six learning examples from scratch, the MBRL system is shown to have potential as an alternative generic learning technique that can be used to complement, or be used as an alternative to, conventional connectionist models to accomplish complex computational tasks.