Deep architectures and classification by intermediary transformations

Abstract

With the development of deep belief nets, the empirical evidence supporting a link between deep architecture neural networks and generalisation with respect to classification has been mounting. An analytical proof of this relation would have an immense impact on machine learning and classification, yet it has not been forthcoming due to the limited understanding of what constitutes an appropriate internal data representation in a multi-layer neural network model that would be conducive to good classification. This work proposes a theory of intermediary transformations, which establishes an objective for an individual layer in a deep architecture classifier that improves data separability according to its class labels. A training algorithm, based on the new theory, for a multi-layer neural network is proposed and evaluated against traditional backpropagation and deep belief net learning. The results confirm that a supervised classification training objective for an individual hidden layer is viable and generalises well. Classification by intermediary transformations offers new directions and insights in the quest to illuminate the black box model of deep architectures.