Abstract
Hierarchical representations of temporal sequences are central to the way we process the world around us. Consider the example of a child reciting a nursery rhyme. The song is composed of multiple phrases, each of which is composed of multiple words, each in turn composed of a series of articulated phonemes. The child must remember which of these elements occurs where and in what order. Depending on the song, phonemes, words, phrases, and even verses may reappear in different contexts within the piece. Each of these elements is a chunk of the complete sequence. How sequence chunks are learned and then compiled hierarchically into high-level structures within the brain is still an open question. This question is the central focus of this thesis. We present two primary contributions with the aim of answering this question. Our first model is a sequence chunker inspired by Jeff Hawkins’ hierarchical temporal memory model. It is designed to encode commonly observed sequences as declaratively represented chunks, and thereafter support sequential execution of items in these learned chunks. Our second model provides a method for combining this sequence chunker with reinforcement learning, enabling the formation of hierarchical sequence representations through the interoperation of these two methods. We draw on research on the basal ganglia, especially Ann Graybiel’s work covering the striatum. In particular, we explore the natural flow between the action-outcome learning of the dorsomedial striatum to the stimulus-response learning of the dorsolateral striatum. We demonstrate how the combination of these two forms of learning can lead to a smooth transition from low-level sequence chunks to deep hierarchical representations of sequences. Our model builds on recent work in neuroscience and makes novel experimental predictions which warrant further investigation.